BACKGROUND: The use of generative large language models (LLMs) with electronic health record (EHR) data is rapidly expanding to support clinical and research tasks. This systematic review characterizes the clinical fields and use cases that have been studied and evaluated to date. METHODS: We followed the Preferred Reporting Items for Systematic Review and Meta-Analyses guidelines to conduct a systematic review of articles from PubMed and Web of Science published between January 1, 2023, and November 9, 2024. Studies were included if they used generative LLMs to analyze real-world EHR data and reported quantitative performance evaluations. Through data extraction, we identified clinical specialties and tasks for each included article, and summarized evaluation methods. RESULTS: Of the 18 735 articles retrieved, 196 met our criteria. Most studies focused on radiology (26.0%), oncology (10.7%), and emergency medicine (6.6%). Regarding clinical tasks, clinical decision support made up the largest proportion of studies (62.2%), while summarizations and patient communications made up the smallest, at 5.6% and 5.1%, respectively. In addition, GPT-4 and GPT-3.5 were the most commonly used generative LLMs, appearing in 60.2% and 57.7% of studies, respectively. Across these studies, we identified 22 unique non-NLP metrics and 35 unique NLP metrics. While NLP metrics offer greater scalability, none demonstrated a strong correlation with gold-standard human evaluations. CONCLUSION: Our findings highlight the need to evaluate generative LLMs on EHR data across a broader range of clinical specialties and tasks, as well as the urgent need for standardized, scalable, and clinically meaningful evaluation frameworks.

OBJECTIVE: Automated literature screening in biomedical research is often hindered by domain shifts and scarcity of labeled data, which limit model accuracy and generalizability. While large language models (LLMs) perform well in zero-shot settings, they often fail to capture complex, domain-specific reasoning patterns. To address this limitation, this study investigates whether an interactive, weakly supervised learning framework combining GPT (generative pre-trained transformer)'s fine-tuning adaptability with DeepSeek's reasoning capabilities can improve literature screening performance across biomedical domains. MATERIALS AND METHODS: We developed an active learning framework that leverages model disagreement between GPT-4o and DeepSeek to improve literature screening performance. This process began with a labeled corpus of 6331 articles on large language models, from which a model disagreement analysis was performed to identify cases where GPT-4o misclassified and DeepSeek produced correct predictions. Three GPT variants-GPT-4o, GPT-4o-mini, and GPT-4.1-nano, were fine-tuned under standard supervised learning settings using these disagreement-based samples. Fine-tuning prompts incorporated classification labels and, when available, rationale traces generated by DeepSeek to provide reasoning-augmented weak supervision. Model performance was evaluated on an independent benchmark set of 291 annotated articles across 10 topic queries in cancer immunotherapy and LLMs in medicine, using standard evaluation metrics, with recall as the primary measure. RESULTS: Fine-tuning GPT models using disagreement-based examples significantly improved performance. GPT-4o-mini achieved the best overall results after fine-tuning, especially with the highest F1 score (0.93, P < .001) and recall (0.95, P < .001). Across the biomedical topics, fine-tuned models consistently outperformed their zero-shot counterparts without increasing reviewer workload. DISCUSSION: These findings demonstrate the effectiveness of disagreement-driven active learning in enhancing GPT-based biomedical literature screening. Lightweight models like GPT-4o-mini benefit most from targeted, reasoning-enriched training, highlighting their suitability for scalable deployment. CONCLUSION: This study introduces an interactive active learning framework that leverages fine-tuned LLMs with reasoning capabilities to enhance literature screening. The approach offers a scalable solution to more efficient and reliable information retrieval in systematic reviews.

Graph learning aims to predict the label for an entire graph. Recently, graph neural network (GNN)-based approaches become an essential strand to learning low-dimensional continuous embeddings of entire graphs for graph label prediction. While GNNs explicitly aggregate the neighborhood information and implicitly capture the topological structure for graph representation, they ignore the relationships among graphs. In this article, we propose a graph-graph (G2G) similarity network to tackle the graph learning problem by constructing a SuperGraph through learning the relationships among graphs. Each node in the SuperGraph represents an input graph, and the weights of edges denote the similarity between graphs. By this means, the graph learning task is then transformed into a classical node label propagation problem. Specifically, we use an adversarial autoencoder to align embeddings of all the graphs to a prior data distribution. After the alignment, we design the G2G similarity network to learn the similarity between graphs, which functions as the adjacency matrix of the SuperGraph. By running node label propagation algorithms on the SuperGraph, we can predict the labels of graphs. Experiments on five widely used classification benchmarks and four public regression benchmarks under a fair setting demonstrate the effectiveness of our method.

We demonstrated the existence of a group algebraic structure hidden in relational knowledge embedding problems, which suggests that a group-based embedding framework is essential for designing embedding models. Our theoretical analysis explores merely the intrinsic property of the embedding problem itself hence is model independent. Motivated by the theoretical analysis, we have proposed a group theory-based knowledge graph embedding framework, in which relations are embedded as group elements, and entities are represented by vectors in group action spaces. We provide a generic recipe to construct embedding models associated with two instantiating examples: SO3E and SU2E, both of which apply a continuous non-Abelian group as the relation embedding. Empirical experiments using these two exampling models have shown state-of-the-art results on benchmark datasets.

It has been shown that deep neural network (DNN) based classifiers are vulnerable to human-imperceptive adversarial perturbations which can cause DNN classifiers to output wrong predictions with high confidence. We propose an unsupervised learning approach to detect adversarial inputs without any knowledge of attackers. Our approach tries to capture the intrinsic properties of a DNN classifier and uses them to detect adversarial inputs. The intrinsic properties used in this study are the output distributions of the hidden neurons in a DNN classifier presented with natural images. Our approach can be easily applied to any DNN classifiers or combined with other defense strategy to improve robustness. Experimental results show that our approach demonstrates state-of-the-art robustness in defending black-box and gray-box attacks.

BACKGROUND: Performing statistical tests is an important step in analyzing genome-wide datasets for detecting genomic features differentially expressed between conditions. Each type of statistical test has its own advantages in characterizing certain aspects of differences between population means and often assumes a relatively simple data distribution (e.g., Gaussian, Poisson, negative binomial, etc.), which may not be well met by the datasets of interest. Making insufficient distributional assumptions can lead to inferior results when dealing with complex differential expression patterns. RESULTS: We propose to capture differential expression information more comprehensively by integrating multiple test statistics, each of which has relatively limited capacity to summarize the observed differential expression information. This work addresses a general application scenario, in which users want to detect as many as DEFs while requiring the false discovery rate (FDR) to be lower than a cut-off. We treat each test statistic as a basic attribute, and model the detection of differentially expressed genomic features as learning a discriminant boundary in a multi-dimensional space of basic attributes. We mathematically formulated our goal as a constrained optimization problem aiming to maximize discoveries satisfying a user-defined FDR. An effective algorithm, Discriminant-Cut, has been developed to solve an instantiation of this problem. Extensive comparisons of Discriminant-Cut with 13 existing methods were carried out to demonstrate its robustness and effectiveness. CONCLUSIONS: We have developed a novel machine learning methodology for robust differential expression analysis, which can be a new avenue to significantly advance research on large-scale differential expression analysis.

Maintaining a normal burning temperature is essential to ensuring the quality of nonferrous metals and cement clinker in a rotary kiln. Recognition of the temperature condition is an important component of a temperature control system. Because of the interference of smoke and dust in the kiln, the temperature of the burning zone is difficult to be measured accurately using traditional methods. Focusing on blurry images from which only the flame region can be segmented, an image recognition system for the detection of the temperature condition in a rotary kiln is presented. First, the flame region is segmented employing a region-growing method with a dynamic seed point. Seven features, comprising three luminous features and four dynamic features, are then extracted from the flame region. Dynamic features constructed from luminous feature sequences are proposed to overcome the problem of mis-recognition when the temperature of the flame region changes rapidly. Finally, classifiers are trained to recognize the temperature state of the burning zone using its features. Experimental results using real datasets demonstrate that the proposed image-based systems for recognizing the temperature condition are effective and robust.

We studied various psychiatric attributes in the categories of depression, anxiety and insomnia on 208 patients with liver cirrhosis, 61 patients with non-cirrhosis chronic diseases, and 60 healthy people. A set of psychiatric attributes were found to be uniquely associated with cirrhosis. Literature shows that these attributes are related to some mechanisms underlying liver cirrhosis, such as zinc/selenium deficiency and vitamin D deficiency. In addition, we investigated the relationships between those psychiatric attributes unique to cirrhosis and the “four diagnostic methods” of Traditional Chinese Medicine (TCM), and discovered interesting correlations. The above two findings can shed new light on the diagnosis and prognosis of liver cirrhosis, and the integration of Traditional Chinese and Western Medicine in the future.

False discovery rate (FDR) control is widely practiced to correct for multiple comparisons in selecting statistically significant features from genome-wide datasets. In this paper, we present an advanced significance analysis method called miFDR that minimizes FDR when the number of the required significant features is fixed. We compared our approach with other well-known significance analysis approaches such as Significance Analysis of Microarrays [1-3], the Benjamini-Hochberg approach [4] and the Storey approach [5]. The results of using both simulated data sets and public microarray data sets demonstrated that miFDR is more powerful.

Tandem affinity purification coupled with mass-spectrometry (TAP/MS) analysis has been increasingly used to identify novel endogenous protein-protein interactions (PPIs) in a high-throughput manner. Computational analysis of TAP/MS data is critical, however, and remains challenging because of the noisy nature of the data. We developed an advanced method for identifying PPIs from TAP/MS data. Our approach APPIC, which stands for Advanced Protein-Protein Interactions Capturing, incorporates an improved statistical method and is more powerful than existing tools.

Recently, High-content screening (HCS) has been combined with RNA interference (RNAi) to become an essential image-based high-throughput method for studying genes and biological networks through RNAi-induced cellular phenotype analyses. However, a genome-wide RNAi-HCS screen typically generates tens of thousands of images, most of which remain uncategorized due to the inadequacies of existing HCS image analysis tools. Until now, it still requires highly trained scientists to browse a prohibitively large RNAi-HCS image database and produce only a handful of qualitative results regarding cellular morphological phenotypes. For this reason we have developed intelligent interfaces to facilitate the application of the HCS technology in biomedical research. Our new interfaces empower biologists with computational power not only to effectively and efficiently explore large-scale RNAi-HCS image databases, but also to apply their knowledge and experience to interactive mining of cellular phenotypes using Content-Based Image Retrieval (CBIR) with Relevance Feedback (RF) techniques.

Motivation: Building an accurate binding model for a transcription factor (TF) is essential to differentiate its true binding targets from those spurious ones. This is an important step toward understanding gene regulation. Results: This paper describes a boosting approach to modeling TF–DNA binding. Different from the widely used weight matrix model, which predicts TF–DNA binding based on a linear combination of position-specific contributions, our approach builds a TF binding classifier by combining a set of weight matrix based classifiers, thus yielding a non-linear binding decision rule. The proposed approach was applied to the ChIP-chip data of Saccharomyces cerevisiae. When compared with the weight matrix method, our new approach showed significant improvements on the specificity in a majority of cases. Contact: [email protected] Supplementary information: The software and the Supplementary data are available at http://biogibbs.stanford.edu/~hong2004/MotifBooster/.

BACKGROUND: Collecting and managing information is a challenging task in a genome-wide profiling research project. Most databases and online computational tools require a direct human involvement. Information and computational results are presented in various multimedia formats (e.g., text, image, PDF, word files, etc.), many of which cannot be automatically processed by computers in biologically meaningful ways. In addition, the quality of computational results is far from perfect and requires nontrivial manual examination. The timely selection, integration and interpretation of heterogeneous biological information still heavily rely on the sensibility of biologists. Biologists often feel overwhelmed by the huge amount of and the great diversity of distributed heterogeneous biological information. DESCRIPTION: We developed an information management application called GeneNotes. GeneNotes is the first application that allows users to collect and manage multimedia biological information about genes/ESTs. GeneNotes provides an integrated environment for users to surf the Internet, collect notes for genes/ESTs, and retrieve notes. GeneNotes is supported by a server that integrates gene annotations from many major databases (e.g., HGNC, MGI, etc.). GeneNotes uses the integrated gene annotations to (a) identify genes given various types of gene IDs (e.g., RefSeq ID, GenBank ID, etc.), and (b) provide quick views of genes. GeneNotes is free for academic usage. The program and the tutorials are available at: http://bayes.fas.harvard.edu/genenotes/. CONCLUSIONS: GeneNotes provides a novel human-computer interface to assist researchers to collect and manage biological information. It also provides a platform for studying how users behave when they manipulate biological information. The results of such study can lead to innovation of more intelligent human-computer interfaces that greatly shorten the cycle of biology research.

The Ubiquitous Bio-Information Computing (UBIC2) project aims to disseminate protocols and software packages to facilitate the development of heterogeneous bio-information computing units that are interoperable and may run distributedly. UBIC2 specifies biological data in XML formats and queries data using XQuery. The UBIC2 programming library provides interfaces for integrating, retrieving, and manipulating heterogeneous biological data. Interoperability is achieved via Simple Object Access Protocol (SOAP) based web services. The documents and software packages of UBIC2 are available at .

In a multimodal conversation, the way users communicate with a system depends on the available interaction channels and the situated context (e.g., conversation focus, visual feedback). These dependencies form a rich set of constraints from various perspectives such as temporal alignments between different modalities, coherence of conversation, and the domain semantics. There is strong evidence that competition and ranking of these constraints is important to achieve an optimal interpretation. Thus, we have developed an optimization approach for multimodal interpretation, particularly for interpreting multimodal references. A preliminary evaluation indicates the effectiveness of this approach, especially for complex user inputs that involve multiple referring expressions in a speech utterance and multiple gestures.

Biological information computing is rapidly advancing from homogeneous data computation to large-scale heterogeneous data computation. However, the development of data specification protocols, software middleware, and Web services, which support large-scale heterogeneous data exchange, integration, and computation, generally falls behind data expansion rates and bioinformatics demands. The ubiquitous bio-information computing (UBlC/sup 2/) project aims to disseminate software packages to assist the development of heterogeneous bio-information computing applications that are interoperable and may run distributedly. UBlC/sup 2/ lays down the software architecture for integrating, retrieving, and manipulating heterogeneous biological information so that data behave like being stored in a unified database, UBIC/sup 2/ programming library implements the software architecture and provides application programming interfaces (APIs) to facilitate the development of heterogeneous bio-information computing applications. To achieve interoperability, UBIC/sup 2/ Web services use XML-based data communication means, which allow distributed applications to consume heterogeneous bio-information regardless of platforms. The documents and software package of UBlC/sup 2/ are available at http://www.ubic2.org.

Multimodal user interfaces allow users to interact with computers through multiple modalities, such as speech, gesture, and gaze. To be effective, multimodal user interfaces must correctly identify all objects which users refer to in their inputs. To systematically resolve different types of references, we have developed a probabilistic approach that uses a graph-matching algorithm. Our approach identifies the most probable referents by optimizing the satisfaction of semantic, temporal, and contextual constraints simultaneously. Our preliminary user study results indicate that our approach can successfully resolve a wide variety of referring expressions, ranging from simple to complex and from precise to ambiguous ones.

This paper proposes an approach for automatic temporal pattern extraction from a temporal sequence, whose signal samples suffer from noise and non-linear temporal warping. The patterns are recurrent signal segments in the original temporal sequence with respect to some criteria defined by the user. We use Threshold hidden Markov model (THMM) to model the signal sequence. A variance of the Segmental K-means algorithm is used to train the THMM The trained THMM can be used to detect the pattern in the temporal sequence. We apply the proposed approach to automatically analyze multiple synchronized temporal sequences from several input modalities. The timing information of the extracted patterns is used to learn the correlation across multiple modalities, which leads to automatic concept learning.

This paper presents an exemplar-based probabilistic approach for face and facial motion tracking. It is well known that high-level knowledge about facial deformations is essential for robust face and facial motion tracking. Face and facial motion tracking problem is usually formulated as a problem of combining the low-level image information and the high-level knowledge. We propose to select only a few representative facial deformation exemplars as the high-level knowledge. A facial deformation can be approximated by a linear combination of the exemplars up to an error term. We develop a probabilistic mechanism that combines the low-level image information and the information provided by the exemplars in terms of maximum a posteriori. The main advantage of this exemplar-based approach is that it avoids manually labelling a large set of training samples, which is required by many other tracking algorithms to train a high-level knowledge model. Therefore, it can be easily set up for different subjects. Moreover, it provides a unified representation for the facial deformations of different subjects.

A real-time speech-driven synthetic talking face provides an effective multimodal communication interface in distributed collaboration environments. Nonverbal gestures such as facial expressions are important to human communication and should be considered by speech-driven face animation systems. In this paper, we present a framework that systematically addresses facial deformation modeling, automatic facial motion analysis, and real-time speech-driven face animation with expression using neural networks. Based on this framework, we learn a quantitative visual representation of the facial deformations, called the motion units (MUs). A facial deformation can be approximated by a linear combination of the MUs weighted by MU parameters (MUPs). We develop an MU-based facial motion tracking algorithm which is used to collect an audio-visual training database. Then, we construct a real-time audio-to-MUP mapping by training a set of neural networks using the collected audio-visual training database. The quantitative evaluation of the mapping shows the effectiveness of the proposed approach. Using the proposed method, we develop the functionality of real-time speech-driven face animation with expressions for the iFACE system. Experimental results show that the synthetic expressive talking face of the iFACE system is comparable with a real face in terms of the effectiveness of their influences on bimodal human emotion perception.

Formant analysis is a technique widely used for speech analysis and synthesis. In this paper, we present a simple, fast and effective method for real time speech driven facial animation based on formant analysis. Speech signal is first processed by a formant analyzer. Since the resulting formants are known to be correlated with vocal tract shape, the formants can be directly mapped to mouth shapes. In addition, median filter and energy modulation is used to smooth the mouth shape sequence. The smoothed mouth shape sequence is used to animate our synthetic 3D head model with synchronized audio. The proposed method is simple and does not rely on contextual information. Thus it is good for real time two-way communication applications. Since the method extracts mouth shapes from acoustic features, it is language independent. In speaker-independent case, the proposed method is also shown to work well.

This dissertation presents an integrated framework for face modeling, facial motion analysis, and facial motion synthesis. This framework systematically addresses three closely related research issues: (1) selecting a quantitative representation of facial deformation for face modeling and animation; (2) automatic facial motion analysis based on the same visual representation; and (3) speech-to-facial-coarticulation modeling. The framework provides a guideline for methodically building a face modeling and animation system. The systematicness of the framework is reflected by the links among its components, whose details are presented. Based on this framework, a face modeling and animation system, called the iFACE system, is developed. The system provides functionalities for customizing a generic face model for an individual, text-driven face animation, offline speech-driven face animation, and real-time speech-driven face animation.

We present a novel approach for learning patterns (sub-images) shared by multiple images without prior knowledge about the number and the positions of the patterns in the images. The patterns may undergo kinds of rigid and non-rigid transformations. To reduce the searching space, the images are pre-segmented and represented by attribute relation graphs (ARGs). The problem is then formulated as learning the isomorphic subgraph, called pattern ARG (PARG) from multiple sample ARGs (SARG) with regard to the attribute similarity and the relation similarity. An inexact graph-matching algorithm is proposed to establish the correspondence between each SARG and the PARG. Inexact graph matching and model editing based on Bayes' decision rule are incorporated into Generalized Expectation and Maximization (GEM) algorithm. The modified GEM algorithm combines soft decisions and hard decisions together to learn both the appearance and the structure of the PARG. In the experiments, the learned PARG successfully captures the appearance and spatial information of the concept shared by the images.

We propose a state-based approach to gesture learning and recognition. Using spatial clustering and temporal alignment, each gesture is defined to be an ordered sequence of states in spatial-temporal space. The 2D image positions of the centers of the head and both hands of the user are used as features; these are located by a color-based tracking method. From training data of a given gesture, we first learn the spatial information and then group the data into segments that are automatically aligned temporally. The temporal information is further integrated to build a finite state machine (FSM) recognizer. Each gesture has a FSM corresponding to it. The computational efficiency of the FSM recognizers allows us to achieve real-time on-line performance. We apply this technique to build an experimental system that plays a game of "Simon Says" with the user.

By using relevance feedback, content-based image retrieval (CBIR) allows the user to retrieve images interactively. Beginning with a coarse query, the user can select the most relevant images and provide a weight of preference for each relevant image to refine the query. The high level concept borne by the user and perception subjectivity of the user can be automatically captured by the system to some degree. This paper proposes an approach to utilize both positive and negative feedbacks for image retrieval. Support vector machines (SVM) is applied to classifying the positive and negative images. The SVM learning results are used to update the preference weights for the relevant images. This approach releases the user from manually providing preference weight for each positive example. Experimental results show that the proposed approach has improvement over the previous approach (Rui et al. 1997) that uses positive examples only.

We propose an approach that extracts patterns from a temporal signal sequence without prior knowledge about the lengths, positions and the number of the patterns. Previous research (Hong et al., 1999) proposes a scheme for extracting recurrent patterns from a noise free signal without temporal warping. To handle noise and nonlinear temporal warping, a threshold finite state machine (TFSM) is proposed to perform spatial-temporal data modeling. The TFSM is first roughly initialized. A variance of segmental K-means is used to train the TFSM. The training results give us both the patterns embedding in the signal sequence and the trained TFSM that can be used to represent and detect the patterns.

Proposes an approach to 2D gesture recognition that models each gesture as a finite state machine (FSM) in the spatial-temporal space. The model construction works in a semi-automatic way. The structure of the model is first manually decided based on the observation of the spatial topology of the data. The model is refined iteratively between two stages: data segmentation and model training. We incorporate a modified Knuth-Morris-Pratt algorithm recognition procedure to speed up recognition. The computational efficiency of the FSM recognizers allows real-time online performance to be achieved.

Face tracking allows hands-free human computer interactions. In spite of advances in computer hardware and efficient and robust vision algorithms for tracking, the requirement of face and facial tracking in particular situations remains unclear. This paper considers the problems of building a human computer interface via face tracking and describes an architecture for a novel tool-Natural Mouse. Natural Mouse allows the merger of state-of-art face-related techniques and human demands. The advantage of the Natural Mouse is that people can dynamically configure it and control it by facial gestures and face motion without the need to wear any accessory. A mouse icon displayed on the computer monitor visualizes the Natural Mouse and helps people to control it. An immediate application of the Natural Mouse will be as a device that enables people with hand and speech disabilities to communicate with a computer.

This paper proposes a new scheme for unsupervised multi-temporal sequence pattern extraction. The main idea of the scheme is iterative coarse to fine data examination. We decompose a pattern into ambiguous subpatterns and distinguishable sub-patterns (DSP). In each iteration, we coarsely examine the training temporal signal sequence by training an Elman neural network. The trained Elman network is used to select the DSP candidate set. Then, we look at the training signals around the DSPs and use maximum likelihood criteria to expand them into whole patterns. We cut out the new found patterns from the training signal sequence and repeat the whole procedure until no more new patterns are found. The experimental result shows this method promising.

This paper presents a system for analyzing and generating human mouth motion. We apply model-based tracking to a set of typical mouth image sequences and obtain model motion sequences, which are used to build the mouth motion space by applying principal component analysis (PCA). Given an abstract description of the mouth motion in the mouth motion space, our system can generate a new mouth motion image sequence.