OBJECTIVE: This study introduces a novel evaluation framework, GPTRadScore, to systematically assess the performance of multimodal large language models (MLLMs) in generating clinically accurate findings from CT imaging. Specifically, GPTRadScore leverages LLMs as an evaluation metric, aiming to provide a more accurate and clinically informed assessment than traditional language-specific methods. Using this framework, we evaluate the capability of several MLLMs, including GPT-4 with Vision (GPT-4V), Gemini Pro Vision, LLaVA-Med, and RadFM, to interpret findings in CT scans. METHODS: This retrospective study leverages a subset of the public DeepLesion dataset to evaluate the performance of several multimodal LLMs in describing findings in CT slices. GPTRadScore was developed to assess the generated descriptions (location, body part, and type) using GPT-4, alongside traditional metrics. RadFM was fine-tuned using a subset of the DeepLesion dataset with additional labeled examples targeting complex findings. Post fine-tuning, performance was reassessed using GPTRadScore to measure accuracy improvements. RESULTS: Evaluations demonstrated a high correlation of GPTRadScore with clinician assessments, with Pearson's correlation coefficients of 0.87, 0.91, 0.75, 0.90, and 0.89. These results highlight its superiority over traditional metrics, such as BLEU, METEOR, and ROUGE, and indicate that GPTRadScore can serve as a reliable evaluation metric. Using GPTRadScore, it was observed that while GPT-4V and Gemini Pro Vision outperformed other models, significant areas for improvement remain, primarily due to limitations in the datasets used for training. Fine-tuning RadFM resulted in substantial accuracy gains: location accuracy increased from 3.41% to 12.8%, body part accuracy improved from 29.12% to 53%, and type accuracy rose from 9.24% to 30%. These findings reinforce the hypothesis that fine-tuning RadFM can significantly enhance its performance. CONCLUSION: GPT-4 effectively correlates with expert assessments, validating its use as a reliable metric for evaluating multimodal LLMs in radiological diagnostics. Additionally, the results underscore the efficacy of fine-tuning approaches in improving the descriptive accuracy of LLM-generated medical imaging findings.

Temporal network embedding (TNE) has promoted the research of knowledge discovery and reasoning on networks. It aims to embed vertices of temporal networks into a low-dimensional vector space while preserving network structures and temporal properties. However, most existing methods have limitations in capturing dynamics over long distances, which makes it difficult to explore multihop topological associations among vertices. To tackle this challenge, we propose LongTNE, which learns the long-range dynamics of vertices to endow TNE with the ability to capture high-order proximity (HP) of networks. In LongTNE, we employ graph self-supervised learning (Graph SSL) to optimize the establishment probability of deep links in each network snapshot. We also present an accumulated forward update (AFU) module to fathom global temporal evolution among multiple network snapshots. The empirical results on six temporal networks demonstrate that, in addition to achieving state-of-the-art performance on network mining tasks, LongTNE can be handily extended to existing TNE methods.

The Relational Triple Extraction (RTE) task is a fundamental and essential information extraction task.Recently, the table-filling RTE methods have received lots of attention.Despite their success, they suffer from some inherent problems such as underutilizing regional information of triple.In this work, we treat the RTE task based on table-filling method as an Object Detection task and propose a one-stage Object Detection framework for Relational Triple Extraction (OD-RTE).In this framework, the vertices-based bounding box detection, coupled with auxiliary global relational triple region detection, ensuring that regional information of triple could be fully utilized.Besides, our proposed decoding scheme could extract all types of triples.In addition, the negative sampling strategy of relations in the training stage improves the training efficiency while alleviating the imbalance of positive and negative relations.The experimental results show that 1) OD-RTE achieves the state-of-the-art performance on two widely used datasets (i.e., NYT and WebNLG).2) Compared with the best performing table-filling method, OD-RTE achieves faster training and inference speed with lower GPU memory usage.To facilitate future research in this area, the codes are publicly available at https://github.com/NingJinzhong/ODRTE.

The task of extracting overlapping and nested events has received significant attention in recent times, as prior research has primarily focused on extracting flat events, overlooking the intricacies of overlapping and nested occurrences. In this work, we present a new approach to Event Extraction (EE) by reformulating it as an object detection task on a table of token pairs. Our proposed one-stage event extractor, called ODEE, can handle overlapping and nested events. The model is designed with a vertex-based tagging scheme and two auxiliary tasks of predicting the spans and types of event trigger words and argument entities, leveraging the full span information of event elements. Furthermore, in the training stage, we introduce a negative sampling method for table cells to address the imbalance problem of positive and negative table cell tags, meanwhile improving computational efficiency. Empirical evaluations demonstrate that ODEE achieves the state-of-the-art performance on three benchmarks for overlapping and nested EE (i.e., FewFC, Genia11, and Genia13). Furthermore, ODEE outperforms current state-of-the-art methods in terms of both number of parameters and inference speed, indicating its high computational efficiency. To facilitate future research in this area, the codes are publicly available at https://github.com/NingJinzhong/ODEE.

Chinese Named Entity Recognition (NER) has continued to attract research attention. However, most existing studies only explore the internal features of the Chinese language but neglect other lingual modal features. Actually, as another modal knowledge of the Chinese language, English contains rich prompts about entities that can potentially be applied to improve the performance of Chinese NER. Therefore, in this study, we explore the bilingual enhancement for Chinese NER and propose a unified bilingual interaction module called the Adapted Cross-Transformers with Global Sparse Attention (ACT-S) to capture the interaction of bilingual information. We utilize a model built upon several different ACT-Ss to integrate the rich English information into the Chinese representation. Moreover, our model can learn the interaction of information between bilinguals (inter-features) and the dependency information within Chinese (intra-features). Compared with existing Chinese NER methods, our proposed model can better handle entities with complex structures. The English text that enhances the model is automatically generated by machine translation, avoiding high labour costs. Experimental results on four well-known benchmark datasets demonstrate the effectiveness and robustness of our proposed model.

Semantic network is a network of concepts connected by semantic relations. It contains two forms ofbinary semantic networkandmultiplex semantic network. The associated fact prediction is a link prediction task that aims to infer the implicitly connected facts by mining the high-level representation of the network. Previous methods for associated fact prediction put much emphasis on the topological feature of network but not utilize the information of semantic expression. This paper proposes aSemanticNetworkEncoder (SemNE), which learns a feature mapping function from the binary semantic networks and can be applied to the multiplex semantic networks in a pre-training manner. SemNE is a two-stage framework that contains an embedding encoder and a prediction decoder. It jointly models the semantic information and network topology to enrich the network representation. A word self-organization method based on the factual boundary is proposed to unify the topological feature and the semantic feature representations. Experimental results on binary semantic networks show that SemNE achieves the state-of-the-art results in associated fact prediction and experimental results on multiplex semantic networks show that SemNE is scalable and can effectively improve the performance of existing models.

The strategies for important node detection according to topological structures are widely explored in complex networks. The diameter is a very important topological parameter among various network topological indicators. However, it is seldom utilized in important node searching methods. In this study, we defined the nodes on the diameter paths as central nodes and proposed a Diameter Center Detection (DCD) method to search the central nodes. In the experiments, the DCD method is applied to three deterministic networks, a series of small-world networks, scale-free networks and five real networks, respectively. The experimental results show that the central nodes searched by DCD have advantages over the nodes of the whole network in the evaluations of various centrality measures, e.g. Betweenness Centrality (BC), Closeness Centrality (CC), Degree Centrality (DC) and k-shell decomposition results. In addition, after deleting central nodes, the network structure changes a lot on the perspective from both diameter and giant component. Furthermore, the experimental results show that the edge deleting policy based on DCD is effective in the way that deleting fewer edges disrupting more node pair connectivity.