Vision Transformers (ViTs) have shown significant promise in computer vision applications. However, their performance in few-shot learning is limited by challenges in refining token-level interactions, struggling with limited training data, and developing a strong inductive bias. Existing methods often depend on inflexible token matching or basic similarity measures, which limit the effective incorporation of global context and localized feature refinement. To address these challenges, we propose Bi-Level Adaptive Token Refinement for Few-Shot Transformers (BATR-FST), a two-stage approach that progressively improves token representations and maintains a robust inductive bias for few-shot classification. During the pre-training phase, Masked image Modeling (MIM) provides Vision Transformers (ViTs) with transferable patch-level representations by recreating masked image regions, providing a robust basis for subsequent adaptation. In the meta-fine-tuning phase, BATR-FST incorporates a Bi-Level Adaptive Token Refinement module that utilizes Token Clustering to capture localized interactions, Uncertainty-Aware Token Weighting to prioritize dependable features, and a Bi-Level Attention mechanism to balance intra-cluster and inter-cluster relationships, thereby facilitating thorough token refinement. Furthermore, Graph Token Propagation ensures semantic consistency between support and query instances, while a Class Separation Penalty preserves different class borders, enhancing discriminative capability. Extensive experiments on three benchmark few-shot datasets demonstrate that BATR-FST achieves superior results in both 1-shot and 5-shot scenarios and improves the few-shot classification via transformers.

In fake news detection, agent systems leverage various tools, including large language models (LLMs), search engines, and other external resources, to conduct multi-dimensional analyses of news items. These tools assess factors like text style, emotional tone, and contextual relevance, providing essential auxiliary information. However, when LLMs are tasked with synthesizing these analyses, they often struggle to appropriately weigh the significance of each dimension, leading to suboptimal performance. To address this limitation, we propose the Adaptive Multi-Dimensional Weight Adjustment (AMDWA) approach. AMDWA differs from traditional agent-based methods in two key ways: it not only integrates the output of the various tools employed by the agent but also incorporates a small fine-tuned language model as part of the multi-dimensional analysis. Furthermore, it employs an attention mechanism to dynamically adjust the weight of each dimension, improving both the accuracy and interpretability of fake news detection. Experimental results demonstrate that AMDWA significantly improves detection performance on two real-world datasets.

Spatial Transcriptomics (ST) has made great progress in breast cancer due to it captures gene expression with fine-grained spots. It has always been low-throughout owing to reliance on special and pricey technologies. Recently, numerous types of models focus on predicting gene expression in windows (i.e., spots) on tissue images, aiming to provide alternative ST data. However, in these models, the interrelation information between windows is not well considered. We propose a Node-Edge Bi-level Network (NebNet ) for gene expression prediction within tissue slide images. Our model learns inter-relation information among windows, by using our message passing mechanism. Using NebNet, without any additional setting, experiments conducted on 10x Genomics breast cancer data show that our NebNet achieves an impressive PCC@S of 8.26 for gene expression prediction. This performance exceeds the current state-of-the-art model by nearly 6.6%. Code is available at https://github.com/biyecc/NebNet.

Sentence embedding is a bridge between Pretrained Language Models (PLMs) and downstream tasks. Pooling is the main way to obtain sentence embeddings from PLMs. In this paper, we explore in depth the specific effects of different pooling operations on sentence embeddings in PLMs and introduce an innovative multidimensional evaluation method that aims to go beyond the limitations of a single metric to provide richer metrics for the comprehensive performance evaluation of sentence embeddings. By fusing Uniformity Alignment, Spearman’s coefficient, visual analysis, and stability metrics, we reveal the limitations of the traditional single-indicator evaluation method. In particular, we innovatively introduce a Dual Pooling strategy and combine it with a visual analysis of the Convex Hull area to provide a new perspective for the performance evaluation of sentence embeddings. Experiments analyzing the performance of supervised and unsupervised models in the Semantic Text Similarity (STS) task validate the effectiveness of the Dual Pooling strategy in examining the performance of sentence embeddings, while the Convex Hull area visualization provides an intuitive tool for the qualitative analysis of sentence embedding quality. Summing up the experimental results, we suggest that the impact of pooling operations should be considered when selecting pre-trained language models, and that multidimensional evaluation methods should be further developed and refined to enable the evaluation of models from different perspectives. We encourage the exploration of new pooling strategies and the development of evaluation tools to promote the development of natural language processing technology.

Graph classification is of great significance to many real-world applications like drug discovery, for which subgraph neural networks are gaining increasing attention due to their strong power in capturing node attributes along with finer substructures for graph representation learning. In this study, focusing on the crucial substructure of community inherent in complex networks, a fuzzy overlapping community guided subgraph neural network, called FOCsubGNN, is innovatively proposed to learn their fine-grained representation. Specifically, fuzzy overlapping community detection enables the most realistic node relationships can be discovered for neighbor aggregation, which provides great potential to achieve accurate graph representation. On this basis, to extract discriminative features for more promising classification, we also bring out an intra-subgraph node discarding based pooling strategy, which allows the graph intrinsic hierarchy can be well integrated into these extracted features, and thus playing a positive role in promoting more superior classification performance. Extensive experiments on 6 popular benchmark datasets demonstrate the effectiveness of FOCsubGNN.

Camouflaged object detection (COD) aims to identify objects that blend into the surrounding backgrounds, which has been a hot topic in recent years, with many different optimization strategies being explored. Among these, collaborative detection, a recently proposed solution for COD, has shown outstanding performance. However, current collaborative detection methods adopt a "one-to-many" pattern, failing to fully leverage the advantages of collaborative detection. Moreover, existing approaches to disguised target detection overlook the importance of high-level features. To address these issues, we propose a novel dualcross query network (DCQNet). It effectively capitalizes on the commonalities among objects and the directive role of high-level features in enhancing low-level features. Specifically, we designed a cross-sample query module and a cross-scale query module to collaboratively locate the object and guide low-level features, respectively. Extensive experimental results demonstrate that DCQNet outperforms state-of-the-art (SOTA) methods on the CoCOD8K dataset.

In Natural Language Processing (NLP), comprehending the semantic connection between two texts, a Semantic Text Similarity (STS) task, poses a significant challenge. This challenge is especially pronounced in resource-constrained and cross-domain contexts, where traditional methods are hindered by the high costs associated with data labeling. We propose an innovative technique, designated as “Cold-Start Self-Training” that reduces reliance on large labeled datasets for STS tasks in resource-restricted settings. This method utilizes dual-view pooling to extract semantic similarity information from unlabeled data and generates pseudo-labeled data to fine-tune the cross-encoder model. Dual-view pooling combines different pooling results of the same text to evaluate semantic similarity without additional model tuning, simplifying the self-training process. Experimental results show that our method significantly improves the cross-encoder model's performance on STS tasks in the medical domain. Our findings provide new strategies for cross-domain STS tasks, challenging the traditional reliance on extensive labeled data. We also validate the potential of unsupervised pretrained models for cross-domain tasks, offering theoretical and practical support for complex challenges.

Continual Learning for Named Entity Recognition (CL-NER) is designed to train models capable of adapting to evolving data by continuously introducing new entity types. This approach is crucial in dynamic environments where data evolves, such as social media, healthcare, and legal documents, necessitating the model to retain the memory of previously learned entity types while learning to identify new ones. However, due to the neural network's tendency to acquire new knowledge and forget old knowledge in continual learning and the unique non-entity type annotations in NER tasks, CL-NER faces severe catastrophic forgetting and semantic drift issues. In this paper, we propose Enhanced Knowledge Distillation and Entropy-based Adaptive Pseudo Label Weights (EDAW) to address the catastrophic forgetting and semantic drift issues in CL-NER. Specifically, we develop an enhanced knowledge distillation method that combines Kullback-Leibler divergence and feature cosine discrepancy. This method effectively minimizes the variance in output probability distributions and aligns the internal feature spaces between new and old models, thus reducing catastrophic forgetting. Additionally, we propose an entropy-based adaptive pseudo label weight method that allows the model to assign different weights to pseudo labels with varying certainties during training, effectively alleviating semantic drift and error accumulation caused by erroneous relabeling of pseudo labels. Notably, this study pioneers the in-clusion of a Chinese dataset in CL-NER, enhancing the model's robustness and demonstrating its efficacy in a multilingual context. Experiments on fourteen CL-NER settings across four public NER datasets show that EDAW improves average Micro-F1 and Macro-F1 scores by 3.44% and 3.72%, respectively, over existing state-of-the-art(SOTA) methods. We make our code available at: https://github.com/livosr/EDAW/tree/master

Vehicle trajectories represent an essential information source in intelligent transportation systems. Prior trajectory completion models based on Automatic Number Plate Recognition (ANPR) data have typically depended on vehicle re-identification results or road network information or have employed static knowledge graphs to integrate the two information sources. However, these methods have not taken into account the implicit temporal characteristics of trajectories in ANPR data and have neglected individual vehicle preferences. To address this void, this study proposes a Temporal Knowledge Graph-based Vehicle Trajectory Completion Model (TKG-VTC). The model implementation comprises three stages: first, ANPR data are converted into a temporal trajectory knowledge graph; second, knowledge representation learning is conducted using nontemporal relations, a biased temporal regularizer and multivector embeddings to embed the knowledge on the graph; and finally, the embedded results are employed to perform link prediction for incomplete trajectories, thereby restoring vehicle trajectories in ANPR data. Through model evaluation metrics and dimensionality reduction experiments, TKG-VTC is observed to demonstrate the best performance in completing trajectories when compared to TComplEx, TNTComplEx, and TeLM. This research introduces an innovative application of employing temporal knowledge graphs for trajectory reconstruction, which eliminates dependence on vehicle re-identification and road network information in previous methodologies. This is advantageous for enhancing the performance and dependability of vehicle trajectory data in intelligent transportation systems, as well as facilitating the implementation of trajectory prediction, demand analysis, and accident warning applications.

Abstract The effective entity recognition method can quickly and accurately identify the citation entity to facilitate citation comparison, thereby reducing the occurrence of academic fraud and other behaviors. But there is no very effective way to solve this problem till now. In recent years, neural network models for named entity recognition (NER) have shown better performances on general domain datasets. After the multi‐feature citation dataset is created, the article proposes contextual multi‐feature embedding (CMFE) method for word embedding which use multi‐feature to enhance semantic and use CNN to get multi‐level feature. Based on CMFE, a multi‐feature semantic fusion model (MFSFM) is proposed. It designs the multi‐convolution kernel mixed residual CNN module to obtain local attention information and enhance the sensitivity of the entity boundary information. The BiLSTM and LSTM is used for timing learning. The experimental results of Chinese citation datasets and Chinese–English mixed citation datasets show that CMFE can better represent semantics, and MFSFM can perform citation entity recognition well. Finally, the experimental results of CONLL2003 dataset show that it is general on NER.

BACKGROUND: Drug repositioning refers to the identification of new indications for existing drugs. Drug-based inference methods for drug repositioning apply some unique features of drugs for new indication prediction. Complementary information is provided by these different features. It is therefore necessary to integrate these features for more accurate in silico drug repositioning. RESULTS: In this study, we collect 3 different types of drug features (i.e., chemical, genomic and pharmacological spaces) from public databases. Similarities between drugs are separately calculated based on each of the features. We further develop a fusion method to combine the 3 similarity measurements. We test the inference abilities of the 4 similarity datasets in drug repositioning under the guilt-by-association principle. Leave-one-out cross-validations show the integrated similarity measurement IntegratedSim receives the best prediction performance, with the highest AUC value of 0.8451 and the highest AUPR value of 0.2201. Case studies demonstrate IntegratedSim produces the largest numbers of confirmed predictions in most cases. Moreover, we compare our integration method with 3 other similarity-fusion methods using the datasets in our study. Cross-validation results suggest our method improves the prediction accuracy in terms of AUC and AUPR values. CONCLUSIONS: Our study suggests that the 3 drug features used in our manuscript are valuable information for drug repositioning. The comparative results indicate that integration of the 3 drug features would improve drug-disease association prediction. Our study provides a strategy for the fusion of different drug features for in silico drug repositioning.

BACKGROUND: As regulators of gene expression, microRNAs (miRNAs) are increasingly recognized as critical biomarkers of human diseases. Till now, a series of computational methods have been proposed to predict new miRNA-disease associations based on similarity measurements. Different categories of features in miRNAs are applied in these methods for miRNA-miRNA similarity calculation. Benchmarking tests on these miRNA similarity measures are warranted to assess their effectiveness and robustness. RESULTS: In this study, 5 categories of features, i.e. miRNA sequences, miRNA expression profiles in cell-lines, miRNA expression profiles in tissues, gene ontology (GO) annotations of miRNA target genes and Medical Subject Heading (MeSH) terms of miRNA-associated diseases, are collected and similarity values between miRNAs are quantified based on these feature spaces, respectively. We systematically compare the 5 similarities from multi-statistical views. Furthermore, we adopt a rule-based inference method to test their performance on miRNA-disease association predictions with the similarity measurements. Comprehensive comparison is made based on leave-one-out cross-validations and a case study. Experimental results demonstrate that the similarity measurement using MeSH terms performs best among the 5 measurements. It should be noted that the other 4 measurements can also achieve reliable prediction performance. The best-performed similarity measurement is used for new miRNA-disease association predictions and the inferred results are released for further biomedical screening. CONCLUSIONS: Our study suggests that all the 5 features, even though some are restricted by data availability, are useful information for inferring novel miRNA-disease associations. However, biased prediction results might be produced in GO- and MeSH-based similarity measurements due to incomplete feature spaces. Similarity fusion may help produce more reliable prediction results. We expect that future studies will provide more detailed information into the 5 feature spaces and widen our understanding about disease pathogenesis.

BACKGROUND: It has been shown that the deregulation of miRNAs is associated with the development and progression of many human diseases. To reduce time and cost of biological experiments, a number of algorithms have been proposed for predicting miRNA-disease associations. However, the existing methods rarely investigated the cause-and-effect mechanism behind these associations, which hindered further biomedical follow-ups. RESULTS: In this study, we presented a CCA-based model in which the possible molecular causes of miRNA-disease associations were comprehensively revealed by extracting correlated sets of genes and diseases based on the co-occurrence of miRNAs in target gene profiles and disease profiles. Our method directly suggested the underlying genes involved, which could be used for experimental tests and confirmation. The inference of associated diseases of a new miRNA was made by taking into account the weight vectors of the extracted sets. We extracted 60 pairs of correlated sets from 404 miRNAs with two profiles for 2796 target genes and 362 diseases. The extracted diseases could be considered as possible outcomes of miRNAs regulating the target genes which appeared in the same set, some of which were supported by independent source of information. Furthermore, we tested our method on the 404 miRNAs under the condition of 5-fold cross validations and received an AUC value of 0.84606. Finally, we extensively inferred miRNA-disease associations for 100 new miRNAs and some interesting prediction results were validated by established databases. CONCLUSIONS: The encouraging results demonstrated that our method could provide a biologically relevant prediction and interpretation of associations between miRNAs and diseases, which were of great usefulness when guiding biological experiments for scientific research.

Summary We present EcForest, an extractive summarization model through Enhanced Sentence Embedding and Cascade Forest. Sentence representation is of great significance for many summarization methods. Bag‐of‐words mostly fails to grasp the semantics, and typical embedding models cannot capture more complex semantic features, such as polysemy and the meaning of a phrase, which is usually ignored by simply averaging the word embeddings included in a sentence. To this end, we propose Enhanced Sentence Embedding (ESE) model to solve such drawbacks via mapping several valid features to dense vectors. Essentially, the enhanced sentence embedding is a novel model for improving the distributed representation of sentence. Our sentence embedding model is universally applicable and it can be adapted to other NLP tasks. Moreover, deep forest is used as a sentence extraction algorithm for its robustness to the hyper‐parameters and its efficient training algorithm compared to deep neural network. The evaluation of variant models proposed in this work proves the validation of the enhanced sentence embedding. The comparison results between EcForest and several baselines on two different datasets demonstrate that the proposed summarization model performs better than or with high competitiveness to the state‐of‐the‐art.

In the era of information overload, text summarization has become a focus of attention in a number of diverse fields such as, question answering systems, intelligence analysis, news recommendation systems, search results in web search engines, and so on. A good document representation is the key point in any successful summarizer. Learning this representation becomes a very active research in natural language processing field (NLP). Traditional approaches mostly fail to deliver a good representation. Word embedding has proved an excellent performance in learning the representation. In this paper, a modified BM25 with Word Embeddings are used to build the sentence vectors from word vectors. The entire document is represented as a set of sentence vectors. Then, the similarity between every pair of sentence vectors is computed. After that, TextRank, a graph-based model, is used to rank the sentences. The summary is generated by picking the top-ranked sentences according to the compression rate. Two well-known datasets, DUC2002 and DUC2004, are used to evaluate the models. The experimental results show that the proposed models perform comprehensively better compared to the state-of-the-art methods.

Accumulating experimental evidence has indicated that long non-coding RNAs (lncRNAs) are critical for the regulation of cellular biological processes implicated in many human diseases. However, only relatively few experimentally supported lncRNA-disease associations have been reported. Developing effective computational methods to infer lncRNA-disease associations is becoming increasingly important. Current network-based algorithms typically use a network representation to identify novel associations between lncRNAs and diseases. But these methods are concentrated on specific entities of interest (lncRNAs and diseases) and they do not allow to consider networks with more than two types of entities. Considering the limitations in previous computational methods, we develop a new global network-based framework, LncRDNetFlow, to prioritize disease-related lncRNAs. LncRDNetFlow utilizes a flow propagation algorithm to integrate multiple networks based on a variety of biological information including lncRNA similarity, protein-protein interactions, disease similarity, and the associations between them to infer lncRNA-disease associations. We show that LncRDNetFlow performs significantly better than the existing state-of-the-art approaches in cross-validation. To further validate the reproducibility of the performance, we use the proposed method to identify the related lncRNAs for ovarian cancer, glioma, and cervical cancer. The results are encouraging. Many predicted lncRNAs in the top list have been verified by the biological studies.

Aggregating evidences have shown that long non-coding RNAs (lncRNAs) generally play key roles in cellular biological processes such as epigenetic regulation, gene expression regulation at transcriptional and post-transcriptional levels, cell differentiation, and others. However, most lncRNAs have not been functionally characterized. There is an urgent need to develop computational approaches for function annotation of increasing available lncRNAs. In this article, we propose a global network-based method, KATZLGO, to predict the functions of human lncRNAs at large scale. A global network is constructed by integrating three heterogeneous networks: lncRNA-lncRNA similarity network, lncRNA-protein association network, and protein-protein interaction network. The KATZ measure is then employed to calculate similarities between lncRNAs and proteins in the global network. We annotate lncRNAs with Gene Ontology (GO) terms of their neighboring protein-coding genes based on the KATZ similarity scores. The performance of KATZLGO is evaluated on a manually annotated lncRNA benchmark and a protein-coding gene benchmark with known function annotations. KATZLGO significantly outperforms state-of-the-art computational method both in maximum F-measure and coverage. Furthermore, we apply KATZLGO to predict functions of human lncRNAs and successfully map 12,318 human lncRNA genes to GO terms.

Motivation: Long non-coding RNAs (lncRNAs) are an enormous collection of functional non-coding RNAs. Over the past decades, a large number of novel lncRNA genes have been identified. However, most of the lncRNAs remain function uncharacterized at present. Computational approaches provide a new insight to understand the potential functional implications of lncRNAs. Results: Considering that each lncRNA may have multiple functions and a function may be further specialized into sub-functions, here we describe NeuraNetL2GO, a computational ontological function prediction approach for lncRNAs using hierarchical multi-label classification strategy based on multiple neural networks. The neural networks are incrementally trained level by level, each performing the prediction of gene ontology (GO) terms belonging to a given level. In NeuraNetL2GO, we use topological features of the lncRNA similarity network as the input of the neural networks and employ the output results to annotate the lncRNAs. We show that NeuraNetL2GO achieves the best performance and the overall advantage in maximum F-measure and coverage on the manually annotated lncRNA2GO-55 dataset compared to other state-of-the-art methods. Availability and implementation: The source code and data are available at http://denglab.org/NeuraNetL2GO/. Contact: [email protected]. Supplementary information: Supplementary data are available at Bioinformatics online.

Two practicable approaches were proposed for Web service retrieval, bipartite-graph matching and KbSM. But their models and similarity metrics of WSDL analysis may ignore some term or semantic feature, and involve formal method problem of representation or difficulty of parameter verification. SLVM and its improved model depend on statistical term measures to implement XML document representation. As a result, they ignore the lexical semantics and the distilled mutual information, leading to text analysis errors. This work proposed a service retrieval method, hybrid SLVM of WSDL, to address the problem of feature extraction. Using WordNet, this method constructed a lexical semantic spectrum to characterize the lexical semantics, and built a special term spectrum based on TF-IDF. Then, feature matrix for WSDL representation was built in the hybrid SLVM. Applying to NWKNN algorithm, on OWLS-TC version 2 dataset, the experimental results show that the feature matrix of our method performs F1 measure and query precisions better than bipartite-graph matching and KbSM.

Single Nucleotide Polymorphisms (SNPs) play an important role in Genome-wide Association Studies. To reduce genotyping costs, several LD tagSNPs selection algorithms have been proposed. In this paper, the advantages and disadvantages of current LD tagSNPs selection algorithms are analyzed. And a refined and heuristic algorithm HTag for LD tagSNPs selection is developed: (1) The tagSNPs selection procedure of Xu et al. is modified to improve selection performance. (2) A strategy to optimize the selection result is proposed. Using data downloaded from the HapMap Project, the performance of these methods is evaluated and our algorithm shows improvements in tagging efficiency.