OBJECTIVE: To optimise the identification of patients presenting with pain in emergency department (ED) settings with limited resources using multiple transfer learning techniques. METHODS: Two strategies were explored: (1) fine-tuning a pre-trained language model, previously fine-tuned on data from a well-resourced ED, using labelled data from a target ED, and (2) continual pre-training using task-specific unlabelled data to enhance clinical text classification. RESULTS: With 2000 labelled samples from a target ED, the combined strategies achieved an F1-score of 92%, demonstrating significant benefits of transfer learning in resource-constrained settings. DISCUSSION: Accurately identifying pain in patients upon arrival to the ED is crucial for timely and effective treatment. Findings suggest that combining both transfer learning strategies can significantly enhance pain identification performances in resource-constrained settings. CONCLUSION: Combining fine-tuning on labelled data and continual pre-training on unlabelled data has potential to optimise model performance in both resource-constrained and well-resourced settings, highlighting the broader applicability and potential of these techniques for improving clinical text classification.

• Introduced a principled definition of an uncertain decision boundary for BERT-based classifiers using a discriminative confidence margin, enabling systematic identification of ambiguous predictions. • Integrated word–class probability embeddings (WCPE) to enrich document representations with explicit class-discriminative signals, particularly effective for uncertain and noisy medical texts. • Proposed a two-stage, uncertainty-aware decision framework that routes confident samples to a baseline BERT model and uncertain samples to a WCPE-enhanced variant, improving robustness under ambiguity. • Developed a data-driven strategy for optimizing the uncertainty threshold (UncertainT), balancing coverage of ambiguous samples and overall classification accuracy. • Established the general applicability of the proposed uncertainty-aware decision framework across multiple datasets and pretrained backbones, while revealing that statistically significant improvements emerge selectively depending on dataset ambiguity and backbone pretraining. Bidirectional Encoder Representations from Transformers (BERT) has achieved state-of-the-art performance in Natural Language Processing (NLP) tasks but struggles with uncertainty management in multi-class medical text classification, where overlapping categories and domain-specific critical terms pose challenges. BERT’s attention mechanism may distribute focus across irrelevant contextual patterns, reducing its ability to prioritize medically significant terms. To address these limitations, we propose a two-stage decision-making framework incorporating an “uncertain boundary” mechanism to separate high-confidence (“certain”) and low-confidence (“uncertain”) cases based on an optimized uncertainty threshold. A baseline BERT model processes high-confidence cases, while uncertain cases are handled by an enhanced BERT model integrating Word-Class Probabilistic Embedding (WCPE) to improve domain-specific representation learning. We evaluate our framework across three datasets-Ohsumed, PubMed, and Drug Review and further examine its robustness under multiple pretrained language model settings. Using standard BERT as the primary backbone, results show consistent accuracy improvements from 0.770 to 0.773 on Ohsumed ( p < 0.05), 0.973 to 0.975 on PubMed ( p < 0.1), and 0.596 to 0.609 on Drug Review ( p < 0.05) with stronger gains on uncertain subsets. These findings show that confidence-based partitioning and model specialization enhance BERT’s classification performance while effectively handling uncertainty in medical NLP.

Text classification techniques are playing a crucial role in identifying relevant texts from a large data set, e.g., various online crimes such as Cyberbullying, terrorist recruiting, propaganda or attack planning. Until now, supervised deep learning has brought about breakthroughs in processing multimedia data; however, there was no good practical way to harvest this opportunity for text classification because acquiring and maintaining a massive amount of training examples are too expensive for a large number of categories (e.g., Yahoo! taxonomy contains nearly 300,000 categories and the Library of Congress Subject Headings (LCSH) contains 394,070 subjects). Therefore, the question of how to effectively learn from sparse or small set of training examples is crucial for the true success of text classification. Semi-supervised approaches have been proposed for this challenge, which usually use a pair or several existing classifiers to extend a small training set. However, extracted pseudo training samples are uncertain because they are determined by a machine rather than people. Also, the massive volume and high variability of text data are creating a number of challenging issues such as the scalability and complicated relations between words. There are two fundamental issues with regards to the performance of existing classifiers: overlook and overload. Overlook means that some objects relevant to a class have been omitted, whereas overload means that some objects assigned to a class are actually not relevant to that class. The two issues are even more serious in the following two cases: (1) large uncertain boundary - the decision boundary between two classes includes many mixed examples (e.g., relevant and nonrelevant documents together), and (2) unbalanced classes - one class (e.g., information about terrorist attacks) is much smaller than another class (e.g., normal descriptions). We propose a three-way decision model [1] for dealing with the uncertain boundary for improving text classification performance based on rough set techniques and centroid solution. It aims to understand the uncertain boundary through partitioning the training samples into three regions (the positive, boundary and negative regions) by two main boundary vectors created from the labeled positive and negative training subsets, respectively, and further resolve the objects in the boundary region by two derived boundary vectors produced according to the structure of the boundary region. Four decision rules are proposed from the training process and applied to the incoming documents for more precise classification. The experimental results on the standard data sets RCV1 and Reuters-21578 show that the usage of boundary vectors is very effective and efficient for dealing with uncertainties of the decision boundary, and the proposed model has significantly improved the performance of binary text classification in terms of F1 measure and AUC area compared with six other popular baseline models.

Text classification is a process of classifying documents into predefined categories through different classifiers learned from labelled or unlabelled training samples. Many researchers who work on binary text classification attempt to find a more effective way to separate relevant texts from a large data set. However, current text classifiers cannot unambiguously describe the decision boundary between positive and negative objects because of uncertainties caused by text feature selection and the knowledge learning process. This paper proposes a three-way decision model for dealing with the uncertain boundary to improve the binary text classification performance based on therough settechniques and centroid solution. It aims to understand the uncertain boundary through partitioning the training samples into three regions (the positive, boundary, and negative regions) by two main boundary vectors$\vec{C_{P}}$and$\vec{C_{N}}$, created from the labeled positive and negative training subsets, respectively, and further resolve the objects in the boundary region by two derived boundary vectors$\vec{B_{P}}$and$\vec{B_{N}}$, produced according to the structure of the boundary region. It involves an indirect strategy which is composed of two successive steps in the whole classification process: ‘two-way to three-way’ and ‘three-way to two-way’. Four decision rules are proposed from the training process and applied to the incoming documents for more precise classification. A large number of experiments have been conducted based on the standard data sets RCV1 and Reuters-21578. The experimental results show that the usage of boundary vectors is very effective and efficient for dealing with uncertainties of the decision boundary, and the proposed model has significantly improved the performance of binary text classification in terms of$F_{1}$measure and$AUC$area compared with six other popular baseline models.

Addressing the problem of information overload, automatic multi-document summarization (MDS) has been widely utilized in the various real-world applications. Most of existing approaches adopt term-based representation for documents which limit the performance of MDS systems. In this paper, we proposed a novel unsupervised pattern-enhanced topic model (PETMSum) for the MDS task. PETMSum combining pattern mining techniques with LDA topic modelling could generate discriminative and semantic rich representations for topics and documents so that the most representative, non-redundant, and topically coherent sentences can be selected automatically to form a succinct and informative summary. Extensive experiments are conducted on the data of document understanding conference (DUC) 2006 and 2007. The results prove the effectiveness and efficiency of our proposed approach.