OBJECTIVE: To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning. METHODS: We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (ie, trainable vectors) with frozen LLM, where the LLM parameters were not updated (ie, frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. RESULTS AND CONCLUSION: The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ∼3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4%-10% for clinical abbreviation disambiguation, and 5.5%-9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the "one model for all" promise from training to deployment using a unified generative LLM.

The differentiable neural architecture search (NAS) framework has obtained extensive attention and achieved remarkable performance due to its search efficiency. However, most existing differentiable NAS methods still suffer from issues of model collapse, degenerated search-evaluation correlation, and inefficient hardware deployment, which causes the searched architectures to be suboptimal in accuracy and cannot meet different computation resource constraints (e.g., FLOPs and latency). In this article, we propose a novel resource-constrained NAS (ReCNAS) method, which can efficiently search high-performance architectures that satisfy the given constraints, and deal with the issues observed in previous differentiable NAS methods from three aspects: search space, search strategy, and resource adaptability. First, we introduce an elastic densely connected layerwise search space, which decouples the architecture depth representation from the search of candidate operations to alleviate the aggregation of skip connections and architecture redundancies. Second, a scheme of group annealing and progressive pruning is proposed to improve the efficiency and bridge the search-evaluation gap, which steadily forces the architecture parameters close to binary distribution and progressively prunes the inferior operations. Third, we present a novel resource-constrained architecture generation method, which prunes the redundant channel throughout the search based on dynamic programming, making the searched architecture scalable to different devices and requirements. Extensive experimental results demonstrate the efficiency and search stability of our ReCNAS, which is capable of discovering high-performance architectures on different datasets and tasks, surpassing other NAS methods, while tightly meeting the target resource constraints without any tuning required. Besides, the searched architectures show strong generalizability to other complex vision tasks.

OBJECTIVE: To develop a natural language processing system that solves both clinical concept extraction and relation extraction in a unified prompt-based machine reading comprehension (MRC) architecture with good generalizability for cross-institution applications. METHODS: We formulate both clinical concept extraction and relation extraction using a unified prompt-based MRC architecture and explore state-of-the-art transformer models. We compare our MRC models with existing deep learning models for concept extraction and end-to-end relation extraction using 2 benchmark datasets developed by the 2018 National NLP Clinical Challenges (n2c2) challenge (medications and adverse drug events) and the 2022 n2c2 challenge (relations of social determinants of health [SDoH]). We also evaluate the transfer learning ability of the proposed MRC models in a cross-institution setting. We perform error analyses and examine how different prompting strategies affect the performance of MRC models. RESULTS AND CONCLUSION: The proposed MRC models achieve state-of-the-art performance for clinical concept and relation extraction on the 2 benchmark datasets, outperforming previous non-MRC transformer models. GatorTron-MRC achieves the best strict and lenient F1-scores for concept extraction, outperforming previous deep learning models on the 2 datasets by 1%-3% and 0.7%-1.3%, respectively. For end-to-end relation extraction, GatorTron-MRC and BERT-MIMIC-MRC achieve the best F1-scores, outperforming previous deep learning models by 0.9%-2.4% and 10%-11%, respectively. For cross-institution evaluation, GatorTron-MRC outperforms traditional GatorTron by 6.4% and 16% for the 2 datasets, respectively. The proposed method is better at handling nested/overlapped concepts, extracting relations, and has good portability for cross-institute applications. Our clinical MRC package is publicly available at https://github.com/uf-hobi-informatics-lab/ClinicalTransformerMRC.

As a critical step for Polarimetric Synthetic Aperture Radar (PolSAR) images interpretation, PolSAR classification have attracted growing interests in the field of remote sensing. Recently, many novel ideas and models based on deep learning have emerged to solve the task of PolSAR image classification. The encode-decode network structure of Fully Convolutional Network (FCN) is proved to be effective for this task. However, due to the inherent disadvantages of FCN and the complex high-dimensional feature representation of PolSAR images, there are still some issues need to be addressed. Aiming at the problem that the edge of adjacent regions is not enough finely classified and the regional consistency of the same object class is relatively weak, we propose a novel PolSAR image classification method called DBFCN, which combines a well-designed edge-aware network and the improved FCN. The experimental results verify that it can effectively improve the classification accuracy of PolSAR images.

As a fundamental but challenging task in the interpretation of remote sensing images, scene classification plays an important role in various applications and has become an active research topic. Many previous works have demonstrated the remarkable performance of the deep convolutional neural networks (CNNs) for remote sensing scene classification. However, the progress made by CNN-based methods for scene classification has gradually reached saturation in recent years, due to the serious dependence on the pretrained CNN models, the limitations of manually designed network architecture and the disadvantages of existing data sets. In this article, a new paradigm to automatically design a suitable CNN architecture for scene classification is investigated. We propose an efficient architecture search framework to discover optimal network architectures in continuous search space with the gradient-based optimization method. Our framework consists of two stages: the search phase and the evaluation phase. During the search process, a greedy and progressive search strategy is introduced to search network building blocks (i.e., cells) through bilevel optimization. Besides, we propose a simple architecture regularization scheme to further improve the search efficiency and the robustness of discovered architectures. After the search process, the optimal cell architectures are determined and then repeatedly stacked to construct the final network for evaluation. For the data set, we propose a mergence strategy to build a new large-scale remote sensing scene image data set that contains rich scene categories and image diversity, making it feasible to find a new CNN model with strong generalization ability for scene classification. Extensive experiments demonstrate the efficiency of the proposed search strategies and the impressive classification performance of searched CNN architectures on seven public benchmark data sets, including four large-scale data sets and three small-scale data sets.

Machine learning techniques have achieved remarkable development in recent years. However, the performance of many machine learning models usually involves careful tuning of hyperparameters. The hyperparameter optimization (HPO) is usually a high-dimensional black box optimization problem and often faces expensive function evaluations. Besides, the task of HPO has gained great attention in academy and industry. In this paper, a novel method for hyperparameter optimization is proposed, referred to as surrogate model assisted quantum-inspired evolutionary algorithm (SA-QEA), which incorporates the principles of quantum-inspired evolutionary algorithm (QEA) and an efficient search framework based on a surrogate model. In the proposed algorithm, we adopt a single individual QEA with neighborhood exploration as the evolution scheme to generate the candidate solutions, and multivariate adaptive regression splines (MARS) is used as a surrogate to approximate the objective function around the individuals. Through conducting comprehensive experimental evaluations on two benchmark problems and three machine learning models, we test our proposed algorithm and compare it with other widely used methods. The results achieve competitive performance and demonstrate the effectiveness of SA-QEA for hyperparameter optimization.

With the rapid development of various technologies of satellite sensor, synthetic aperture radar (SAR) image has been an import source of data in the application of change detection. In this paper, a novel method based on a convolutional neural network (CNN) for SAR image change detection is proposed. The main idea of our method is to generate the classification results directly from the original two SAR images through a CNN without any preprocessing operations, which also eliminate the process of generating the difference image (DI), thus reducing the influence of the DI on the final classification result. In CNN, the spatial characteristics of the raw image can be extracted and captured by automatic learning and the results with stronger robustness can be obtained. The basic idea of the proposed method includes three steps: it first produces false labels through unsupervised spatial fuzzy clustering. Then we train the CNN through proper samples that are selected from the samples with false labels. Finally, the final detection results are obtained by the trained convolutional network. Although training the convolutional network is a supervised learning fashion, the whole process of the algorithm is an unsupervised process without priori knowledge. The theoretical analysis and experimental results demonstrate the validity, robustness, and potential of our algorithm in simulated and real data sets. In addition, we try to apply our algorithm to the change detection of heterogeneous images, which also achieves satisfactory results.

Change detection in synthetic aperture radar (SAR) images is to detect the changes happening during a period of time in the same area, which has important application research value. In this paper, we propose a novel method based on spatial fuzzy clustering (SFCM) and deep auto-encoder for the change-detection of SAR. In this method, the difference image (DI) is generated by the log-ratio operator. Then, the spatial fuzzy clustering (SFCM) algorithm is used to analyze the DI. The spatial fuzzy clustering (SFCM) algorithm adds spatial information to the fuzzy cluster, which effectively reduces the influence of speckle noise. Finally, we choose appropriate samples to train the deep auto-encoder. Real data and theoretical analysis show the effectiveness and robustness of the proposed method.