Paired image-mask synthesis is essential for automating nanoparticle phenotyping in drug research, as it creates large-scale annotated image datasets for effective learning. However, challenges of complex spatial and morphological variations, such as dense, sparse, and overlapping structures, hinder existing synthesis methods, which often fail to produce high-quality images and masks from limited data. To tackle these challenges, This work introduces a novel conditional Generative Adversarial Network tailored for high-quality image-mask synthesis under limited training conditions. The proposed model features a dual-branch generator that independently synthesizes images and masks while maintaining structural consistency. It also employs a diffusion-based multi-level discriminator to enhance image quality feedback across various noise levels and spatial scales. Qualitative and quantitative evaluations show that the proposed method enhances both synthetic image quality and mask accuracy. This advancement has the potential to streamline the synthesis of large-scale image-mask pairs for nanoparticles, ultimately benefiting automated phenotyping and drug screening efforts.

Automated nanoparticle phenotyping in drug discovery requires robust learning models trained on large, annotated datasets of microscope images. However, obtaining detailed pixel-level annotations for large-scale biomedical images is a challenging and labor-intensive task. Image synthesis offers a potential solution to augment limited training datasets through the generation of paired image-mask samples. While existing synthesis methods prioritize image fidelity, they often overlook the accuracy of the corresponding masks in terms of structural consistency with the generated images. To address this gap, we propose LayoutDualGAN, a layout-conditioned GAN that synthesizes both high-fidelity nanoparticle images and structurally more consistent masks. Our approach features a dual-path generator comprising an instance-level mask generator and a mask-aware image generator. It enforces a tight coupling between generated images and their corresponding masks, guaranteeing enhanced image-mask consistency. Additionally, we design a robust multilevel discriminator that boosts discrimination feedback across image and instance levels, ensuring approximately realistic image synthesis. Experimental results demonstrate that the proposed model, trained on an extremely small dataset, outperforms existing methods in terms of both image quality and mask accuracy for the task of annotated nanoparticle image generation.

CT image reconstruction is a challenging inverse problem of estimating image intensities from sensor signals. While deep learning-based methods hold promise, they are typically classified into fully-supervised, self-supervised, and case-by-case approaches, each offering distinct advantages and limitations. Among these, case-by-case reconstruction techniques, exemplified by models like NeRP, provide the advantage of flexible data-sampling and image sizes, making them well-suited for the complex CT inverse problem. However, existing case-by-case models encounter difficulties in achieving precise reconstructions, particularly in sparse-view scenarios. To address this limitation, we propose tomo-NeRL, a novel implicit reconstruction model that incorporates tomographic information of individual pixels within the reconstruction network. By harnessing this tomo-graphic data for implicit inverse learning, tomo-NeRL enhances reconstruction quality while effectively mitigating artifacts. The core innovation of tomo-NeRL lies in defining and extracting tomographic insights, such as visual characteristics and spatial variations specific to tomographic imaging, from the sinogram data for individual pixels. Moreover, our work introduces an adaptive inversion framework that amalgamates tomographic information from diverse projection angles, constructing a latent feature space that enhances the reconstruction process. Extensive experiments unequivocally showcase tomo-NeRL’s outstanding reconstruction performance, excelling in artifact suppression and detail preservation, especially in sparse-view scenarios.

Automated biomedical image analysis, such as nanoparticle segmentation and phenotyping for drug discovery, requires robust learning models that are trained on large datasets of microscope images with appropriate annotations. However, current image synthesis techniques have limited performance in terms of diversity and fidelity, which can impact the accuracy of these models. Additionally, most learning-based image synthesis methods require a significant amount of training data for successful model convergence, resulting in training challenges on limited data. To address these limitations, we propose a layout-controllable conditional GAN with multi-level discriminators to synthesize more diverse and realistic homogeneous nanoparticle image-mask pairs with minimal guidance from training data. Our approach utilizes a variant image layout as the condition for the generator to vary the spatial distribution of nanoparticles in an image, generating image-mask pairs with more diversity. We also design image-level and object-level discriminators for multilevel discrimination, with loss constraints on images, masks and their correlations, ensuring high-fidelity and accurate synthesis. Experimental results demonstrate that the proposed network, trained on a small dataset, is capable of generating homogeneous nanoparticle image-mask pairs with sufficient diversity and fidelity.

This paper addresses the challenges faced in computed tomography (CT) reconstruction, including artifacts and poor image quality due to imaging imperfections like under-sampling. While deep learning-based CT reconstruction models have emerged to combat these issues, they often lack precision and interpretability when employing end-to-end neural networks to directly transform signals from sinograms into CT images. Additionally, conventional supervised methods for guiding network training face hurdles in obtaining supervision information from CT images, particularly in biomedical imaging contexts. To surmount these limitations, we introduce a novel self-supervised CT reconstruction approach employing an Adaptive Domain Transformation (ADT) module. The ADT module reverses the sinogram signals and translates them into tomographic insights through a Tomographic Inverse Transform component, followed by a Tomographic Attention component that adeptly leverages this information to reconstruct CT image intensities. Further quality enhancement is achieved with a refinement network, and notably, the Radon Transform is used to process the reconstructed CT images, generating simulated sinograms for loss calculation without the need for extra supervision from CT images. Empirical evidence supports the efficacy of our approach, showcasing tangible enhancements in both the reconstruction of anatomical structures and the mitigation of artifacts.

Automated segmentation of organs and lesions is crucial for effective treatment design and care planning in the medical field. However, accurately segmenting organs and lesions is challenging due to the complex distribution of lesions within organs, such as cases with early diseases characterized by small volumes and blurred boundaries. In this work, we propose Med-DualNet, a dual-branch segmentation network that incorporates Hard-Patch Mining and Joint Error Adjustment strategies to address these challenges. Branch I focuses on organ segmentation, while Branch II targets challenging lesion segmentation using a two-stage network and dedicated strategies. The Hard-Patch Mining strategy in Stage-one dynamically generates a subset of image patches that highlight challenging aspects of lesion segmentation, including small volumes and blurred boundaries. Furthermore, the Joint Error Adjustment strategy in Stage-two facilitates the learning of challenging lesion features. Experimental results on three public medical datasets demonstrate the improved performance of the proposed model in organ and lesion segmentation, as measured by visual and quantitative metrics.

Automated nanoparticle phenotyping is a critical aspect of high-throughput drug research, which requires analyzing nanoparticle size, shape, and surface topography from microscopy images. To automate this process, we present an instance segmentation pipeline that partitions individual nanoparticles on microscopy images. Our pipeline makes two key contributions. Firstly, we synthesize diverse and approximately realistic nanoparticle images to improve robust learning. Secondly, we improve the BlendMask model to segment tiny, overlapping, or sparse particle images. Specifically, we propose a parameterized approach for generating novel pairs of single particles and their masks, encouraging greater diversity in the training data. To synthesize more realistic particle images, we explore three particle placement rules and an image selection criterion. The improved one-stage instance segmentation network extracts distinctive features of nanoparticles and their context at both local and global levels, which addresses the data challenges associated with tiny, overlapping, or sparse nanoparticles. Extensive experiments demonstrate the effectiveness of our pipeline for automating nanoparticle partitioning and phenotyping in drug research using microscopy images.

With the aim of screening the prognostic genes for breast cancer (BRCA) and exploring the possible mechanism and clinical value of these genes in the growth and regression stage of disease, we study the genes in the public gene expression omnibus (GEO) GSE22820 and the cancer genome atlas (TCGA). To achieve high-confidence gene candidates for BRCA, we present a hybrid gene and module analysis pipeline that strategically considers data mining on different datasets. Ultimately, four gene candidates, i.e., PLIN1, GPD1, LIPE, and CHRDL1, are targeted for BRCA. Afterwards, Kaplan-Meier survival analysis is performed on these genes for verification, revealing that the overall survival time of patients with low expression of these genes was shorter than that of patients with high expression (with P<0.05). Moreover, in order to study the role of these genes in the mechanisms and functionality related to cytoplasmic lipid metabolism, functional enrichment and pathway analysis are implemented. The results indicate that the expression of the four discovered genes plays an adverse role in BRCA development and could serve as effective biomarkers for predicting the formation and progression of BRCA.

In the high-throughput drug research, statistical analysis of nanoparticles has been one of the focuses in drug carrier systems. This can be completed via electronic microscopy imaging and image analysis such as image segmentation. For example, in some cases selecting and counting the nanoparticles in the field of view are important for drug screening. In order to minimize manual interactions and avoid extensive workloads, we present a pipeline that is featured by deep learning-based instance segmentation, with experiments implemented on both real and synthetic data. The proposed instance segmentation approach, namely Modified BlendMask, aims to improve the accuracy of nanoparticle detection and further refine the automated nanoparticle count. In this framework, we are devoted to address the problem of missing detection, i.e., false negative, introduced by overlap and blur, sparse particle distribution, and tiny particle occurrence in images. Sufficient experiments demonstrate the reasonableness and effectiveness of the proposed pipeline and instance segmentation method for automated nanoparticle count, with an overall count accuracy of 70.2% for 38670 particles on the 1141 test images.

Nowadays, deep convolutional neural networks are playing an increasingly important role in single-image super-resolution vision applications. Yet, most of the existing deep convolution-based methodologies are insufficiently intelligent to capture targeted information when the distribution of spatial and channel information is uneven for low-resolution images. To address this research issue, we propose a cascade attention blend residual network, with the non-local channel and multi-scale attention being considered for channel-wise dependencies and multi-scale receptive fields, respectively. Cascading both attentions in a potent blend residual block aims to learn more spatial and channel correlations between low- and super-resolution images. Experimental results demonstrate that the proposed method achieves promising performance for super-resolution image reconstruction, as well as gains an average reduction of 50.9% network parameters, compared to some state-of-the-art methods.

Visible-infrared person re-identification (VI-ReID) is a challenging problem that aims to match pedestrians captured by visible and infrared cameras. Prevailing methods in this field mainly focus on learning sharable feature representations from the last layer of deep convolution neural networks(CNNs). However, due to the large intra-modality variations and cross-modality variations, the last layer’s sharable feature representations are less discriminative. To remedy this, we propose a novel Two-Phase Feature Fusion Network(TFFN) to enhance the discriminative feature learning via feature fusion. Specifically, TFFN contains two fusion modules: (1) Multi-Level Fusion Module(MLFM) that reweights and fuses intra-modality multi-level features to utilize high- and low-level information; (2) Graph-Level Fusion Module (GLFM) that mines and fuses rich mutual information across the two modalities by employing a cross-modality graph attention network. Additionally, for effective fusion, we develop a deep supervision method to enhance the discrimination of pre-fusion features and eliminate noise information. Extensive experiments show that TFFN outperforms the state-of-the-art methods on two mainstream VI-ReID datasets: SYSU-MM01 and RegDB.

Instance segmentation of scanning electron microscope images provides useful information for quantitative analysis of particle morphometry and distribution that contributes to various biomedical research such as the phenotyping of drug delivery systems. Compared to the conventional segmentation methodologies, the learning-based approaches stand out, benefiting from the prosperous development of artificial intelligence. However, most of the current learning-based segmentation methods require sufficient manually annotated training data, which is considered to be laborious. To alleviate this problem, we present a novel weakly supervised framework for instance segmentation on scanning electron microscope images. In the proposed framework, only one instance from each raw image is manually labeled to generate a synthesized dataset, which will be further used to select the training set. With the weakly annotated training dataset, the instance segmentation network is trained and applied to segment the particles of raw testing images. Based on our experimental results, the trained network gains 75% recall and 74% average precision on the tested images, which is seen as a reasonable performance considering the data complexity in our research. The overall experiments demonstrate that the proposed weakly supervised framework is able to provide an efficient solution to the instance segmentation of biomedical images.

OBJECTIVE: We describe how electronic health record (EHR) audit files can be used to understand how time is spent in primary care (PC). MATERIALS/METHODS: We used audit file data from the Geisinger Clinic to quantify elements of the clinical workflow and to determine how these times vary by patient and encounter factors. We randomly selected audit file records representing 36 437 PC encounters across 26 clinic locations. Audit file data were used to estimate duration and variance of: (1) time in the waiting room, (2) nurse time with the patient, (3) time in the exam room without a nurse or physician, and (4) physician time with the patient. Multivariate modeling was used to test for differences by patient and by encounter features. RESULTS: On average, a PC encounter took 54.6 minutes, with 5 minutes of nurse time, 15.5 minutes of physician time, and the remaining 62% of the time spent waiting to see a clinician or check out. Older age, female sex, and chronic disease were associated with longer wait times and longer time with clinicians. Level of service and numbers of medications, procedures, and lab orders were associated with longer time with clinicians. Late check-in and same-day visits were associated with shorter wait time and clinician time. CONCLUSIONS: This study provides insights on uses of audit file data for workflow analysis during PC encounters. DISCUSSION: Scalable ways to quantify clinical encounter workflow elements may provide the means to develop more efficient approaches to care and improve the patient experience.

In order to preserve sufficient fluorescence intensity and improve the quality of fluorescence images in optical projection tomography (OPT) imaging, a feasible acquisition solution is to temporally formalize the fluorescence and bright-field imaging procedure as two consecutive phases. To be specific, fluorescence images are acquired first, in a full axial-view revolution, followed by the bright-field images. Due to the mechanical drift, this approach, however, may suffer from a deviation of center of rotation (COR) for the two imaging phases, resulting in irregular 3D image fusion, with which gene or protein activity may be located inaccurately. In this paper, we address this problem and consider it into a framework based on sinogram unification so as to precisely fuse 3D images from different channels for CORs between channels that are not coincident or if COR is not in the center of sinogram. The former case corresponds to the COR deviation above; while the latter one correlates with COR alignment, without which artefacts will be introduced in the reconstructed results. After sinogram unification, inverse radon transform can be implemented on each channel to reconstruct the 3D image. The fusion results are acquired by mapping the 3D images from different channels into a common space. Experimental results indicate that the proposed framework gains excellent performance in 3D image fusion from different channels. For the COR alignment, a new automated method based on interest point detection and included in sinogram unification, is presented. It outperforms traditional COR alignment approaches in combination of effectiveness and computational complexity.