Deep neural networks have advanced semi-supervised learning for biomedical image segmentation. Few models typically focus on intrinsic low- and high-frequency (LF and HF) information to improve performance. Introducing frequency information as perturbation avoids the negative impact on consistency-based semi-supervised models caused by artificial perturbations. Furthermore, when the image lacks HF information, typical wavelet models XNet and XNetv2 will lack effective input information and perturbation. XNetv2 also lacks sufficient fusion due to network perturbation. We propose an asymmetric model IYNet to introduce network perturbation, including a dual-branch and a single-branch network. IYNet perform wavelet-based image-level fusion of LF and HF information to avoid vanishing input perturbation and achieve complementary information. The fusion results and the raw image will be used as input for the dual-branch and single-branch networks, respectively. Furthermore, we introduce a fusion module based on channel and spatial attention across all scales to improve LF and HF information transfer. We validate the effectiveness of IYNet with varying proportions of LF and HF information. IYNet outperforms other state-of-the-art (SOTA) models in semi-supervision. Moreover, IYNet can be competitive with SOTA fully-supervised models. IYNet Code is available at https://github.com/withNamer/IYNet.

Computed tomography angiography (CTA) plays a crucial role in the diagnosis of thoracic vascular diseases, yet the need for iodinated contrast agents raises safety and accessibility concerns. Generating synthetic contrast-enhanced CT (CECT) from non-contrast CT (NCCT) offers a promising solution to these challenges. However, existing generative methods often struggle to preserve fine vascular details and continuity along the anisotropic z-axis in volumetric data. In this work, we present AngioDiff, a novel diffusion-based framework for high-fidelity CTA synthesis from NCCT. Our method leverages a conditional diffusion model with a mean reverting prior and incorporates a sliding window mechanism with asynchronous denoising to effectively model large-scale 3D CT volumes. To further stream-line the process and address edge artifacts, we introduce a sequence padding strategy that simplifies training and sampling while enhancing structural continuity. Furthermore, our network design combines spatial and axial attention modules to adequately capture intra-slice and inter-slice dependencies. Comprehensive experiments on multi-center datasets demonstrate that AngioDiff consistently outperforms state-of-the-art methods in both 2D slice-based and 3D volume-based quantitative metrics, achieving superior anatomical fidelity and volumetric consistency. This work highlights the clinical potential of diffusion-based models for agent-free CTA, offering a safer and more accessible alter-native to traditional imaging protocols.

Ultrasound videos are an important form of clinical imaging data, and deep learning-based analysis can improve diagnostic accuracy and clinical efficiency. However, the scarcity of labeled data and the inherent challenges of video analysis have impeded the advancement of related methods. In this work, we introduce E-ViM3, a data-efficient Vision Mamba network that preserves the 3D structure of video data, enhancing long-range dependencies and inductive biases to better model spatial-temporal correlations. With our design of Enclosure Global Tokens (EG T), the model captures and aggregates global features more effectively than competing methods. We further employ a tailored masked video modeling approach for self-supervised pre-training to enhance its data efficiency, with the proposed Spatial- Temporal Chained (STC) masking strategy designed to adapt to different video scenarios. Experiments demonstrate that E-ViM3 achieves state-of-the-art performance on different tasks across four datasets of varying sizes: EchoNet-Dynamic, CAMUS, MICCAI-BUV, and WHBUS. Furthermore, our model attains competitive results even with limited labeled data, highlighting its potential impact on real-world clinical applications. Codes are available at https://github.com/HenryZhou19/E-ViM3.

In applications of diffusion models, controllable generation is of practical significance, but is also challenging. Current methods for controllable generation primarily focus on modifying the score function of diffusion models, while Mean Reverting (MR) Diffusion directly modifies the structure of the stochastic differential equation (SDE), making the incorporation of image conditions simpler and more natural. However, current training-free fast samplers are not directly applicable to MR Diffusion. And thus MR Diffusion requires hundreds of NFEs (number of function evaluations) to obtain high-quality samples. In this paper, we propose a new algorithm named MaRS (MR Sampler) to reduce the sampling NFEs of MR Diffusion. We solve the reverse-time SDE and the probability flow ordinary differential equation (PF-ODE) associated with MR Diffusion, and derive semi-analytical solutions. The solutions consist of an analytical function and an integral parameterized by a neural network. Based on this solution, we can generate high-quality samples in fewer steps. Our approach does not require training and supports all mainstream parameterizations, including noise prediction, data prediction and velocity prediction. Extensive experiments demonstrate that MR Sampler maintains high sampling quality with a speedup of 10 to 20 times across ten different image restoration tasks. Our algorithm accelerates the sampling procedure of MR Diffusion, making it more practical in controllable generation.

Breast cancer remains a leading cause of death among women, with early detection significantly improving prognosis. Non-contrast computed tomography (NCCT) scans of the chest, routinely acquired for thoracic assessments, often capture the breast region incidentally, presenting an underexplored opportunity for opportunistic breast lesion detection without additional imaging cost or radiation. However, the subtle appearance of lesions in NCCT and the difficulty of jointly modeling lesion detection and malignancy classification pose unique challenges. In this work, we propose Dual-Res Tandem Mamba-3D (DRT-M3D), a novel multitask framework for opportunistic breast cancer analysis on NCCT scans. DRT-M3D introduces a dual-resolution architecture, which captures fine-grained spatial details for segmentation-based lesion detection and global contextual features for breast-level cancer classification. It further incorporates a tandem input mechanism that models bilateral breast regions jointly through Mamba-3D blocks, enabling cross-breast feature interaction by leveraging subtle asymmetries between the two sides. Our approach achieves state-of-the-art performance in both tasks across multi-institutional NCCT datasets spanning four medical centers. Extensive experiments and ablation studies validate the effectiveness of each key component.

Accurate particle tracking is essential for studying intracellular dynamics. While deep learning has advanced tracking performance, its reliance on large labeled datasets hampers generalization to unlabeled biological data. To bridge this gap, we propose a novel particle tracking enhancement framework that designed for annotation-scarce scenarios. At its core is ParticleDiff, a conditional diffusion-based trajectory generator tailored for biological motion. By leveraging historical trajectories as context, ParticleDiff conditionally predicts future positions in an autoregressive loop, ensuring both temporal coherence and biological plausibility. These generated trajectories are used to augment training for downstream tracking models, improving accuracy in data-scarce scenarios. Experiments demonstrate that trajectories generated by ParticleDiff closely resemble real biological data. As a result, tracking models trained on ParticleDiff-augmented data consistently outperform those using synthetic trajectories from other generators, achieving performance comparable to models trained on real annotations. Our framework significantly improves tracking accuracy while reducing reliance on labeled data, thereby enhancing the generalization of deep learning-based tracking models in life science applications. Code is available at https://github.com/imzhangyd/ParticleDiff.

MOTIVATION: Ensuring connectivity and preventing fractures in tubular object segmentation are critical for downstream analyses. Despite advancements in deep neural networks that have significantly improved tubular object segmentation, existing methods still face limitations. They often rely heavily on precise annotations, hindering their scalability to large-scale unlabeled image datasets. Additionally, current evaluation metrics are insufficient for effectively capturing segmentation fractures. RESULTS: To address these challenges, we propose a semi-supervised fracture-attention model (SSFA) for tubular object segmentation. SSFA enhances connectivity, reduces fractures, and maintains volumetric accuracy. It outperforms state-of-the-art models in topological performance. Extensive experiments on four public datasets validate the effectiveness of SSFA. Furthermore, we introduce a novel evaluation metric, the fracture rate, which provides an intuitive and quantitative assessment of segmentation fractures. AVAILABILITY AND IMPLEMENTATION: Our source code is available at http://github.com/Yanfeng-Zhou/SSFA.

Semantic segmentation of electron microscopy (EM) images is crucial for nanoscale analysis. With the development of deep neural networks (DNNs), semantic segmentation of EM images has achieved remarkable success. However, current EM image segmentation models are usually extensions or adaptations of natural or biomedical models. They lack the full exploration and utilization of the intrinsic characteristics of EM images. Furthermore, they are often designed only for several specific segmentation objects and lack versatility. In this study, we quantitatively analyze the characteristics of EM images compared with those of natural and other biomedical images via the wavelet transform. To better utilize these characteristics, we design a high-frequency (HF) fusion network, GobletNet, which outperforms state-of-the-art models by a large margin in the semantic segmentation of EM images. We use the wavelet transform to generate HF images as extra inputs and use an extra encoding branch to extract HF information. Furthermore, we introduce a fusion-attention module (FAM) into GobletNet to facilitate better absorption and fusion of information from raw images and HF images. Extensive benchmarking on seven public EM datasets (EPFL, CREMI, SNEMI3D, UroCell, MitoEM, Nanowire and BetaSeg) demonstrates the effectiveness of our model. The code is available at https://github.com/Yanfeng-Zhou/GobletNet.

3D tubular objects have complex spatial shapes. Direct volume visualization cannot intuitively display their morphological characteristics and surface abnormalities. Straightening reformation is an effective visualization method for tubular objects. It uses a swept frame to sample cross sections along the centerline of the tubular object to generate straightening result. So far, however, current methods cannot visualize the full 3D view and fail to interface with downstream morphological analysis. Furthermore, current swept frames impose strict restrictions on the shape of tubular objects and are computationally expensive. In this study, we propose a novel swept frame based on vector rotation and construct an automatic straightening reformation pipeline. Our method is applicable to various tubular objects and can be efficiently executed recursively while ensuring that the straightening results have no rotation bias. Extensive experiments on eight different tubular objects and quantitative evaluation on various downstream applications demonstrate the effectiveness and universality of our straightening pipeline. Code is available at https://github.com/Yanfeng-Zhou/SATO.

XNet introduces a wavelet-based X-shaped unified architecture for fully-and semi-supervised biomedical segmentation. So far, however, XNet still faces the limitations, including performance degradation when images lack high-frequency (HF) information, underutilization of raw images and insufficient fusion. To address these issues, we propose XNet v2, a low-and high-frequency complementary model. XNet v2 performs wavelet-based image-level complementary fusion, using fusion results along with raw images inputs three different sub-networks to construct consistency loss. Furthermore, we introduce a feature-level fusion module to enhance the transfer of low-frequency (LF) information and HF information. XNet v2 achieves state-of-the-art in semi-supervised segmentation while maintaining competitive results in fully-supervised learning. More importantly, XNet v2 excels in scenarios where XNet fails. Compared to XNet, XNet v2 exhibits fewer limitations, better results and greater universality. Extensive experiments on three 2D and two 3D datasets demonstrate the effectiveness of XNet v2. Code is available at https://github.com/Yanfeng-Zhou/XNetv2.

Unsupervised Domain Adaptation (UDA) is a learning technique that transfers knowledge learned in the source domain from labelled training data to the target domain with only unlabelled data. It is of significant importance to medical image segmentation because of the usual lack of labelled training data. Although extensive efforts have been made to optimize UDA techniques to improve the accuracy of segmentation models in the target domain, few studies have addressed the robustness of these models under UDA. In this study, we propose a two-stage training strategy for robust domain adaptation. In the source training stage, we utilize adversarial sample augmentation to enhance the robustness and generalization capability of the source model. And in the target training stage, we propose a novel robust pseudo-label and pseudo-boundary (PLPB) method, which effectively utilizes unlabeled target data to generate pseudo labels and pseudo boundaries that enable model self-adaptation without requiring source data. Extensive experimental results on cross-domain fundus image segmentation confirm the effectiveness and versatility of our method. Source code of this study is openly accessible at https://github.com/LinGrayy/PLPB.

MOTIVATION: Intracellular organelle networks (IONs) such as the endoplasmic reticulum (ER) network and the mitochondrial (MITO) network serve crucial physiological functions. The morphology of these networks plays a critical role in mediating their functions. Accurate image segmentation is required for analyzing the morphology and topology of these networks for applications such as molecular mechanism analysis and drug target screening. So far, however, progress has been hindered by their structural complexity and density. RESULTS: In this study, we first establish a rigorous performance baseline for accurate segmentation of these organelle networks from fluorescence microscopy images by optimizing a baseline U-Net model. We then develop the multi-resolution encoder (MRE) and the hierarchical fusion loss (Lhf) based on two inductive components, namely low-level features and topological self-similarity, to assist the model in better adapting to the task of segmenting IONs. Empowered by MRE and Lhf, both U-Net and Pyramid Vision Transformer (PVT) outperform competing state-of-the-art models such as U-Net++, HR-Net, nnU-Net, and TransUNet on custom datasets of the ER network and the MITO network, as well as on public datasets of another biological network, the retinal blood vessel network. In addition, integrating MRE and Lhf with models such as HR-Net and TransUNet also enhances their segmentation performance. These experimental results confirm the generalization capability and potential of our approach. Furthermore, accurate segmentation of the ER network enables analysis that provides novel insights into its dynamic morphological and topological properties. AVAILABILITY AND IMPLEMENTATION: Code and data are openly accessible at https://github.com/cbmi-group/MRE.

BACKGROUND: Fluorescence microscopy (FM) is an important and widely adopted biological imaging technique. Segmentation is often the first step in quantitative analysis of FM images. Deep neural networks (DNNs) have become the state-of-the-art tools for image segmentation. However, their performance on natural images may collapse under certain image corruptions or adversarial attacks. This poses real risks to their deployment in real-world applications. Although the robustness of DNN models in segmenting natural images has been studied extensively, their robustness in segmenting FM images remains poorly understood RESULTS: To address this deficiency, we have developed an assay that benchmarks robustness of DNN segmentation models using datasets of realistic synthetic 2D FM images with precisely controlled corruptions or adversarial attacks. Using this assay, we have benchmarked robustness of ten representative models such as DeepLab and Vision Transformer. We find that models with good robustness on natural images may perform poorly on FM images. We also find new robustness properties of DNN models and new connections between their corruption robustness and adversarial robustness. To further assess the robustness of the selected models, we have also benchmarked them on real microscopy images of different modalities without using simulated degradation. The results are consistent with those obtained on the realistic synthetic images, confirming the fidelity and reliability of our image synthesis method as well as the effectiveness of our assay. CONCLUSIONS: Based on comprehensive benchmarking experiments, we have found distinct robustness properties of deep neural networks in semantic segmentation of FM images. Based on the findings, we have made specific recommendations on selection and design of robust models for FM image segmentation.

Fully- and semi-supervised semantic segmentation of biomedical images have been advanced with the development of deep neural networks (DNNs). So far, however, DNN models are usually designed to support one of these two learning schemes, unified models that support both fully- and semi-supervised segmentation remain limited. Furthermore, few fully-supervised models focus on the intrinsic low frequency (LF) and high frequency (HF) information of images to improve performance. Perturbations in consistency-based semi-supervised models are often artificially designed. They may introduce negative learning bias that are not beneficial for training. In this study, we propose a wavelet-based LF and HF fusion model XNet, which supports both fully- and semi-supervised semantic segmentation and outperforms state-of-the-art models in both fields. It emphasizes extracting LF and HF information for consistency training to alleviate the learning bias caused by artificial perturbations. Extensive experiments on two 2D and two 3D datasets demonstrate the effectiveness of our model. Code is available at https://github.com/Yanfeng-Zhou/XNet.

Deep neural networks are susceptible to adversarial examples, posing a significant security risk in critical applications. Adversarial Training (AT) is a well-established technique to enhance adversarial robustness, but it often comes at the cost of decreased generalization ability. This paper proposes Robustness Critical Fine-Tuning (RiFT), a novel approach to enhance generalization without compromising adversarial robustness. The core idea of RiFT is to exploit the redundant capacity for robustness by fine-tuning the adversarially trained model on its non-robust-critical module. To do so, we introduce module robust criticality (MRC), a measure that evaluates the significance of a given module to model robustness under worst-case weight perturbations. Using this measure, we identify the module with the lowest MRC value as the non-robust-critical module and fine-tune its weights to obtain fine-tuned weights. Subsequently, we linearly interpolate between the adversarially trained weights and fine-tuned weights to derive the optimal fine-tuned model weights. We demonstrate the efficacy of RiFT on ResNet18, ResNet34, and WideResNet34-10 models trained on CIFAR10, CIFAR100, and Tiny-ImageNet datasets. Our experiments show that RiFT can significantly improve both generalization and out-of-distribution robustness by around 1.5% while maintaining or even slightly enhancing adversarial robustness. Code is available at https://github.com/Immortalise/RiFT.

The scarcity of annotated data, particularly for rare diseases, limits the variability of training data and the range of detectable lesions, presenting a significant challenge for supervised anomaly detection in medical imaging. To solve this problem, we propose a novel unsupervised method for medical image anomaly detection: Attention-Augmented Differentiable top-k Feature Adaptation (ADFA). The method utilizes Wide-ResNet50-2 (WR50) network pre-trained on ImageNet to extract initial feature representations. To reduce the channel dimensionality while preserving relevant channel information, we employ an attention-augmented patch descriptor on the extracted features. We then apply differentiable top-k feature adaptation to train the patch descriptor, mapping the extracted feature representations to a new vector space, enabling effective detection of anomalies. Experiments show that ADFA outperforms state-of-the-art (SOTA) methods on multiple challenging medical image datasets, confirming its effectiveness in medical anomaly detection.

How deep neural networks (DNNs) learn from noisy labels has been studied extensively in image classification but much less in image segmentation. So far, our understanding of the learning behavior of DNNs trained by noisy segmentation labels remains limited. In this study, we address this deficiency in both binary segmentation of biological microscopy images and multi-class segmentation of natural images. We generate extremely noisy labels by randomly sampling a small fraction (e.g., 10%) or flipping a large fraction (e.g., 90%) of the ground truth labels. When trained with these noisy labels, DNNs provide largely the same segmentation performance as trained by the original ground truth. This indicates that DNNs learn structures hidden in labels rather than pixel-level labels per se in their supervised training for semantic segmentation. We refer to these hidden structures in labels as meta-structures. When DNNs are trained by labels with different perturbations to the meta-structure, we find consistent degradation in their segmentation performance. In contrast, incorporation of meta-structure information substantially improves performance of an unsupervised segmentation model developed for binary semantic segmentation. We define meta-structures mathematically as spatial density distributions and show both theoretically and experimentally how this formulation explains key observed learning behavior of DNNs.

To identify and localize macromolecules of interest in crowded intracellular environment, the low signal-to-noise ratio and missing imaging wedge of cryo-electron tomography (cryo-ET) data pose substantial technical challenges. Currently, mainstream approaches of 3D particle picking in cryo-ET either follow the ‘segment-then-cluster’ strategy, or extract potential structural regions as sub-tomograms and then perform classification. Different from these two-step methods, we solve the problem using a one-step instance segmentation approach, termed 3D-SOLOv2. Specifically, the category and mask of each 3D particle are predicted according to the particle’s location and size. To solve the lack of real masks for 3D particles in cryo-ET, a Gaussian-shaped mask is proposed to approximate real masks. When tested on simulated datasets of SHREC2020 challenge, our model achieves the fastest inference speed and the state-of-the-art performance for both localization and classification tasks. When tested on real cryo-ET dataset of EMPIAR-10045, our model also achieves better performance than other methods.

Accurate analysis of the dynamic behavior of particles in living cells plays an important role in understanding the biological mechanism. One of the key steps of this task is to describe quantitatively the particles' stochastic switching process between different motion states based on single-particle tracking (SPT) trajectories. However, there is still a lack of robust solutions for discriminating different motion states within one trajectory. Researchers just made preliminary attempts and they have not taken the coherence within one state and dissimilarity between different states into consideration. Here, we propose a novel local attention-based long short-term memory (LSTM) neural network for motion switching detection in an end-to-end fashion, directly output the types of the motion states and corresponding starting and ending time of each motion state. The results of the experiments demonstrate that our method outperforms the state-of-the-art particle mobility analysis approach, effectively improving the performance with simulated data. Experiments on in-house datasets also demonstrate accurate descriptions of the dynamic mode of particles, making it possible to answer the important biological questions considered inaccessible before. The code and data used in this paper are publicly available at GitHub: https://github.com/youxiaobo/Att-BiLSTM.

MOTIVATION: Cryoelectron tomography (cryo-ET) visualizes structure and spatial organization of macromolecules and their interactions with other subcellular components inside single cells in the close-to-native state at submolecular resolution. Such information is critical for the accurate understanding of cellular processes. However, subtomogram classification remains one of the major challenges for the systematic recognition and recovery of the macromolecule structures in cryo-ET because of imaging limits and data quantity. Recently, deep learning has significantly improved the throughput and accuracy of large-scale subtomogram classification. However, often it is difficult to get enough high-quality annotated subtomogram data for supervised training due to the enormous expense of labeling. To tackle this problem, it is beneficial to utilize another already annotated dataset to assist the training process. However, due to the discrepancy of image intensity distribution between source domain and target domain, the model trained on subtomograms in source domain may perform poorly in predicting subtomogram classes in the target domain. RESULTS: In this article, we adapt a few shot domain adaptation method for deep learning-based cross-domain subtomogram classification. The essential idea of our method consists of two parts: (i) take full advantage of the distribution of plentiful unlabeled target domain data, and (ii) exploit the correlation between the whole source domain dataset and few labeled target domain data. Experiments conducted on simulated and real datasets show that our method achieves significant improvement on cross domain subtomogram classification compared with baseline methods. AVAILABILITY AND IMPLEMENTATION: Software is available online https://github.com/xulabs/aitom. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Synthetic images are widely used in image segmentation for algorithm training and performance assessment. Recently, advances in image synthesis techniques, especially generative adversarial networks (GANs), have made it possible to generate fluorescence microscopy images with remarkably realistic appearance. However, intuitive and specific metrics to assess the quality of these images remain lacking. Here, we propose three quality metrics that quantify the fidelity of the foreground signal, the background noise, and blurring, respectively, of synthesized fluorescence microscopy images. Using these metrics, we examine images of mitochondria synthesized by two representative GANs: pix2pix, which requires paired training data, and CycleGAN, which does not require paired training data. We find that both networks generate realistic images and achieve similar fidelity in reproducing background noise and blurring of real images. However, CycleGAN achieves significantly higher fidelity than pix2pix in reproducing intensity patterns of real mitochondria. When used to train the U-Net for segmentation, images synthesized by both networks achieve performance on par with real images. Overall, we have developed a method to assess quality of synthetic fluorescence microscopy images and to evaluate their training performance in image segmentation. The quality metrics proposed are general and can be used to assess fluorescence microscopy images synthesized by different methods.

Convolutional neural networks (CNNs) recently have achieved remarkable success in segmentation of biological fluorescence microscopy images. Because many of these networks were developed initially for general computer vison tasks such as object detection and object recognition, it is necessary to characterize their performance to determine how they meet the needs of related biological studies. So far, performance characterization of such networks has focused primarily on segmentation accuracy. It remains unclear how different networks compare in their robustness in handling images of different conditions and their sensitivity in detecting subtle geometrical changes of biological structures. Here, we develop a method that uses realistic synthetic images to characterize the robustness and sensitivity of such networks. We use the method to compare the performance of two widely adopted CNNs: the fully convolutional network (FCN) and the U-Net, in segmentation of complex morphology of mitochondria. We also compare them against an adaptive active-mask algorithm in performance. We find that both networks outperform the adaptive active-mask algorithm in robustness and sensitivity and that U-Net outperforms FCN Overall, our study provides new insights into the performance of CNNs in segmentation of fluorescence microscopy images.

Neurons depend critically on active transport of cargoes throughout their complex neurite networks for their survival and function. Defects in this process have been strongly associated with many human neurodevelopmental and neurodegenerative diseases. To understand related neuronal physiology and disease mechanisms, it is essential to measure the traffic flow within the neurite networks. Currently, however, image analysis methods required for this measurement are lacking. To address this deficiency, we developed a method that could measure the flow rates of cargo traffic at any specified locations along individual branches of the neurite networks. Our method is based on detecting and counting cargo trajectories passing through the specified locations of measurement in kymographs, which are spatiotemporal maps of cargo movement within one-dimensional neurites. A main focus of our method development is robust performance, which ensures that our method works reliably and accurately under low signal-to-noise ratios. We validated and benchmarked our method using both synthetic and actual image data and found its accuracy to be >85% on average under normal conditions. Our method can be used to measure traffic flow in not just neurite networks but also other intracellular networks such as cytoskeletal filament networks.

MOTIVATION: Cellular Electron CryoTomography (CECT) enables 3D visualization of cellular organization at near-native state and in sub-molecular resolution, making it a powerful tool for analyzing structures of macromolecular complexes and their spatial organizations inside single cells. However, high degree of structural complexity together with practical imaging limitations makes the systematic de novo discovery of structures within cells challenging. It would likely require averaging and classifying millions of subtomograms potentially containing hundreds of highly heterogeneous structural classes. Although it is no longer difficult to acquire CECT data containing such amount of subtomograms due to advances in data acquisition automation, existing computational approaches have very limited scalability or discrimination ability, making them incapable of processing such amount of data. RESULTS: To complement existing approaches, in this article we propose a new approach for subdividing subtomograms into smaller but relatively homogeneous subsets. The structures in these subsets can then be separately recovered using existing computation intensive methods. Our approach is based on supervised structural feature extraction using deep learning, in combination with unsupervised clustering and reference-free classification. Our experiments show that, compared with existing unsupervised rotation invariant feature and pose-normalization based approaches, our new approach achieves significant improvements in both discrimination ability and scalability. More importantly, our new approach is able to discover new structural classes and recover structures that do not exist in training data. AVAILABILITY AND IMPLEMENTATION: Source code freely available at http://www.cs.cmu.edu/∼mxu1/software . CONTACT: [email protected]. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

MOTIVATION: Quantitative shape analysis is required by a wide range of biological studies across diverse scales, ranging from molecules to cells and organisms. In particular, high-throughput and systems-level studies of biological structures and functions have started to produce large volumes of complex high-dimensional shape data. Analysis and understanding of high-dimensional biological shape data require dimension-reduction techniques. RESULTS: We have developed a technique for non-linear dimension reduction of 2D and 3D biological shape representations on their Riemannian spaces. A key feature of this technique is that it preserves distances between different shapes in an embedded low-dimensional shape space. We demonstrate an application of this technique by combining it with non-linear mean-shift clustering on the Riemannian spaces for unsupervised clustering of shapes of cellular organelles and proteins. AVAILABILITY AND IMPLEMENTATION: Source code and data for reproducing results of this article are freely available at https://github.com/ccdlcmu/shape_component_analysis_Matlab The implementation was made in MATLAB and supported on MS Windows, Linux and Mac OS. CONTACT: [email protected].

Super-resolution localization microscopy (SRLM) is a new imaging modality that is capable of resolving cellular structures at nanometer resolution, providing unprecedented insight into biological processes. Each SRLM image is reconstructed from a time series of images of randomly activated fluorophores that are localized at nanometer resolution and represented by clusters of particles of varying spatial densities. SRLM images differ significantly from conventional fluorescence microscopy images because of fundamental differences in image formation. Currently, however, quantitative image analysis techniques developed or optimized specifically for SRLM images are lacking, which significantly limit accurate and reliable image analysis. This is especially the case for image segmentation, an essential operation for image analysis and understanding. In this study, we propose a simple SRLM image segmentation technique based on estimating and smoothing spatial densities of fluorophores using adaptive anisotropic kernels. Experimental results showed that the proposed method provided robust and accurate segmentation of SRLM images and significantly outperformed conventional segmentation approaches such as active contour methods in segmentation accuracy.

We propose an automated algorithm for segmentation of mitochondria from widefield fluorescence microscopy images for quantitative morphology characterization. Mitochondria are membrane-bound organelles that are essential to cells of higher living organisms. Reliable and precise quantitative characterization of their shape is crucial to understanding related physiology and disease mechanisms. Building upon the active-mask framework developed for segmentation of confocal fluorescence microscope images, we propose a new adaptive region-based distributing function to effectively address the problem of halo artifacts that are common in widefield fluorescence images. Such artifacts prevent the segmentation of weak features of mitochondria using existing algorithms. We compare the algorithm to the original active-mask algorithm as well as the geodesic active contour algorithm based on hand-segmented ground truth, and find that it performs significantly better both qualitatively and quantitatively.

A fundamental requirement of micromanipulation is to control the impact force and subsequently the contact force in the transition of the micromanipulator end-effector from noncontact to contact state. This is especially important in protecting fragile microstructures and preventing undesirable motion. This paper proposes a method of using the integration of selective focusing and microforce control to achieve fast transition control while minimizing impact force. The method is applied to contact transition in microassembly pick-and-place operations. The initial long-range approach motion of the end-effector towards its target is controlled based on focus measures computed from images captured through a microscope. When the end-effector comes into focus near the target, the system switches to microforce control to minimize impact force and to regulate the contact force. An optics model for microscope focusing is proposed to characterize the dynamic behavior of the end-effector images during the approach motion. The connection between this model and the scale-space theory of computer vision is emphasized. Three different focus measures are tested and compared in performance. The proposed method has been experimentally verified to be able to achieve fast transition control with minimal impact force.

This paper reports on the construction of two new focus measure operators M/sub WT//sup 1/ an M/sub WT//sup 2/ defined in the wavelet transform domain. M/sub WT//sup 2/ provides significantly better focus performance in depth resolution than previously reported spatial domain operators. M/sub WT//sup 1/ provides performance equivalent to that of the best spatial domain operator but has lower computational cost than M/sub WT//sup 2/. Both operators can be used with a wide variety of wavelet bases optimized for different applications. Selection of wavelet bases is studied based on their number of vanishing moments, size of support and symmetry. The depth resolution of these operators makes them an important cue in the segmentation of low depth-of-field microscopic images. An unsupervised segmentation technique based on graph partition is then introduced. It uses M/sub WT//sup 2/ together with proximity and image intensity as segmentation features. This segmentation method does not depend on the connection of local image features and remains robust under defocusing. Experimental results confirm the effectiveness of the proposed focus measures and the segmentation algorithm. These techniques are especially suitable for high resolution microscopic computer vision tasks in high precision micromanipulation and microassembly applications.

When assembling MEMS devices or manipulating biological cells it is often beneficial to have information about the force that is being applied to these objects. This force information is difficult to measure at these scales. We demonstrate a method to reliably measure nanonewton scale forces applied to a micro scale cantilever beam using a computer vision approach. A template matching algorithm is used to estimate the beam deflection to sub-pixel resolution in order to determine the force applied to the beam. The template, in addition to containing information about the geometry of the beam, contains information about the elastic properties of the beam. Minimizing the error between this elastic template and the actual image by means of numerical optimization techniques, we are able to measure forces to within /spl plusmn/3 nN. In addition, we also discuss how this method can be generalized to measure forces in elastic configurations other than a simple cantilever beam using a micro-tweezer as an example. This provides the opportunity for this method to be used with specially designed micromanipulators to provide force as well as vision feedback for micromanipulation tasks.

This paper reports on an experimental micro-assembly workcell developed for efficient and reliable 3D assembly of large numbers of micro-machined thin metal parts into micromachined holes in 4 inch silicon wafers. The major objective is to integrate techniques of micro-gripper design, microscopic imaging and high precision motion control to build a prototype system for industrial applications. The workcell consists of a multiple-view imaging system, a 4-DOF micromanipulator with high resolution rotation control, a large working space 4-DOF precision positioning system, a flexible micro-gripper, and a control software system. A piezoelectric force sensing unit is developed to be integrated with the manipulator system to enhance pickup reliability. Operations are partially guided by a human operator through a graphical user interface. This system provides a highly flexible testbed for wafer-level 3D microassembly.