Sperm morphology measurement is vital for diagnosing male infertility, which involves quantification of multiple subcellular parts for each sperm. Instance-aware part segmentation networks have been introduced to address this task by automatically identifying individual sperm and segmenting their subcellular parts. However, major limitations of state-of-the-art instance-aware part segmentation networks include: 1) they are time-consuming and computational expensive due to sequential processing and multi-stage frameworks; 2) they perform poorly for densely packed sperm that overlap or cross over one another. To overcome these challenges, this paper proposes 1) integrating instance identification and subcellular part segmentation within a single-stage framework to save inference time and memory usage; 2) dividing a sperm target into simpler components (head and tail) to improve prediction accuracy, followed by a contrastive learning-based matching method to pair the head and tail. Experimental results on our clinically collected human sperm dataset demonstrated that the proposed network not only outperformed state-of-the-art CP-Net (by 3.5% APp vol) but also achieved realtime inference (48.0 frames per second), effectively meeting the clinical requirements for automated parts segmentation of sperm. final part segmentation results. 2) Since the sperm head and tail have simpler shapes, they are detected separately to improve segmentation accuracy. A contrastive learning-based method is then designed to pair head and tail based on similarity of feature embeddings extracted from the proposed instance prediction branch. The proposed method significantly outperformed existing networks, particularly in handling densely packed sperm. The presented method has applicability to analyzing sperm and more broadly other cell types.

Automated morphological measurement of cellular and subcellular structures in live cells is important for evaluating cell functions. Due to their small size and transparent appearance, visualizing cellular and subcellular structures often requires high magnification microscopy and fluorescent staining. However, high magnification microscopy gives a limited field of view, and fluorescent staining alters cell viability and/or activity. Therefore, microscopy image enhancement methods have been developed to predict detailed intracellular structures in live cells. Existing image enhancement networks are mostly CNN-based models lacking global information or Transformer-based models lacking local information. For these purposes, a novel CNN-Transformer combined bilateral U-Net (CTBUnet) is proposed to effectively aggregate both local and global information. Experiments on the collected sperm cell enhancement dataset demonstrate the effectiveness of proposed network for both super-resolution and virtual staining prediction.

This paper presents an automated point-of-care semen analysis method that uses smartphone imaging to visually measure sperm concentration and motility of semen samples. The proposed method follows the same visual tracking scheme as laboratory semen analysis systems, aiming to match clinical standards while being suitable for point-of-care use. A boundary-sensitive segmentation network is developed to identify and distinguish sperm from impurities in raw semen. A novel occlusion-aware multi-sperm tracking algorithm is proposed to tackle challenges posed by smartphone imaging and undiluted raw semen samples. For automated motility measurement, an occlusion-awareness module is proposed to robustly track multiple sperm during frequent sperm crossover/occlusion. The module combines the segmented contour and kinematic-based probabilistic modeling to determine the occlusion status of both targets and measurements, facilitating fundamental improvement to feasible joint event enumeration to enable robust data association. The proposed method achieved a high success rate of 95.14% for tracking occluded sperm, with low mean errors for sperm concentration (2.03 million/ml) and motility (1.58%), outperforming existing multi-sperm tracking methods. In clinical tests involving 50 participants, our method exhibited good agreement with clinical standards (Spearman rank correlation coefficients of 0.94 for concentration and 0.89 for motility) even when used by inexperienced users.Note to Practitioners—Semen analysis is the gold standard method for assessing male reproductive capacity. Clinical semen analysis routinely uses professional computer-assisted semen analysis (CASA) systems to examine sperm concentration and motility; however, clinical visits for semen analysis are not always feasible due to the unavailability of such professional systems and the mental stress brought by clinical visits. This work provides a point-of-care semen analysis method. For hardware, a smartphone microscopic imaging modality was developed to enable clear visualization of sperm with the built-in smartphone camera. For software, an occlusion-aware multi-sperm tracking algorithm was proposed to automatically measure sperm concentration and motility. In addition to intensively validating the proposed point-of-care method against clinical CASA systems, this work also analyzed different types of tracking failure and quantified their effects on the automated evaluation of sperm concentration and motility. The techniques pave the way for further improvement in both point-of-care and clinical semen analysis. Overall, this work offers an accessible and reliable tool for automated male fertility evaluation.

Automated positioning systems play a pivotal role in micro-scale cell manipulation. In clinical intracytoplasmic sperm injection (ICSI) for infertility treatment, a motile sperm needs to be immobilized by glass micropipette tapping for subsequent surgical steps. The process requires accurate tracking of the target sperm and precise alignment between the sperm tail and the micropipette. Manual sperm immobilization suffers from inconsistent success rates, and current robotic systems developed for the task fail to comply with the standard clinical setup. Instead of using a motorized micromanipulator as in existing robotic systems, this paper presents an automated and compact three-dimensional (3-D) positioning stage for sperm immobilization that can be seamlessly integrated into standard clinical platforms. To tackle the challenge of accurately tracking the target sperm with degraded detection quality due to the complex 3-D motion of the positioning stage, a multi-stage sperm tracking scheme is designed for detection-to-tracklet association. To prevent physical contact between the sperm head and the micropipette, an adaptive tail-tapping planning strategy based on the sperm head orientation analysis is established. A visual servo controller equipped with a dynamic sperm motion observer is further employed to achieve precise positioning of the target sperm during the immobilization process. Experimental results demonstrated that the proposed system achieved a sperm tracking accuracy of 88.12%, and a sperm positioning accuracy of$2.3~\pm ~1.2~\mu $m. Further experiments revealed the system achieved a success rate of 93.5% and a time cost of 5.5 s for automated sperm immobilization. Note to Practitioners—This work presents an automated positioning system for the robotic immobilization of live human sperm in the intracytoplasmic sperm injection (ICSI) process. Conventional robotic systems developed for sperm immobilization use motorized stages which only provide two degrees of freedom (DOF) and disturb standard clinical setups with bulky sizes and additional equipment. Leveraging the advantages of piezoelectric positioners, a compact 3-D positioning system is developed to perform robotic sperm immobilization in an all-in-one manner. A sperm immobilization tracker is developed to track the target sperm during the immobilization process. The proposed multi-stage data association metric can effectively resist the noisy detection results due to occlusion and out-of-focus blur introduced by the 3-D movement of the positioning stage. Based on the analysis of the sperm head orientation, an adaptive tail-tapping planning strategy is established to avoid the risk of contacting the sperm head where DNA is contained. The developed positioning system can be easily integrated into a standard microscopy operation platform for biological cell manipulation. The proposed tracking scheme is applicable in various microscopy cell analysis scenarios where the detection results are inevitably affected by the degraded image quality.

Traditional sperm morphology analysis is based on tedious manual annotation. Automated morphology analysis of a high number of sperm requires accurate segmentation of each sperm part and quantitative morphology evaluation. State-of-the-art instance-aware part segmentation networks follow a "detect-then-segment" paradigm. However, due to sperm’s slim shape, their segmentation suffers from large context loss and feature distortion due to bounding box cropping and resizing during ROI Align. Moreover, morphology measurement of sperm tail is demanding because of the long and curved shape and its uneven width. This paper presents automated techniques to measure sperm morphology parameters automatically and quantitatively. A novel attention-based instance-aware part segmentation network is designed to reconstruct lost contexts outside bounding boxes and to fix distorted features, by refining preliminary segmented masks through merging features extracted by feature pyramid network. An automated centerline-based tail morphology measurement method is also proposed, in which an outlier filtering method and endpoint detection algorithm are designed to accurately reconstruct tail endpoints. Experimental results demonstrate that the proposed network outperformed the state-of-the-art top-down RP-R-CNN by 9.2% ${\mathbf{AP}}_{vol}^p$, and the proposed automated tail morphology measurement method achieved high measurement accuracies of 95.34%,96.39%,91.20% for length, width and curvature, respectively.

Automated positioning systems play a pivotal role in micro-scale cell manipulation. In clinical intracytoplasmic sperm injection (ICSI) of in vitro fertilization (IVF) treatment, a motile sperm needs to be immobilized by glass micropipette tapping for subsequent surgical steps. The process requires accurate tracking of the target sperm and precise alignment between the sperm tail and the micropipette. Manual sperm immobilization suffers from inconsistent success rates, and current robotic systems developed for the task fail to comply with the standard clinical setup. Instead of using a motorized micromanipulator as in existing robotic systems, this paper presents an automated, compact three-dimensional positioning stage for sperm immobilization that can be seamlessly integrated into standard clinical platforms. Based on the analysis of the sperm head orientation, an adaptive tail tapping planning strategy is established to avoid the risk of touching the sperm head where DNA is contained. A visual servo controller equipped with a dynamic sperm motion observer is employed to achieve precise tracking and positioning of the target sperm three-dimensionally. Experimental results revealed the system achieved a success rate of 93.5% and a time cost of 5.5 s for automated sperm immobilization.

Instance segmentation of biological cells is important in medical image analysis for identifying and segmenting individual cells, and quantitative measurement of subcellular structures requires further cell-level subcellular part segmentation. Subcellular structure measurements are critical for cell phenotyping and quality analysis. For these purposes, instance-aware part segmentation network is first introduced to distinguish individual cells and segment subcellular structures for each detected cell. This approach is demonstrated on human sperm cells since the World Health Organization has established quantitative standards for sperm quality assessment. Specifically, a novel Cell Parsing Net (CP-Net) is proposed for accurate instance-level cell parsing. An attention-based feature fusion module is designed to alleviate contour misalignments for cells with an irregular shape by using instance masks as spatial cues instead of as strict constraints to differentiate various instances. A coarse-to-fine segmentation module is developed to effectively segment tiny subcellular structures within a cell through hierarchical segmentation from whole to part instead of directly segmenting each cell part. Moreover, a sperm parsing dataset is built including 320 annotated sperm images with five semantic subcellular part labels. Extensive experiments on the collected dataset demonstrate that the proposed CP-Net outperforms state-of-the-art instance-aware part segmentation networks.