Segmenting individual trophectoderm (TE) cells is essential for developing quantitative metrics to assess the developmental potential of human blastocysts. The elongated shape and circular arrangement of TE cells lead to continuously varying orientations across the image, posing challenges for existing cell instance segmentation methods that assume uniformly oriented cells. As a result, most methods segment the TE as a single region, and the development of quantitative, cell-level metrics predictive of live birth potential remains unexplored. In this work, we propose an instance segmentation model that represents elongate, circularly arranged TE cells using elliptical distance maps, with which superior performance in both segmentation accuracy and metric extraction was achieved, compared with state-of-the-art methods. The extracted metrics, including TE cell number, the mean and standard deviation of cell length, width, area, and mean inter-cell distance, serve as effective predictors of a blastocyst’s live birth potential. When used to predict live birth, these metrics achieved a significantly higher area under the receiver operating characteristic curve (AUC = 0.693) than traditional TE morphological grades (AUC = 0.585). The source code is publicly available at https://github.com/robotVisionHang/TESeg.

Quantitative analysis of cleavage-stage embryos is a critical step for embryo evaluation in in vitro fertilization (IVF) treatment. Deep learning-based instance segmentation methods have emerged as a promising solution. However, existing methods, typically designed for opaque instances in nature scenes, face challenges when applied to segmenting overlapping cells in cleavage-stage embryos. The semi-transparency of the cells in cleavage-stage embryos leads to vague boundaries and complex overlaps between cells, making it difficult for existing models to segment these cells without merging or missing cells. In this work, we propose a novel instance segmentation network for segmenting overlapping, semi-transparent cleavage-stage embryo cells. Specifically, the network introduces two specialized branches that learn complementary representations: an overlap branch for addressing inter-instance overlaps and a boundary branch for dealing with intra-instance shapes. Then, an attention-based feature fusion module is designed to integrate learned overlap and boundary features to enhance the instance representation for the segmentation branch. Finally, an overlap-aware post-processing module leverages the outputs from the overlap and boundary branches to adaptively adjust suppression thresholds, ensuring the preservation of distinct yet highly overlapping cells. Experiments on two cleavage-stage embryo datasets demonstrate that our method outperformed state-of-the-art methods in both segmentation accuracy and recall, highlighting its potential for quantitative embryo evaluation in clinical IVF.

Robotic steering of magnetic guidewires has shown great potential in accelerating endovascular interventions, enhancing the success rate of time-sensitive surgeries such as stroke treatment. Incomplete state feedback of the guidewire from 2D perspective images and unknown interactions with the surrounding vessel wall raise challenges in modeling and steering control. These two factors, however, are commonly overlooked by existing works. In this paper, 2D perspective images of the guidewire, which comply with prevalent medical imaging modalities, are used as the only feedback. A model-based external force observer is proposed that allows the guidewire to perceive the unknown interactions, and a compliance controller is subsequently designed to handle the external force while steering the guidewire. Experiments conducted in a human-sized phantom demonstrate how the compliance controller preserves stability and safety by adapting to the estimated external force.

Video object detection (VOD) of motile cells (e.g., bacteria and sperm) under microscopy is challenging due to motion blur, sporadic out-of-focus, and pose variations. Compared with VOD in generic scenes, the lower contrast and smaller color space of microscopy imaging further introduce feature overlap between the foreground objects and the background objects (e.g., impurity cells and contaminants). Transformer-based methods have achieved great success in the VOD of generic scenes by utilizing object queries to model the inner-frame objects and the inter-frame objects. However, the appearance overlap problem in microscopy video frames significantly compromises the inter-frame query aggregation by introducing background features into the object query. To tackle this challenge, this paper reports a static-dynamic query-based VOD network that treats object queries of the current video frame and reference video frames differently. Specifically, a two-stage framework is implemented that first generates high-quality object queries of reference frames with a static Transformer decoder pre-trained on a still image dataset. The network is then trained on a per-frame annotated dataset using a dynamic Transformer decoder to model the object queries of the current frame. A Reference Query Relation Module is further proposed to enhance the reference queries for more effective aggregation with the current query. Experiments on clinically collected biopsied sperm datasets validated the effectiveness of the proposed method.

Sperm morphology measurement is vital for diagnosing male infertility, which involves quantification of multiple subcellular parts for each sperm. Instance-aware part segmentation networks have been introduced to address this task by automatically identifying individual sperm and segmenting their subcellular parts. However, major limitations of state-of-the-art instance-aware part segmentation networks include: 1) they are time-consuming and computational expensive due to sequential processing and multi-stage frameworks; 2) they perform poorly for densely packed sperm that overlap or cross over one another. To overcome these challenges, this paper proposes 1) integrating instance identification and subcellular part segmentation within a single-stage framework to save inference time and memory usage; 2) dividing a sperm target into simpler components (head and tail) to improve prediction accuracy, followed by a contrastive learning-based matching method to pair the head and tail. Experimental results on our clinically collected human sperm dataset demonstrated that the proposed network not only outperformed state-of-the-art CP-Net (by 3.5% APp vol) but also achieved realtime inference (48.0 frames per second), effectively meeting the clinical requirements for automated parts segmentation of sperm. final part segmentation results. 2) Since the sperm head and tail have simpler shapes, they are detected separately to improve segmentation accuracy. A contrastive learning-based method is then designed to pair head and tail based on similarity of feature embeddings extracted from the proposed instance prediction branch. The proposed method significantly outperformed existing networks, particularly in handling densely packed sperm. The presented method has applicability to analyzing sperm and more broadly other cell types.

Automated morphological measurement of cellular and subcellular structures in live cells is important for evaluating cell functions. Due to their small size and transparent appearance, visualizing cellular and subcellular structures often requires high magnification microscopy and fluorescent staining. However, high magnification microscopy gives a limited field of view, and fluorescent staining alters cell viability and/or activity. Therefore, microscopy image enhancement methods have been developed to predict detailed intracellular structures in live cells. Existing image enhancement networks are mostly CNN-based models lacking global information or Transformer-based models lacking local information. For these purposes, a novel CNN-Transformer combined bilateral U-Net (CTBUnet) is proposed to effectively aggregate both local and global information. Experiments on the collected sperm cell enhancement dataset demonstrate the effectiveness of proposed network for both super-resolution and virtual staining prediction.

Zero-shot anomaly detection (ZSAD) aims to develop a foundational model capable of detecting anomalies across arbitrary categories without relying on reference images. However, since "abnormality" is inherently defined in relation to "normality" within specific categories, detecting anomalies without reference images describing the corresponding normal context remains a significant challenge. As an alternative to reference images, this study explores the use of widely available product standards to characterize normal contexts and potential abnormal states. Specifically, this study introduces AnomalyVLM, which leverages generalized pretrained vision-language models (VLMs) to interpret these standards and detect anomalies. Given the current limitations of VLMs in comprehending complex textual information, AnomalyVLM generates hybrid prompts-comprising prompts for abnormal regions, symbolic rules, and region numbers-from the standards to facilitate more effective understanding. These hybrid prompts are incorporated into various stages of the anomaly detection process within the selected VLMs, including an anomaly region generator and an anomaly region refiner. By utilizing hybrid prompts, VLMs are personalized as anomaly detectors for specific categories, offering users flexibility and control in detecting anomalies across novel categories without the need for training data. Experimental results on four public industrial anomaly detection datasets, as well as a practical automotive part inspection task, highlight the superior performance and enhanced generalization capability of AnomalyVLM, especially in texture categories. An online demo of AnomalyVLM is available at https://github.com/caoyunkang/Segment-Any-Anomaly.

Image anomaly localization is a pivotal technique in industrial inspection, often manifesting as a supervised task where abundant normal samples coexist with rare abnormal samples. Existing supervised methods in this context are prone to overfitting, as they primarily encounter anomalies that represent only a fraction of the open-world anomalies. Conversely, unsupervised methods excel in performance, yet they disregard the essential biased knowledge pertaining to both seen and unseen anomalies within the open world. To bridge this gap and refine unsupervised methods for supervised applications, this study introduces a comprehensive framework called biased students (BiaS), mainly comprising a three-step strategy. This strategy encompasses biased knowledge generation, transfer, and fusion. BiaS effectively segregates the vast anomaly space into two subsets: 1) unseen anomalies and 2) seen anomalies. Subsequently, it generates specialized biased knowledge for these subsets and transfers this knowledge to two distinct subnetworks. As a result, one subnetwork becomes adept at detecting unseen anomalies, while the other excels in localizing seen anomalies. To optimize their capabilities, BiaS synergistically fuses these subnetworks based on their expertise. Rigorous experimentation has empirically validated the effectiveness, generality, and scalability of BiaS, underscoring its potential to enhance unsupervised methods and effectively address the challenges of supervised anomaly localization.

Ultra-wideband (UWB) technology has been widely applied in the high-precision indoor positioning system. However, the complicated indoor environment makes signals propagate in non-line-of-sight (NLoS) situations, which seriously deteriorates the positioning accuracy. This work proposes an outlier-aware method to improve the positioning accuracy under NLoS scenarios. End-to-end optimization and positioning are achieved by combining the measurement error mitigation process with the positioning process. Experiments on public benchmarks illustrate that the proposed method enhances the performance of indoor positioning in NLoS situations.