VLDB 2026 Research / reviewers in the wild / expert
Mingzhu Sun
dblp:18/11433
· DBLP profile ↗
28ranked-venue papers
4as first author
22since 2021 · last 2026
—ORCID · conflict
Domains — the database's venue-derived domains; a paper can count in several
Applied, interdisciplinary, general and emerging computing · 16 · 3 first-author · 12 since 2021Artificial intelligence and machine learning · 10 · 1 first-author · 9 since 2021Systems, architecture and hardware · 7 · 1 first-author · 6 since 2021Graphics, computer vision, multimedia, augmented reality and games · 2 · 1 since 2021
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2026 | MeepleLM: A Virtual Playtester Simulating Diverse Subjective ExperiencesabstractZizhen Li, Chuanhao Li, Yibin Wang, Jianwen Sun, Yukang Feng, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Yifei Huang, Kaipeng Zhang. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026. Zizhen Li, Chuanhao Li 0001, Yukang Feng, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Kaipeng Zhang |
ACL (1) | 8 |
| 2026 | DMDGRN: A data augmentation-based multilayer directed graph convolutional network for gene regulatory network inference
Pi-Jing Wei, Mingzhu Sun, Zheng Ding, Chun-Hou Zheng 0001 |
J. Biomed. Informatics | 2 |
| 2026 | Robotic Double Patch Clamp Based on Interactive Mechanical Modeling for Functional Connectivity Measurement Between NeuronsabstractDouble patch clamp technique, using two micropipette electrodes to patch and measure electrophysiological signals of two neurons, is essential for investigating the functional connections between neurons in brain. However, the interactive mechanical disturbances from the dual-micropipette motions inside viscoelastic brain tissue cause dynamic drifts of neurons, making double patching low efficiency and challenging. In this paper, an interactive mechanical modeling of two micropipettes approaching two cells in elastic environment was established to estimate the dynamic drift of the cells. Based on that, a synchronous descent strategy, an appropriate relative position of two cells, and an online trajectory plan of two micropipettes were determined to improve double patch clamp efficiency. Finally, a robotic double patch clamp operation process was established for functional connectivity measurement between neurons in brain slice. The effectiveness of the proposed work is validated through both finite element modeling and experiments. The double patch clamp experiments on neurons in visual cortex demonstrate that our method achieves a 40% improvement in success rate and a 38% improvement in speed in comparison to the traditional manual method. With the above advantages, diverse functional connectivity activities between neurons were found using our system, paving a solid ground for further research. Biting Ma, Jinyu Qiu, Shaojie Fu, Yuzhu Liu, Mingzhu Sun, Xin Zhao 0010, Qili Zhao |
IEEE Trans Autom. Sci. Eng. | 8 |
| 2026 | A Retinal Image Sequence Registration Method Based on Longitudinal 3D Fundoscopy Scene ModelingabstractAccurate and reliable global registration is the basis and prerequisite for longitudinal studies of related diseases that are based on retinal image sequences. However, most existing retinal image registration methods do not consider the implicit inherent spatiotemporal transformation process between the retinal images to be registered, resulting in low accuracy or poor reliability. In this paper, we propose a 3D spatiotemporal model that simulates the fundoscopy scene which is on a longitudinal time-axis. There are two main components: 1) a dynamic eyeball model that simulates the changes in eyeball shape owing to natural growth or disease progression (e.g., myopia); and 2) a camera array model that simulates the changes in eyeball pose and fundus camera parameters between each fundoscopy. Based on the 3D spatiotemporal model, we implement a retinal image sequence registration framework using a frame-to-reference and frame-to-frame joint registration strategy. The framework uses the matched keypoints as a medium, and adopts pose estimation and optimization algorithm to align all images in the sequence into the same longitudinal fundoscopy scene. Benefiting from the spatiotemporal simulation of the longitudinal fundoscopy scene, the proposed method can generate a large amount of realistic synthetic data given only one retinal image. We conduct comprehensive experiments on two image pair registration datasets and three image sequence registration datasets. The results show that the proposed method achieves state-of-the-art registration accuracy, reliability, and applicability. To promote the study in related fields, we make all codes and datasets publicly available. Zengshuo Wang, Haohan Zou, Xin Zhao 0010, Mingzhu Sun |
IEEE Trans. Image Process. | 5 |
| 2025 | InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning StylesabstractZizhen Li, Chuanhao Li, Yibin Wang, Qi Chen, Diping Song, Yukang Feng, Jianwen Sun, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Kaipeng Zhang. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. Zizhen Li, Chuanhao Li 0001, Diping Song, Yukang Feng, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Kaipeng Zhang |
EMNLP | 10 |
| 2025 | Nonlinear Viscoelastic Model-based Deformation Optimization for Robotic Micropuncture in Retinal Vein CannulationabstractMicropuncture is a critical step in drug injection during retinal vein cannulation (RVC) surgery. Minimizing deformation during the micropuncture process is beneficial to reduce mechanical damage. However, this goal is challenging due to the viscoelastic characteristics of retinal tissue. In this paper, a robotic micropuncture scheme for deformation optimization that incorporates a nonlinear force model is proposed. Before micropuncture, a preload strategy is utilized to ensure stable contact between needle and retinal vein. Secondly, a nonlinear viscoelastic (NV) model is developed to characterize the nonlinearity and relaxation behavior of the tissue. Finally, a speed optimization framework, based on the NV model and physical constraint, is adopted to minimize deformation. The effectiveness of the proposed scheme is validated through in vitro experiments conducted on open-sky porcine eyes. With average force error of 1.48 μN, stable contact can be achieved via proportion-integral-differential controller. The experimental results demonstrate that the NV model is more suitable for force modeling of retinal tissue. Furthermore, the optimized speed results in an average deformation of 0.5727 mm, which represents a reduction of at least 21.02% compared to the linear model. Thanks to the proposed scheme, the robotic micropuncture based on a varying speed trajectory can reduce deformation and enhance the safety of RVC surgery. Bo Hu 0013, Rongxin Liu, Zengshuo Wang, Mingzhu Sun, Xin Zhao 0010 |
IROS | 6 |
| 2025 | Modeling and Simulation of Single-micropipette Cell Rotation for Imitation LearningabstractCell rotation plays a crucial role in micromanipulation. Among manual cell rotation techniques, single-micropipette cell rotation is widely adopted due to its high efficiency and flexibility. However, there is currently no method capable of achieving automated single-micropipette cell rotation. In this study, we developed the first three-dimensional (3D) simulation system for single-micropipette cell rotation. Based on this simulation system, we successfully achieved single-micropipette cell rotation imitation learning (IL) for the first time. Specifically, we first analyze the forces acting on cells in the fluid, establishing a dynamic model that describes the cell's behavior in response to the flow velocity at the holding micropipette's orifice, the relative position of the micropipette, and time. We then developed the cell rotation simulation environment by discretizing the model and designing the simulation's cell and holding micropipette models based on real-world conditions. Finally, we designed a network architecture for IL using this model, achieving single-micropipette cell rotation in simulation. The results demonstrate that the simulation system exhibits a relative error range of 5.34% to 12.21% compared to real-world experiments, indicating a high degree of accuracy. Additionally, the single-micropipette cell rotation task achieved a success rate of 69% with an average completion time of 17.13 seconds, closely matching the expert data's average time of 17.69 seconds, confirming the feasibility of the simulation system. Zefu Wang, Yuchen Hua, Huiying Gong, Zhanli Yang, Yaowei Liu, Mingzhu Sun |
IROS | 8 |
| 2025 | Automated Dual-Micropipette Coordination Microinjection for Batch Zebrafish Larvae Based on Pose EstimationabstractZebrafish are widely used in the biomedical field, as an ideal model for microinjection. In automated zebrafish microinjection, posture adjustment is the first and key step, which takes a lot of skill, and injection success assessment is a challenging task. Constrained by these two aspects, it is difficult to further enhance the efficiency and success rate of injection. In this study, we propose an automated dual-micropipette coordination microinjection system. Zebrafish are randomly arranged in our system, reducing the operational difficulty, and the yolk is positioned using a pose estimation algorithm, followed by injection accomplished with dual-micropipette. Due to the reduction of posture adjustment time by half, the proposed system achieves the shortest injection time of 15.2s. Moreover, the simplicity of the system and the ease of operation contribute to the clinical feasibility of our system. Rongxin Liu, Huiying Gong, Zengshuo Wang, Yaowei Liu, Xin Zhao 0010, Mingzhu Sun |
IROS | 8 |
| 2025 | Adaptive self-evolving extreme learning machine-based terminal sliding mode control with application in retinal vein injection
Bo Hu 0013, Lu Liu 0002, Rongxin Liu, Mingzhu Sun, Xin Zhao 0010 |
Eng. Appl. Artif. Intell. | 5 |
| 2025 | An Adaptive Finite-Time Sliding Mode Control for Retinal Vein Micro-Puncture With Silicon PhantomabstractRetinal vein occlusion (RVO) is a prevalent ocular pathology that may result in hemorrhage and even blindness. Currently, a procedure termed retinal vein cannulation (RVC), involving puncturing the retinal vein and injecting medication, has been developed. However, RVC requires extremely high precision at the micron scale. To address the challenges in the micro-puncturing process of the RVC, an adaptive finite-time sliding mode (AFSM) control scheme with a smooth motion generator has been proposed to assist surgeons in achieving precise micro-punctures using a piezo-actuated stage. Firstly, an S-curve-based smooth motion planning approach incorporating force feedback is designed to detect the successful micro-puncture state, thus addressing the challenge of limited force perception during the procedure. Subsequently, an AFSM control scheme has been developed to track the desired motion. Finally, a micro-puncture system, equipped with a silicon phantom, is established for experimental purposes. The experimental results demonstrate that the proposed control scheme significantly enhances the tracking performance during the micro-puncture process. The smooth motion planning and AFSM control scheme prove to be effective for the automatic control of the piezo-actuated end-effector, thereby providing improved assistance to surgeons in the RVC process. Note to Practitioners—During the procedure of retinal vein micro-puncture, it is crucial to ensure a smooth motion planning and accurate tracking to guide the needle tip into the retinal vein lumen. In light of the difficulties in lack of depth perception, a motion generator has been proposed with an adaptive micro-puncture state detection mechanism based on force feedback. To reliably track the desired motion, the AFSM controller is designed to ensure tracking accuracy and robustness, and finite-time stability. In particular, the adaptive gain of the AFSM controller does not require uncertain prior information, making it friendly to clinical applications. The experimental results based on silicone phantom, demonstrate the effectiveness of the controller in achieving successful micro-puncture with precise tracking performance. The implementation of the AFSM controller enables the automated micro-puncture task, reducing the risk of damage during operation. Bo Hu 0013, Rongxin Liu, Xin Zhao 0010, Mingzhu Sun |
IEEE Trans Autom. Sci. Eng. | 5 |
| 2025 | Robotic Fluorescent Lighting Method Based on Dynamic Fluorescence Imaging Modeling for In Vivo Cell ManipulationabstractIn vivo cell manipulation is crucial for understanding organ functions and dysfunctions at the cellular level. Due to its low optical transparency, the in vivo environment needs to be lighted up first to visualize the targets for manipulation. At present, operators usually carefully blow a micropipette filled with fluorescent dye solution to create a fluorescence field around the micropipette opening, lighting up the in vivo environment. This manual dynamic control of the fluorescence field is usually a low-accuracy, labor-intensive, and high-skill requirement task, becoming even more challenging when the micropipette moves to locate or approach target cells. A large number of in vivo lighting-control tests may improve manual control efficiency of the fluorescent field, but they usually cost many precise animal samples, making them costly and often unaffordable. To enhance the dynamic control efficiency of the fluorescent field while reducing experimental costs, a robotic fluorescent lighting control method is proposed based on an in vivo dynamic fluorescent imaging simulator. First, a dynamic fluorescence imaging model composing an injection model, a diffusion model, and a luminescence model of fluorescent molecules is established to simulate the fluorescence field in vivo. The fluorescent intensity distribution obtained through the model is highly in accordance with experimental results. Based on this model, an adaptive sliding mode controller is employed to achieve the desired fluorescent intensity in the brain tissue. Both the simulation results and the experimental results demonstrate significant advantages of the proposed controller in terms of control accuracy and stability compared to the traditional PID controller and manual operation. Note to Practitioners—Forming a stable fluorescent light field with a specified fluorescent intensity is of vital importance operation for in vivo micromanipulations. At present, manual adjustment of the fluorescent field by blowing the micropipette with mouth is time-consuming and energy-draining. In this paper, we first propose a dynamic fluorescence imaging simulator to address the economic and time costs of live experiments. We then design an adaptive sliding mode controller to dynamically control the fluorescent intensity at a specified position in the fluorescent field. Simulations based on the established model, along with experiments on brain tissues, demonstrate the controller’s effectiveness. Our system is expected to reduce human involvement and enable high-precision lighting for in vivo cell manipulation in the future. Jie Li 0115, Zizhen Li, Jinyu Qiu, Bo Hu 0013, Mingzhu Sun, Xin Zhao 0010, Qili Zhao |
IEEE Trans Autom. Sci. Eng. | 7 |
| 2025 | Precise Robotic Picking Up of Polar Body for Biopsy ApplicationabstractPolar body biopsy has been widely applied in preimplantation genetic diagnosis for assisted reproductive technology. The key step in the polar body biopsy is picking up the polar body from the oocyte/embryo using a micropipette. Unfortunately, the almost transparent appearance of the polar body as well as its dynamic drift when the micropipette approaches it inside the cell makes it a challenging task to pick it up with less cytoplasm loss for the cell. The unnecessary cytoplasm loss in the picking up process of the polar body easily causes damage to the development competence of the cell and may lead to disturbances to the biopsy results of the polar body. This paper proposes a precise robotic picking up method of polar bodies with less cytoplasm loss for biopsy purposes. First, a defocus imaging method is proposed to locate the polar body with an almost transparent appearance. Then, the dynamic drift of the polar body with the micropipette moving inside the cell is modeled online based on force analysis to determine an appropriate trajectory for the micropipette to approach the polar body. Further, an Active Disturbance Rejection Controller (ADRC) is designed to move the micropipette along the desired trajectory to approach the polar body and then aspirate it into the micropipette. The experimental results on porcine oocytes demonstrate that our system is capable of localizing the polar body with a success rate of 95% and an average error of$1.12\pm 0.14~\mu $m. Moving along the determined trajectory, the micropipette is capable of approaching the edge of the polar body with an average error of$1.84\pm 0.31~\mu $m (n =20), which is only 11% of the results obtained without dynamic drift estimation of the polar body. With this advantage, our system picks up the polar body with a close 60% improvement in success rate (95% vs 60%) and only half of the average cytoplasm loss (5% vs 10%) in comparison to operation results without dynamic drift estimation. Note to Practitioners—Picking up of polar body from the oocyte/embryo using a micropipette is a vital operation in the polar body biopsy. Precisely picking up the polar body with less cytoplasm is vital to maintaining the developmental competence of the embryo/oocyte and reducing disturbances to biopsy result. This article presented a precise robotic picking up process of polar body. This process introduced defocus imaging method for polar body localization, dynamic drift estimation of polar body, and micropipette trajectory design and motion control by Active Disturbance Rejection Controller (ADRC). Experimental results have demonstrated the efficiency of the proposed robotic picking up process. Application of this process may provide an economical and practical method to carry polar body biopsy for practitioners. Jinyu Qiu, Ke Li 0026, Yuzhu Liu, Chaoyu Cui, Shaojie Fu, Biting Ma, Qiongao Zhang, Maosheng Cui, Mingzhu Sun, Xin Zhao 0010, Qili Zhao |
IEEE Trans Autom. Sci. Eng. | 11 |
| 2025 | Prediction-Based Method for Micropipette Approaching to Optimal Spindle Removal PositionabstractDuring somatic cell nuclear transfer (SCNT), precise removal of the oocyte genetic material is critical. However, due to the invisibility of the genetic material (spindle) under brightfield observation and the potential displacement caused by the movement of micropipettes, accurately positioning the micropipette at the spindle poses a significant challenge. This study introduces an approach for optimal spindle removal by predicting its position. Initially, the polarization imaging system visualized the oocyte spindle, while a Multi-Feature Adaptive Kernel Correlation Filter (MFAKCF) algorithm tracked the spindle with 92.84% accuracy. Subsequently, enhancements were made to the Nonlinear Mass-Spring-Damper (NMSD) model to simulate live oocyte mechanical characteristics. Adjustments to NMSD model parameters simulated spindle displacement variations under diverse experimental conditions. Finally, the optimal spindle removal position was determined using NMSD model to simulate the micropipette approach to the spindle and the resulting position of spindle displacement. Experimental validation showed that the predictive accuracy of this model was 97.26%, with an average positional error of$0.4~\mu $m. Using this approach method can reduce cytoplasm loss to 4.5% and have a 100% enucleation success rate. Thus, the proposed prediction based optimal spindle removal position method can effectively anticipates spindle final positions, aiding in minimizing cytoplasmic loss during spindle removal. Note to Practitioners—Accurate oocyte nucleus removal is crucial for somatic cell nuclear transfer (SCNT) but challenging due to spindle invisibility under brightfield microscopy and displacement caused by micropipette movements. This study proposes a predictive method to address these issues. The polarization imaging system was used to visualize the spindle, and a MFAKCF tracking algorithm enabled real-time tracking. An enhanced NMSD model simulated spindle displacement, enabling accurate position prediction and reducing cytoplasmic loss during removal. This method offers a reliable tool for precise spindle removal in SCNT and broader applications in biomedical micromanipulation. Zuqi Wang, Detian Zhang, Zhaotong Chu, Qili Zhao, Mingzhu Sun, Maosheng Cui, Xin Zhao 0010, Yaowei Liu |
IEEE Trans Autom. Sci. Eng. | 7 |
| 2024 | Retinal image registration method for myopia development
Zengshuo Wang, Haohan Zou, Shan Guo, Xin Zhao 0010, Mingzhu Sun |
Medical Image Anal. | 7 |
| 2024 | Robotic Patch Clamp Electrophysiological Signal Measurement in Multiple Brain Regions Based on Brain Slice RegistrationabstractPatch clamp technique is a ”gold standard” approach to the study of ion channel biophysics and pharmacology. Due to the complexity of the operation and the heavy reliance on the experience of the experimenter, more and more researchers are focusing on patch clamping automation. In manual patch clamping, the operators patch the neurons in the specific brain region by experience, which lacks quantitative indicators of brain region selection. At present, the existing automated patch clamp focuses more on searching and measuring single cells under a high magnification lens, but ignores the localization of different brain regions under a low magnification lens, affecting the efficiency and application of the automated system. Addressing this, a 2D-3D registration method is proposed to register the experimental microscopic image of brain slices onto the brain atlases. This method is then embedded into the patch clamp system for brain region selection and electrophysiological recordings of neurons in multiple brain regions. It is the first time to achieve automated selection of brain regions in patch clamp experiments. The experimental results indicate that brain slice registration and brain region selection completely meet the requirements of the patch clamp experiment. Compared with the original robotic patch clamp process, the total time for the whole-cell patch clamp initialization with the improved process is reduced by 280 seconds, and the efficiency is increased by 42%. Meanwhile, the time for brain region switching is reduced to$1/6$of the original. In the future, this method will be applied to automatic image-guided patch clamp systems.Note to Practitioners—Calibrating the depth of the experimental brain slice in the brain and dividing the distribution of each brain region in the brain slice are the most important preparations for electrophysiological experiments. In this paper, we design 2D-3D registration to establish a method for brain region selection and robotic patch clamping of different brain regions in one brain slice. We have successfully detected the electrophysiological signals of neurons in 3 major brain regions, including the visual region, auditory region, and hippocampus. The experimental results show that the method proposed in this paper is suitable for the existing robotic patch clamp system and greatly improves the degree of automation. Ke Li 0026, Huiying Gong, Jinyu Qiu, Qili Zhao, Xin Zhao 0010, Mingzhu Sun |
IEEE Trans Autom. Sci. Eng. | 7 |
| 2024 | Automatic High-Throughput Injection System for Zebrafish Larvae Based on Precise Positioning of Injection TargetabstractZebrafish microinjection is widely used in vascular biology, neurology, and other research areas. The microinjection of zebrafish larvae is a complicated 3D task because zebrafish larvae are independent individuals with complex body structures compared with biological cells. In this study, we propose an automatic high-throughput injection system for zebrafish larvae under an optical inverted microscope. The proposed system combines the fixing device design, object detection, visual positioning, and 3D injection path planning to improve the survival rate of zebrafish after injection. Experimental results demonstrated the capability and efficiency of the proposed system, which achieved a success rate of 95.0% and a survival rate of 98.7% for zebrafish injection. The system can be applied to various biomedical and biochemical experiments.Note to Practitioners—Zebrafish larvae have long been an important model organism in neuroscience, biomedicine, and drug discovery. In various methods for external substance transportation into the zebrafish, microinjection is more efficient but also more difficult. At present, high-throughput injection of zebrafish larvae is mainly based on traditional manual operations. The injection speed and success rate of manual injection gradually decrease with the increase in zebrafish numbers. In this study, an automatic injection system is designed and implemented for high-throughput zebrafish larvae injection. We first use a simply designed mold to fix batch zebrafish larvae. Then we propose a coarse-fine two-step positioning method and perform a 3D injection path planning to achieve precise microinjection. The proposed injection system frees the operators from the repetitive and tedious injection work, providing technical support for the life science experiment. Huiying Gong, Rongxin Liu, Qili Zhao, Yaowei Liu, Xin Zhao 0010, Mingzhu Sun |
IEEE Trans Autom. Sci. Eng. | 9 |
| 2024 | Aspirating Cell Into Orifice of Micropipette for Precise Cell Transportation Using MicropipetteabstractSingle-cell transportation is one of the most common cell operations. Transporting cells with micropipettes is convenient for a wide range of biomedical applications. For high-efficiency cell transportation, the cells must be aspirated into the orifice of a micropipette. However, this is very difficult to achieve, as there is relative movement between the cell and the culture medium when the fluid drives the cell in the culture medium. It is crucial to use cell dynamics rather than fluid dynamics as the control objects to improve control performance and stop the cell immediately when it approaches the micropipette. In this study, the cell dynamics were modeled using a second-order model by integrating the dynamic model between the fluid and the cell into a first-order fluid dynamic model. A backstepping controller-based extended state observer was proposed to control the cell movement inside the micropipette. Experiments demonstrated that the proposed controller could aspirate cells into the orifice of the micropipette with high accuracy and no overshoot. Furthermore, the proposed controller was applied to automated somatic cell nuclear transfer, and it significantly boosted operational efficiency. Note to Practitioners—The need to apply advanced automation methods to transfer cells in life sciences has increased at a steady pace. The key feature of such systems is the ability to select and transfer cells at a predetermined position in space and time for biological applications. We propose a cell positioning control method based on vision-guided robotics that can directly aspirate cells to specified positions near the orifice of a micropipette. In somatic cell nuclear transfer, the proposed method of transferring somatic cells into oocytes occurs at a faster pace than manual operation. This provides essential functionality for single-cell transfer and is an appropriate technology for practitioners with this functional requirement. Xiangfei Zhao, Mingzhu Sun, Qili Zhao, Yaowei Liu, Xin Zhao 0010 |
IEEE Trans Autom. Sci. Eng. | 2 |
| 2023 | HiSwin UT: Hybrid Swin Transformers Network for Segmenting 3D in vivo Two-photon Images of Rodent CerebrovasculatureabstractTwo-photon microscopy is currently the preferred technique for in vivo vasculature imaging. The ability to generate high-resolution 3D images of blood vessels could improve our understanding of normal vascular physiology and disease pathogenesis. Automatic segmenting and mapping of these 3D blood vessel networks remain challenging due to vascular geometries' complexity and signal-to-noise ratio limitations. In this paper, we propose a novel segmentation network named HYbrid Swin Unet Transformers (HiSwin UT) to address this problem. The network explores the use of the Swin transformer to segment 3D volumetric vessels in vivo images acquired by two-photon microscopy. We propose to use a hybrid of hierarchical Swin transformers with 3D convolutional neural networks (3D CNNs) as both the encoder and the decoder. The Swin transformer guards guide the 3D CNN laborers at every stage in both the encoding and decoding paths to enlarge the receptive field of the network. Specifically, the 3D vasculature images are projected into a sequence of embedding and fed into the network. The encoders extract features at five different resolutions by using a combination of sifted windows self-attention layers and residual convolutional layers. Then these latent multi-scale features are connected to each decoder via skip connection to cater to the diverse range of vessels in terms of their sizes and shapes. We evaluate this HiSwin UT model on the public two-photon vasculature dataset, extensive experimental results demonstrate the superiority of our network compared with the state-of-the-art models. The source code of the HiSwin UT will be publicly available at https://github.com/UTINK/HiSwinUT. Jie Li 0115, Chong Wang 0011, Zizhen Li, Mingzhu Sun, Xin Zhao 0010 |
BIBM | 5 |
| 2023 | Automatic Cell Rotation Method Based on Deep Reinforcement LearningabstractCell rotation is widely used to adjust cell posture in sub-cellular micromanipulations. The trajectory planning of the injection micropipette is needed, so that the cells can be rotated with the minimum deformation to reduce cell damage and keep cell viability. Due to the uncertainty of cell properties and manipulation environment, it is difficult to identify the parameters of the mechanical models in traditional robotic cell rotation methods. In this paper, deep reinforcement learning is introduced into cell manipulation for the first time to perform trajectory planning of the micropipette. We first abstract the cell rotation process by using the mechanical model and microscopic vision techniques and build a cell rotation simulation environment. Then we design a reward function by combining various factors of cell rotation and implement a reinforcement learning framework based on deep Q-learning (DQL). Finally, we train the cell rotation process based on the deep reinforcement learning algorithm. The simulation results indicate the proposed DQL agent achieved an average success rate of 97% without useless exploration. Moreover, the proposed method rotated the cells in a way that causes less mechanical damage than humans, demonstrating the DRL ability for cell rotation with high efficiency and low cell damage. Huiying Gong, Yaowei Liu, Qili Zhao, Xin Zhao 0010, Mingzhu Sun |
ICRA | 6 |
| 2023 | Precise Aspiration and Positioning Control Based on Dynamic Model Inside and Outside the MicropipetteabstractCell aspiration is a common technique in cell manipulation for cell transfer or intracellular property measurement. In this paper, we present a robotic micromanipulation system for cell aspiration and positioning by a micropipette. Considering the relative motion of the object and the fluid, we first establish an overall dynamic model of microbead motion inside and outside the micropipette based on computational fluid dynamics (CFD). Then we design an adaptive sliding mode controller (ASMC) for microbead aspiration outside the micropipette and positioning inside the pipette based on the dynamic model. The controller is proven to achieve asymptotic stability by Lyapunov techniques. Simulation and experimental results demonstrate the effectiveness of the fluid model and the performance of the designed control system. Note to Practitioners—Cell aspiration with a micropipette is a key technology in cell manipulation. Generally, there is relative motion between the aspirated object and the fluid, resulting in large overshoot even aspiration failure. In this paper, we set up an overall dynamic model of microbead motion inside and outside the micropipette, combining microbead motion dynamics, fluid dynamics and pneumatic pump modeling. Based on this model, we design an ASMC for microbead aspiration outside the micropipette and positioning inside the pipette. In simulations and experiments, the positioning errors of the microbead of different sizes converge to zero without overshoot, revealing the strong robustness of the controller. Applications for this technology include cell or sperm aspiration and injection. Mingzhu Sun, Yatong Yao, Xiangfei Zhao, Huiying Gong, Jinyu Qiu, Yaowei Liu, Xin Zhao 0010 |
IEEE Trans Autom. Sci. Eng. | 1 |
| 2022 | Simultaneous Depth Estimation and Localization for Cell Manipulation Based on Deep LearningabstractVisual localization, which is a key technology to realize the automation of cell manipulation, has been widely studied. Since the depth of field of the microscope is narrow, the planar localization and depth estimation are usually coupled together. At present, most methods adopt the serial working mode of focusing first and then planar localization, but they usually do not have good real-time performance and stability. In this paper, a simultaneous depth estimation and localization network was developed for cell manipulation. The network takes a focused image and a defocus-offset image as inputs, and outputs the defocus in the depth direction and the offset in the plane at the same time after going through defocus-offset information extraction, defocus classification mapping and offset regression mapping. To train and test our network, we also create two datasets: An Adherent Cell dataset and an Injection Micropipette dataset. The experimental results demonstrated that the proposed method achieves the detection of all test samples with a frame rate of more than 40Hz, and the maximum errors of depth estimation and localization are$\boldsymbol{2.44\mu m}$and$\boldsymbol{0.49\mu m}$, respectively. The proposed method has good stability, which is mainly reflected in its strong generalization ability and anti-noise ability. Zengshuo Wang, Huiying Gong, Ke Li 0026, Yue Du, Yaowei Liu, Xin Zhao 0010, Mingzhu Sun |
IROS | 8 |
| 2021 | Robotic Cardinal Vein Microinjection of Zebrafish Larvae Based on 3D PositioningabstractZebrafish (Danio Rerio) larvae have long been an important model organism for biomedicine and drug discovery. It is difficult to deliver the external materials into the circulatory system by conventional exposing administration, while vein microinjection is more efficient but more challenging. In this paper, a robotic cardinal vein microinjection system was presented for zebrafish larvae. The key points of injection pipette during penetration were analyzed, and their 3D positions were identified online by combining motion analysis and visual calibration. 3D path planning of the pipette tip was designed automatedly before injection. The injection system was evaluated by injecting green fluorescent microspheres into 20 zebrafish larvae. The experiment results show that the fluorescent microspheres spread rapidly in the blood circulation system after injection, achieving a success rate of 85.0% and a survival rate of 94.1%. The superior performance provided by the system will significantly facilitate tumor xenografts and canner studies on zebrafish. Mingzhu Sun, Yatong Yao, Huiying Gong, Dongyan Chen, Xin Zhao 0010 |
ICRA | 1 |
| 2020 | Robotic Batch Somatic Cell Nuclear Transfer Based on Microfluidic GrooveabstractSomatic cell nuclear transfer (SCNT), which is an important procedure in cloning, has been conducted manually for decades. The operating efficiency drops sharply in batch SCNT because of the long-time observation under microscopy and the time-wasting traditional process. Though the operating time was reduced by robotic SCNT in previous studies, the traditional operating process was still used. In this article, we designed a new robotic batch SCNT process based on a microfluidic groove and two micropipettes in parallel. By using this new SCNT process, the operating area switching, objective lens conversing, and focusing on traditional SCNT process were eliminated, and oocyte localization was simplified, which saved much operating time. Experimental results showed that the new robotic batch process reduced about 50 s (41.7%) compared with the manual process (proposed 70 s versus manual 120 s). A success rate of 93.3% (n = 30) and a survival rate of 96.4% were achieved (n = 28), which were similar to manual process. The new robotic batch SCNT method demonstrated a high degree of efficiency and reproducibility. Yaowei Liu, Xuefeng Wang 0003, Qili Zhao, Xin Zhao 0010, Mingzhu Sun |
IEEE Trans Autom. Sci. Eng. | 5 |
| 2017 | Design and Implementation of the Three-Dimensional Observation System for Adult Zebrafish
Xuefeng Wang 0003, Mingzhu Sun, Xin Zhao 0010 |
ICVS | 3 |
| 2017 | Automated cell transportation for batch-cell manipulationabstractBatch-cell manipulation is a key technology in biological applications. Robotic manipulation has important significance to improve the operation success rate and reduce the technical threshold, but the problem of inefficiency still exists in batch-cell experiments. In this paper, an automated cell transportation system is designed for batch-cell manipulation. It has some technical aspects such as a cell groove to contain the cells, the micromanipulator and motor stage control methods, and computer vision algorithms. Since the cells are arranged in a line in the groove, the transportation system improves the efficiency of finding the cells in the petri dish. Furthermore, the minimum pressure to drag and release the cell are analyzed theoretically, so that the other cells will not be affected when manipulating one cell. The visual algorithms to detect the cell position and cell holding state are evaluated by porcine oocyte. Experimental results show both algorithm has high success rates: 96% and 100%. Finally, cell rotating experiments are introduced to verify the effectiveness of the transportation system. The average transfer efficiency has been improved by 20% compared to manual operation. The results show that this system can be used in many manipulations. Xuefeng Wang 0003, Yaowei Liu, Shibao Li, Maosheng Cui, Mingzhu Sun, Xin Zhao 0010 |
IROS | 5 |
| 2015 | Nonuniform resampling for staggered SARabstractStaggered SAR is an innovative 2-D imaging system utilizing the digital beam-forming (DBF) technique and the continuous variation of pulse repetition frequency (PRF). However, due to the PRF variation, the echo data in the azimuth is non-uniformly sampled. To achieve good imaging quality, it is indispensable to interpolate the data of the staggered SAR to be uniform before further imaging processing. In order to realize efficient interpolation for different PRF variation schemes, we propose a non-uniform fast Fourier transform (NUFFT)-based interpolation method to resample the non-uniform data of the staggered SAR in this paper. The complexity of the proposed interpolation method is in the order of O(MlogM), where M is the number of azimuth sampling. Simulation is performed based on the echo data of point target from the staggered SAR. Mingzhu Sun, Yongjiang Yu, Zhe Liu 0007, Wenchao Li 0002 |
IGARSS | 1 |
| 2015 | Robotic Cell Rotation Based on the Minimum Rotation ForceabstractIn this paper, a robotic cell rotation method based on the minimum rotation force is presented to adjust oocyte orientation in biological applications. In this method, the minimum rotation force, which can control the rotation angle (RA) of the oocyte quantitatively and generate minimum oocyte deformations, is derived through a force analysis on the oocyte in rotation. To exert this force on the oocyte, the moving trajectories (MT) of the injection micropipette (IM), are determined using mechanical properties of the oocytes. Further, by moving the IM along the designed MT, the rotation force control is achieved. To verify the feasibility of this method, a robotic rotation experiment for batch porcine oocytes are performed. Experimental results demonstrate that this system rotates the oocyte a10t an average speed of 28.6s/cell and with a success rate of 93.3%. More importantly, this method can generate much less oocyte deformations during cell rotation process compared with the manual method, while the average control error of RA in each step is only 1.2° (versus averagely 8.3° in manual operation), which demonstrates that our method can effectively reduce cell deformations and improve control accuracy of the RA. Note to Practitioners - Using an IM to rotate the oocyte is the most popular method to adjust oocyte orientations in many cellular biological applications. To rotate the oocyte precisely and reduce mechanical damages to the oocyte, the rotation force exerted on the oocyte should be estimated and controlled precisely. However, it is a challenging task and has not been resolved well. This paper calculates the minimum rotation force through the force analysis on oocyte and uses it to improve control accuracy of the RA and limit the cell deformations. Using calibrated oocyte mechanical properties, the MT of the IM corresponding to the minimum rotation force is designed. Then, by moving the IM along designed MT, the rotation force control is achieved online. This method does not rely on the force sensors and is realized on traditional micro-operation systems. Coupled with previous works, this method is able to operate batch oocytes one by one. Therefore, it can easily be applied in biological applications and replace manual operations. Qili Zhao, Mingzhu Sun, Maosheng Cui, Yanding Qin, Xin Zhao 0010 |
IEEE Trans Autom. Sci. Eng. | 2 |
| 2013 | A common factor-analytic model for classificationabstractIn this era of data explosion, much research has been directed to the problem of filtering and extracting useful information from extremely large datasets. The focus is on discriminant analysis of high-dimensional data, where the number of dimensions p is very large relative to the number of observations n. Mixture discriminant analysis provides an effective parametric approach, where each class density is modeled using mixtures of common factor analyzers. Although the adoption of mixture models with common factor loadings in the components significantly reduces the number of parameters to be estimated, the number of variables has to be reduced first to a more manageable level. Thus we consider the problem of dimension reduction for high-dimensional data. In this paper, we propose a factor-analytic model with common factor loadings for classification. We apply our model to a breast cancer study involving microarray gene expression data, which shows the parametric approach can select informative genes that improve the prediction of disease outcome. Mingzhu Sun, Geoffrey J. McLachlan |
BIBM | 1 |