Double patch clamp technique, using two micropipette electrodes to patch and measure electrophysiological signals of two neurons, is essential for investigating the functional connections between neurons in brain. However, the interactive mechanical disturbances from the dual-micropipette motions inside viscoelastic brain tissue cause dynamic drifts of neurons, making double patching low efficiency and challenging. In this paper, an interactive mechanical modeling of two micropipettes approaching two cells in elastic environment was established to estimate the dynamic drift of the cells. Based on that, a synchronous descent strategy, an appropriate relative position of two cells, and an online trajectory plan of two micropipettes were determined to improve double patch clamp efficiency. Finally, a robotic double patch clamp operation process was established for functional connectivity measurement between neurons in brain slice. The effectiveness of the proposed work is validated through both finite element modeling and experiments. The double patch clamp experiments on neurons in visual cortex demonstrate that our method achieves a 40% improvement in success rate and a 38% improvement in speed in comparison to the traditional manual method. With the above advantages, diverse functional connectivity activities between neurons were found using our system, paving a solid ground for further research.

Physical parameters of the intracellular environment such as mass density, intracellular pressure and elasticity have significant effects on the physiological activities of the cell and intracellular operation results. However, the significantly different measurement principles of the above parameters make it a challenging task for in situ measurement of them for the same cell, which significantly limits the study of their comprehensive regulation mechanisms to cell physiological activities and intracellular operation results. For the first time, a robotic in situ measurement system of multiple intracellular physical parameters is proposed based on a self-developed three-micropipettes system in this paper. Using this system, the mass density, elasticity and intracellular pressure of the same cell are measured automatically in sequence, according to a robotic in situ measurement process. Experimental results on sheep oocytes demonstrate an 83.3% measurement success rate at an average speed of 97.75 s/cell. The measurement results of the above three parameters are close to the reported results of individual, while with a significantly shorter operation time than theirs combined in references. Our system lays a solid foundation for the future research on the comprehensive regulation mechanism of these parameters to cell physiological activities and intracellular operation results.

In recent years, the perforated patch clamp technique has been widely applied in cellular electrophysiology research due to its low mechanical disturbance and almost no loss of cellular content in cell membrane perforation process. However, the current passive release process of perforating materials prolongs the perforation process and easily disturbs the gigaseal formation process, significantly lowering the efficiency of perforated patch clamp operation. Addressing this, a robotic perforated patch clamp system was developed based on the active release control of perforating materials in this paper. First, a novel holding module of the patch micropipette integrated with an independently driven transmission channel of the perforating materials was developed for the first time. Then, through release tests of the perforating materials, the appropriate drive mode of transmission channel was determined to be the hydraulic mode with a faster response and higher stability. Further, the concentration gradient field of the perforating materials at the opening of the channel was modeled according to Fick’s law to prevent the false release of them in gigaseal formation process. Furthermore, a cell circuit model was developed to detect perforation degree online for feedback control of the perforating materials release. Experimental results on pyramidal neurons in mouse brain slices demonstrated that, in comparison to the traditional method with passive releases of perforating materials, the proposed system was capable of perforating cell membrane at an almost doubled throughput and with a 57% improvement in the success rate. In comparison to the traditional non-perforated whole-cell patch clamp method, the signal recording duration of neurons operated by our method was doubled due to its fewer negative influences on gigaseal and almost no cellular content loss. Note to Practitioners—The perforated patch clamp technique, utilizes the cell membrane-perforating molecules to drill subnanometer-sized conductive pores in the cell membrane aspirated into a micropipette for the measurement of cellular electrophysiological signals. Unfortunately, the poor controllability of the current passive release of perforating materials in the patch clamp operation easily leads to a long drug diffusion process, and also, disturbs the gigaseal formation between the aspirated cell membrane and micropipette, which is required for the measurement of the extremely weak cellular electrophysiological signals. For the first time, an active release control method of perforating materials was developed in this paper based on the self-developed novel three-channel holding device. With active release control of perforating materials, the proposed method was capable of perforating cell membranes at a doubled speed with a significantly higher success rate, and doubled recording duration in comparison to the traditional perforated patch clamp methods and non-perforated whole-cell patch clamp method, respectively, due to its fewer negative influences on gigaseal and almost no cellular content loss. With the above advantages, our robotic perforated patch clamp method may be applied in cellular electrophysiology research in the future.

In vivo cell manipulation is crucial for understanding organ functions and dysfunctions at the cellular level. Due to its low optical transparency, the in vivo environment needs to be lighted up first to visualize the targets for manipulation. At present, operators usually carefully blow a micropipette filled with fluorescent dye solution to create a fluorescence field around the micropipette opening, lighting up the in vivo environment. This manual dynamic control of the fluorescence field is usually a low-accuracy, labor-intensive, and high-skill requirement task, becoming even more challenging when the micropipette moves to locate or approach target cells. A large number of in vivo lighting-control tests may improve manual control efficiency of the fluorescent field, but they usually cost many precise animal samples, making them costly and often unaffordable. To enhance the dynamic control efficiency of the fluorescent field while reducing experimental costs, a robotic fluorescent lighting control method is proposed based on an in vivo dynamic fluorescent imaging simulator. First, a dynamic fluorescence imaging model composing an injection model, a diffusion model, and a luminescence model of fluorescent molecules is established to simulate the fluorescence field in vivo. The fluorescent intensity distribution obtained through the model is highly in accordance with experimental results. Based on this model, an adaptive sliding mode controller is employed to achieve the desired fluorescent intensity in the brain tissue. Both the simulation results and the experimental results demonstrate significant advantages of the proposed controller in terms of control accuracy and stability compared to the traditional PID controller and manual operation. Note to Practitioners—Forming a stable fluorescent light field with a specified fluorescent intensity is of vital importance operation for in vivo micromanipulations. At present, manual adjustment of the fluorescent field by blowing the micropipette with mouth is time-consuming and energy-draining. In this paper, we first propose a dynamic fluorescence imaging simulator to address the economic and time costs of live experiments. We then design an adaptive sliding mode controller to dynamically control the fluorescent intensity at a specified position in the fluorescent field. Simulations based on the established model, along with experiments on brain tissues, demonstrate the controller’s effectiveness. Our system is expected to reduce human involvement and enable high-precision lighting for in vivo cell manipulation in the future.

Polar body biopsy has been widely applied in preimplantation genetic diagnosis for assisted reproductive technology. The key step in the polar body biopsy is picking up the polar body from the oocyte/embryo using a micropipette. Unfortunately, the almost transparent appearance of the polar body as well as its dynamic drift when the micropipette approaches it inside the cell makes it a challenging task to pick it up with less cytoplasm loss for the cell. The unnecessary cytoplasm loss in the picking up process of the polar body easily causes damage to the development competence of the cell and may lead to disturbances to the biopsy results of the polar body. This paper proposes a precise robotic picking up method of polar bodies with less cytoplasm loss for biopsy purposes. First, a defocus imaging method is proposed to locate the polar body with an almost transparent appearance. Then, the dynamic drift of the polar body with the micropipette moving inside the cell is modeled online based on force analysis to determine an appropriate trajectory for the micropipette to approach the polar body. Further, an Active Disturbance Rejection Controller (ADRC) is designed to move the micropipette along the desired trajectory to approach the polar body and then aspirate it into the micropipette. The experimental results on porcine oocytes demonstrate that our system is capable of localizing the polar body with a success rate of 95% and an average error of$1.12\pm 0.14~\mu $m. Moving along the determined trajectory, the micropipette is capable of approaching the edge of the polar body with an average error of$1.84\pm 0.31~\mu $m (n =20), which is only 11% of the results obtained without dynamic drift estimation of the polar body. With this advantage, our system picks up the polar body with a close 60% improvement in success rate (95% vs 60%) and only half of the average cytoplasm loss (5% vs 10%) in comparison to operation results without dynamic drift estimation. Note to Practitioners—Picking up of polar body from the oocyte/embryo using a micropipette is a vital operation in the polar body biopsy. Precisely picking up the polar body with less cytoplasm is vital to maintaining the developmental competence of the embryo/oocyte and reducing disturbances to biopsy result. This article presented a precise robotic picking up process of polar body. This process introduced defocus imaging method for polar body localization, dynamic drift estimation of polar body, and micropipette trajectory design and motion control by Active Disturbance Rejection Controller (ADRC). Experimental results have demonstrated the efficiency of the proposed robotic picking up process. Application of this process may provide an economical and practical method to carry polar body biopsy for practitioners.

Cell transportation is a key step in many biological applications. Currently, most cell transportation research relies on microscopic vision feedback, which limits its application at places when a microscopic view is unavailable. For the first time, this paper proposed a robotic microscopic view-free cell transportation method based on pressure variation model inside a self-made bilayer micropipette. The bilayer micropipette, made by inserting a thin micropipette into a thicker one, picks the target cell with the outsider micropipette and holds it with the inner micropipette in cell transportation. To achieve this, the appropriate fluidic forces to pick and place the cell were determined based on force analysis. Then, the appropriate inner diameter of the inner thin micropipette was determined based on the micropipette aspiration model. Further, the pressure variation inside the bilayer micropipette during holding and releasing process were modeled to conduct a robotic microscopic vision-fee cell transportation. Experimental results demonstrated that our system was capable of transporting zebrafish embryos at an average speed of 25 s/cell with a success rate of 90%, with a transportation efficiency comparable to the related methods with microscopic view feedback. Besides, ignorable mechanical harm to the development competence of the zebrafish embryos was confirmed through culturing experiments. Note to Practitioners—Cell transportation is an important step in many biomedical researches. Currently, cell transportation usually relies on microscopy for picking up, and releasing cells, limiting its application in special environments without microscopic view feedback. In this paper, a robotic microscopic vision-free cell transportation method is proposed for the first time. In this method, cells are transported within a self-made bilayer micropipette. Pressure variations inside the micropipette are utilized to determine cell picking up and release. This robotic transport method demonstrated a high success rate and low harm to cells in experiments. Our research may contribute to the future development of vision-free and fully automated robotic cell culture techniques. Our research may spike inspiration for future research on microscopic vision-free cell transportation in fully automated cell culture system.

During somatic cell nuclear transfer (SCNT), precise removal of the oocyte genetic material is critical. However, due to the invisibility of the genetic material (spindle) under brightfield observation and the potential displacement caused by the movement of micropipettes, accurately positioning the micropipette at the spindle poses a significant challenge. This study introduces an approach for optimal spindle removal by predicting its position. Initially, the polarization imaging system visualized the oocyte spindle, while a Multi-Feature Adaptive Kernel Correlation Filter (MFAKCF) algorithm tracked the spindle with 92.84% accuracy. Subsequently, enhancements were made to the Nonlinear Mass-Spring-Damper (NMSD) model to simulate live oocyte mechanical characteristics. Adjustments to NMSD model parameters simulated spindle displacement variations under diverse experimental conditions. Finally, the optimal spindle removal position was determined using NMSD model to simulate the micropipette approach to the spindle and the resulting position of spindle displacement. Experimental validation showed that the predictive accuracy of this model was 97.26%, with an average positional error of$0.4~\mu $m. Using this approach method can reduce cytoplasm loss to 4.5% and have a 100% enucleation success rate. Thus, the proposed prediction based optimal spindle removal position method can effectively anticipates spindle final positions, aiding in minimizing cytoplasmic loss during spindle removal. Note to Practitioners—Accurate oocyte nucleus removal is crucial for somatic cell nuclear transfer (SCNT) but challenging due to spindle invisibility under brightfield microscopy and displacement caused by micropipette movements. This study proposes a predictive method to address these issues. The polarization imaging system was used to visualize the spindle, and a MFAKCF tracking algorithm enabled real-time tracking. An enhanced NMSD model simulated spindle displacement, enabling accurate position prediction and reducing cytoplasmic loss during removal. This method offers a reliable tool for precise spindle removal in SCNT and broader applications in biomedical micromanipulation.

Patch clamp technique is a ”gold standard” approach to the study of ion channel biophysics and pharmacology. Due to the complexity of the operation and the heavy reliance on the experience of the experimenter, more and more researchers are focusing on patch clamping automation. In manual patch clamping, the operators patch the neurons in the specific brain region by experience, which lacks quantitative indicators of brain region selection. At present, the existing automated patch clamp focuses more on searching and measuring single cells under a high magnification lens, but ignores the localization of different brain regions under a low magnification lens, affecting the efficiency and application of the automated system. Addressing this, a 2D-3D registration method is proposed to register the experimental microscopic image of brain slices onto the brain atlases. This method is then embedded into the patch clamp system for brain region selection and electrophysiological recordings of neurons in multiple brain regions. It is the first time to achieve automated selection of brain regions in patch clamp experiments. The experimental results indicate that brain slice registration and brain region selection completely meet the requirements of the patch clamp experiment. Compared with the original robotic patch clamp process, the total time for the whole-cell patch clamp initialization with the improved process is reduced by 280 seconds, and the efficiency is increased by 42%. Meanwhile, the time for brain region switching is reduced to$1/6$of the original. In the future, this method will be applied to automatic image-guided patch clamp systems.Note to Practitioners—Calibrating the depth of the experimental brain slice in the brain and dividing the distribution of each brain region in the brain slice are the most important preparations for electrophysiological experiments. In this paper, we design 2D-3D registration to establish a method for brain region selection and robotic patch clamping of different brain regions in one brain slice. We have successfully detected the electrophysiological signals of neurons in 3 major brain regions, including the visual region, auditory region, and hippocampus. The experimental results show that the method proposed in this paper is suitable for the existing robotic patch clamp system and greatly improves the degree of automation.

Zebrafish microinjection is widely used in vascular biology, neurology, and other research areas. The microinjection of zebrafish larvae is a complicated 3D task because zebrafish larvae are independent individuals with complex body structures compared with biological cells. In this study, we propose an automatic high-throughput injection system for zebrafish larvae under an optical inverted microscope. The proposed system combines the fixing device design, object detection, visual positioning, and 3D injection path planning to improve the survival rate of zebrafish after injection. Experimental results demonstrated the capability and efficiency of the proposed system, which achieved a success rate of 95.0% and a survival rate of 98.7% for zebrafish injection. The system can be applied to various biomedical and biochemical experiments.Note to Practitioners—Zebrafish larvae have long been an important model organism in neuroscience, biomedicine, and drug discovery. In various methods for external substance transportation into the zebrafish, microinjection is more efficient but also more difficult. At present, high-throughput injection of zebrafish larvae is mainly based on traditional manual operations. The injection speed and success rate of manual injection gradually decrease with the increase in zebrafish numbers. In this study, an automatic injection system is designed and implemented for high-throughput zebrafish larvae injection. We first use a simply designed mold to fix batch zebrafish larvae. Then we propose a coarse-fine two-step positioning method and perform a 3D injection path planning to achieve precise microinjection. The proposed injection system frees the operators from the repetitive and tedious injection work, providing technical support for the life science experiment.

Single-cell transportation is one of the most common cell operations. Transporting cells with micropipettes is convenient for a wide range of biomedical applications. For high-efficiency cell transportation, the cells must be aspirated into the orifice of a micropipette. However, this is very difficult to achieve, as there is relative movement between the cell and the culture medium when the fluid drives the cell in the culture medium. It is crucial to use cell dynamics rather than fluid dynamics as the control objects to improve control performance and stop the cell immediately when it approaches the micropipette. In this study, the cell dynamics were modeled using a second-order model by integrating the dynamic model between the fluid and the cell into a first-order fluid dynamic model. A backstepping controller-based extended state observer was proposed to control the cell movement inside the micropipette. Experiments demonstrated that the proposed controller could aspirate cells into the orifice of the micropipette with high accuracy and no overshoot. Furthermore, the proposed controller was applied to automated somatic cell nuclear transfer, and it significantly boosted operational efficiency. Note to Practitioners—The need to apply advanced automation methods to transfer cells in life sciences has increased at a steady pace. The key feature of such systems is the ability to select and transfer cells at a predetermined position in space and time for biological applications. We propose a cell positioning control method based on vision-guided robotics that can directly aspirate cells to specified positions near the orifice of a micropipette. In somatic cell nuclear transfer, the proposed method of transferring somatic cells into oocytes occurs at a faster pace than manual operation. This provides essential functionality for single-cell transfer and is an appropriate technology for practitioners with this functional requirement.

Cell rotation is widely used to adjust cell posture in sub-cellular micromanipulations. The trajectory planning of the injection micropipette is needed, so that the cells can be rotated with the minimum deformation to reduce cell damage and keep cell viability. Due to the uncertainty of cell properties and manipulation environment, it is difficult to identify the parameters of the mechanical models in traditional robotic cell rotation methods. In this paper, deep reinforcement learning is introduced into cell manipulation for the first time to perform trajectory planning of the micropipette. We first abstract the cell rotation process by using the mechanical model and microscopic vision techniques and build a cell rotation simulation environment. Then we design a reward function by combining various factors of cell rotation and implement a reinforcement learning framework based on deep Q-learning (DQL). Finally, we train the cell rotation process based on the deep reinforcement learning algorithm. The simulation results indicate the proposed DQL agent achieved an average success rate of 97% without useless exploration. Moreover, the proposed method rotated the cells in a way that causes less mechanical damage than humans, demonstrating the DRL ability for cell rotation with high efficiency and low cell damage.

Somatic cell nuclear transfer (SCNT), which is an important procedure in cloning, has been conducted manually for decades. The operating efficiency drops sharply in batch SCNT because of the long-time observation under microscopy and the time-wasting traditional process. Though the operating time was reduced by robotic SCNT in previous studies, the traditional operating process was still used. In this article, we designed a new robotic batch SCNT process based on a microfluidic groove and two micropipettes in parallel. By using this new SCNT process, the operating area switching, objective lens conversing, and focusing on traditional SCNT process were eliminated, and oocyte localization was simplified, which saved much operating time. Experimental results showed that the new robotic batch process reduced about 50 s (41.7%) compared with the manual process (proposed 70 s versus manual 120 s). A success rate of 93.3% (n = 30) and a survival rate of 96.4% were achieved (n = 28), which were similar to manual process. The new robotic batch SCNT method demonstrated a high degree of efficiency and reproducibility.

Robotic manipulation of deformable objects has been a classic topic in robotics. Compared to synthetic deformable objects such as rubber balls and clothes, biological cells are highly deformable and more prone to damage. This article presents robotic manipulation of deformable cells for orientation control (both out-of-plane and in-plane), which is required in both clinical (e.g., in vitro fertilization) and biomedical (e.g., clone) applications. Compared to manual cell orientation control based on empirical experience, the robotic approach, based on modeling and path planning, effectively rotates a cell, while consistently maintaining minimal cell deformation to avoid cell damage. A force model is established to determine the minimal force applied by the micropipette to rotate a spherical or, more generally, ellipsoidal oocyte. The force information is translated into indentation through a contact mechanics model, and the manipulation path of the micropipette is formed by connecting the indentation positions on the oocyte. An optimal controller is designed to compensate for the variations of mechanical properties across oocytes. The polar body of an oocyte is detected by deep neural networks with robustness to shape and size differences. In experiments, the system achieved an accuracy of 97.6% in polar body detection and an accuracy of 0.7° in oocyte orientation control with maximum oocyte deformation of 2.70 μm throughout the orientation control process.

Robotic manipulation of deformable objects (vs. rigid objects) has been a classic topic in robotics. Compared to deformable synthetic objects such as rubber balls and clothes, biological cells are highly deformable and more prone to damage. This paper presents robotic manipulation of deformable cells for orientation control (both out-of-plane and in-plane), which is required in both clinical (e.g., in vitro fertilization) and biomedical (e.g., clone) applications. Compared to manual cell rotation control based on empirical experience, the robotic approach, based on mathematical modeling and path planning, effectively rotates a cell while consistently maintaining minimal cell deformation to avoid cell damage. A force model is established to determine the minimal force applied by the micropipette to rotate a spherical or more generally, an ellipsoidal mouse oocyte. The force information is translated into indentation through a contact mechanics model, and the manipulation path of the micropipette is formed by connecting the indentation positions on the oocyte. A compensation controller is designed to compensate for the variations of mechanical properties across cells. The polar body of an oocyte is detected by deep neural networks with robustness to shape and size differences. Experimental results demonstrate that the system achieved an accuracy of 97.6% in polar body detection and an accuracy of 0.7° in oocyte orientation control with maximum oocyte deformation of 2.69 μm.

Magnetic micromanipulation is an untethered technique and has enabled numerous applications in the scale of millimeters to micrometers from the tissue level to cell level. However, existing systems are not capable of maneuvering a sub-micrometer object for precise force control, preventing the realization of intracellular manipulation or `fantastic voyage' inside a single cell. The magnetic micromanipulation task achieved in this work is sub-micrometer position control and piconewton force control of a sub-micron (0.7 μm) magnetic bead inside a single human bladder cancer cell (RT4). The magnetic bead was 3D positioned in the cell using a generalized predictive controller that effectively tackled the control challenge caused by the slow visual feedback (1 Hz) from high-resolution confocal microscopy. The average positioning error was quantified to be 0.43 μm, which is slightly larger than Brownian motion-imposed constraint (0.31 μm). The system is capable of three-dimensionally applying a maximum force of 60 pN with a resolution of 4 pN. In experiments, a 0.7 μm magnetic bead was controlled to move from an initial position in a cell to target positions on the cell nucleus. Force-displacement data were obtained from multiple locations along the cell nucleus' major and minor axes. The results revealed, for the first time, significantly higher stiffness exists in the cell nucleus' major axis than the minor axis. This stiffness polarity was likely attributed to the aligned stress fibers of actin filament inside the cells.

In this paper, a robotic cell rotation method based on the minimum rotation force is presented to adjust oocyte orientation in biological applications. In this method, the minimum rotation force, which can control the rotation angle (RA) of the oocyte quantitatively and generate minimum oocyte deformations, is derived through a force analysis on the oocyte in rotation. To exert this force on the oocyte, the moving trajectories (MT) of the injection micropipette (IM), are determined using mechanical properties of the oocytes. Further, by moving the IM along the designed MT, the rotation force control is achieved. To verify the feasibility of this method, a robotic rotation experiment for batch porcine oocytes are performed. Experimental results demonstrate that this system rotates the oocyte a10t an average speed of 28.6s/cell and with a success rate of 93.3%. More importantly, this method can generate much less oocyte deformations during cell rotation process compared with the manual method, while the average control error of RA in each step is only 1.2° (versus averagely 8.3° in manual operation), which demonstrates that our method can effectively reduce cell deformations and improve control accuracy of the RA. Note to Practitioners - Using an IM to rotate the oocyte is the most popular method to adjust oocyte orientations in many cellular biological applications. To rotate the oocyte precisely and reduce mechanical damages to the oocyte, the rotation force exerted on the oocyte should be estimated and controlled precisely. However, it is a challenging task and has not been resolved well. This paper calculates the minimum rotation force through the force analysis on oocyte and uses it to improve control accuracy of the RA and limit the cell deformations. Using calibrated oocyte mechanical properties, the MT of the IM corresponding to the minimum rotation force is designed. Then, by moving the IM along designed MT, the rotation force control is achieved online. This method does not rely on the force sensors and is realized on traditional micro-operation systems. Coupled with previous works, this method is able to operate batch oocytes one by one. Therefore, it can easily be applied in biological applications and replace manual operations.