High-speed object tracking holds significant relevance across robotic domains, such as drones and autonomous driving. Compared to conventional cameras, event cameras are equipped with the ability to capture object motion information at exceptionally high temporal resolution with relatively low power consumption and remain immune from motion-blurring effects. Regrettably, many existing methods adopt a framebased approach by stacking events into Event Frame, which overlooks the sparsity and high temporal resolution of events. This approach is also reliant on the huge pre-training backbone and reaches a performance plateau but demands unrealistically large networks and high power consumption, rendering it impractical for real-time applications in battery-constrained robotic scenarios. In this paper, we propose an efficient and effective single-modality tracker using Point Cloud representation named E2B (Event to Box). By directly handling the raw output of event cameras without dataformat transformation, E2B leverages events' coordinate guidance to accurately map Event Cloud features to 2D bounding boxes. Moreover, E2B incorporates the pyramid structure into the multi-stage feature extraction architecture to effectively track objects across diverse scales. In the experiments, E2B performs outstandingly on two large-scale and one synthetic event-based tracking datasets, covering both indoor and outdoor environments, as well as rigid and non-rigid objects.

Recording fast motion in a high FPS (frame-per-second) requires expensive high-speed cameras. As an alternative, interpolating low-FPS videos from commodity cameras has attracted significant attention. If only low-FPS videos are available, motion assumptions (linear or quadratic) are necessary to infer intermediate frames, which fail to model complex motions. Event camera, a new camera with pixels producing events of brightness change at the temporal resolution of μs (10–6second), is a game-changing device to enable video interpolation at the presence of arbitrarily complex motion. Since event camera is a novel sensor, its potential has not been fulfilled due to the lack of processing algorithms. The pioneering work Time Lens introduced event cameras to video interpolation by designing optical devices to collect a large amount of paired training data of high-speed frames and events, which is too costly to scale. To fully unlock the potential of event cameras, this paper proposes a novel TimeReplayer algorithm to interpolate videos captured by commodity cameras with events. It is trained in an unsupervised cycleconsistent style, canceling the necessity of high-speed training data and bringing the additional ability of video extrapolation. Its state-of-the-art results and demo videos in supplementary reveal the promising future of event-based vision.

In manual cell injection the operator relies completely on visual information for task feedback and is subject to extended training times as well as poor success rates and repeatability. From this perspective, enhancing human-in-the-loop intracellular injection through haptic interaction offers significant benefits. This paper outlines two haptic virtual fixtures aiming to assist the human operator while performing cell injection. The first haptic virtual fixture is a parabolic force field designed to assist the operator in guiding the micropipette's tip to a desired penetration point on the cell's surface. The second is a planar virtual fixture which attempts to assist the operator from moving the micropipette's tip beyond the deposition target location inside the cell. Preliminary results demonstrate the operation of the haptically assisted microrobotic cell injection system.

The ability of efficiently delivering soluable/insoluable drug compounds or biomolecules into individual biological cells and quantifying their cellular responses is important for genetics, proteomics, and drug discovery. This paper presents a fully automated system for zebrafish embryo injection, which overcomes the problems inherent in manual injection, such as human fatigue and large variations in success rates due to poor reproducibility. Based on ldquolooking-then-movingrdquo control, the microrobotic system performs injection at a speed of 15 zebrafish embryos (chorion unremoved) per minute. Besides a high injection speed that compares favorably with that of a highly proficient injection technician, a vacuum-based embryo holding device enables fast immobilization of a large number of zebrafish embryos, shortening the embryo patterning process from minutes to seconds. The recognition of embryo structures from image processing identifies a desired destination inside the embryo for material deposition, together with precise motion control resulting in a success rate of 100%. Carefully tuning suction pressure levels as well as injection and retraction speeds produced a high survival rate of 98%. The quantitative performance evaluation of the automated system was based on the continuous injection of 250 zebrafish embryos. The technologies can be extended to other biological injection applications such as the injection of mouse embryos,Drosophilaembryos, andC.elegansto enable high-throughput biological and pharmaceutical research.

Injection of foreign materials (e.g., DNA, RNAi, sperm, protein, and drug compounds) into individual cells has significant implications in genetics, transgenics, assisted reproduction, and drug discovery. This video presents a microrobotic system for fully automated zebraflsh embryo injection, which overcomes the problems inherent in manual operation, such as human fatigue and large variations in success rates due to poor reproducibility. Based on computer vision and motion control, the automated microrobotic system is capable of immobilizing a large number of zebraflsh embryos into a regular pattern within seconds and injecting 15 embryos (chorion unremoved) per minute with a success rate, survival rate, and phenotypic rate all close to 100%.

This paper presents a semi-automated microrobotic system for adherent cell injection. Different from embryos/oocytes that have a spherical shape and regular morphology, adherent cells are flat with a thickness of a few micrometers and are highly irregular in morphology. Based on computer vision microscopy and motion control, the system coordinately controls a three-degrees-of-freedom microrobot and a precision XY stage. The microrobotic system demonstrates an injection speed of 25 endothelial cells per minute with a survival rate of 96% and a success rate of 82% (n=1012). The system has a high degree of performance consistency. It is immune to operator proficiency variations and from human fatigue, requiring a human operator to select injection destinations through computer mouse clicking as the only operator intervention. The microrobotic adherent cell injection system makes the injection of thousands of adherent cells practical and will enable our testing of intracellular behavior of semiconductive quantum dots (QDs).

Manipulation and characterization of individual biological cells require cellular forces be precisely measured in real time. This paper presents a computer vision-based cellular force measurement platform that allows for the use a single vision sensor to simultaneously obtain two forms of feedback (i.e., vision and force). A novel silicone elastomer-based cell holding device and a sub-pixel visual tracking algorithm are developed. Deflections of elastic, low-stiffness structures are visually tracked, and material deflections are subsequently transformed into cellular forces. Experimental results demonstrate that the current vision-based force sensing system is capable of performing robust cellular force measurements at a full 30 Hz with a 3.7 muN resolution. Importantly, the vision-based cellular force sensing framework established in this study is not scale or cell line dependent. The device design, visual tracking algorithm, and experimental technique form a powerful framework that permits visually resolving cellular forces in real time with a picoNewton (26 pN) resolution for applications in single cell manipulation and characterization