Segment Anything Model (SAM) exhibits remarkable zero-shot segmentation capability; however, its prohibitive computational costs make edge deployment challenging. Although post-training quantization (PTQ) offers a promising compression solution, existing methods yield unsatisfactory results when applied to SAM, owing to its specialized model components and promptable workflow: (i) The mask decoder's attention exhibits extreme activation outliers, and we find that aggressive clipping (even 100x), without smoothing or isolation, is effective in suppressing outliers while maintaining performance. Unfortunately, traditional distribution-based metrics (e.g., MSE) fail to provide such large-scale clipping. (ii) Existing quantization reconstruction methods neglect semantic interactivity of SAM, leading to misalignment between image feature and prompt intention. To address the above issues, we propose SAQ-SAM in this paper, which boosts PTQ for SAM from the perspective of semantic alignment. Specifically, we propose Perceptual-Consistency Clipping, which exploits attention focus overlap to promote aggressive clipping while preserving semantic capabilities. Furthermore, we propose Prompt-Aware Reconstruction, which incorporates image-prompt interactions by leveraging cross-attention in mask decoder, thus facilitating alignment in both distribution and semantic. Moreover, to ensure the interaction efficiency, we design a layer-skipping strategy for image tokens in encoder. Extensive experiments are conducted on various SAM sizes and tasks, including instance segmentation, oriented object detection, and semantic segmentation, and the results show that our method consistently exhibits advantages. For example, when quantizing SAM-B to 4-bit, SAQ-SAM achieves 11.7% higher mAP than the baseline in instance segmentation task.

Training a language model to be both helpful and harmless requires careful calibration of refusal behaviours: Models should refuse to follow malicious instructions or give harmful advice (e.g."how do I kill someone?"), but they should not refuse safe requests, even if they superficially resemble unsafe ones (e.g. "how do I kill a Python process?"). Avoiding such false refusal, as prior work has shown, is challenging even for highly-capable language models. In this paper, we propose a simple and surgical method for mitigating false refusal in language models via single vector ablation. For a given model, we extract a false refusal vector and show that ablating this vector reduces false refusal rate while preserving the model's safety and general capabilities. We also show that our approach can be used for fine-grained calibration of model safety. Our approach is training-free and model-agnostic, making it useful for mitigating the problem of false refusal in current and future language models.

Magnetic levitation control provides a promising solution for wireless capsule endoscopy by minimizing tissue pressure and reducing patient discomfort and risks. Compared to electromagnetic actuation systems, using permanent magnets as the actuation source provides stronger magnetic fields at a lower cost. However, permanent magnet-based actuation systems are highly nonlinear and necessitate complex system modeling. In this study, we propose a deep reinforcement learning (DRL)-based control method for permanent magnet levitation. This approach utilizes DRL to learn optimal control strategies in complex and dynamic environments, without the need for detailed modeling of nonlinear physical phenomena such as magnetic interactions, manipulator dynamics, and capsule-environment interactions. A simulation environment was developed where a manipulator equipped with a permanent magnet actuates the internal magnet of a capsule. A multistage reward function and a recurrent neural network with memory capabilities were designed to improve control stability and accuracy. After Sim-to-Sim transfer, the proposed method successfully controlled five degrees of freedom, achieving navigation accuracies of 1.68 mm in the training environment and 9.18 mm in the testing environment. The system maintained stable performance and high accuracy while supporting dynamic tracking at speeds of up to 30 mm/s. Additionally, the method demonstrated significant resistance to disturbances.

While magnetic micro-robots have demonstrated significant potential across various applications, including drug delivery and microsurgery, the open issue of precise navigation and control in complex fluid environments is crucial for in vivo implementation. This paper introduces a novel flow-aware navigation and control strategy for magnetic micro-robots that explicitly accounts for the impact of fluid flow on their movement. First, the proposed method employs a Physics-Informed U-Net (PI-UNet) to refine the numerically predicted fluid velocity using local observations. The predicted velocity is then incorporated into a flow-aware A* path planning algorithm, ensuring efficient navigation while mitigating flow-induced disturbances. Finally, a control scheme is developed to compensate for the predicted fluid velocity, thereby optimizing the micro-robot’s performance. A series of simulation studies and real-world experiments are conducted to validate the efficacy of the proposed approach. This method enhances both planning accuracy and control precision, expanding the potential applications of magnetic micro-robots in fluid-affected environments typical of many medical scenarios.

Colon endoscopic robots represent a promising screening modality for the visualization of colon cancers with high sensitivity. However, current colonoscopy robots are often characterized by intricate and bulky mechanical structures, which pose practical challenges when moving through the complex and narrow environment of the colon. Moreover, these robots are typically equipped with a single camera, limiting their ability to accurately estimate the depth of haustral folds in the colon, which is of great importance for the active colonic navigation of the robots. To address these challenges, we develop a track-based stereoscopic endoscopic robot (TSER) which is equipped with four tracks positioned at the corners of its body. This innovative design maximizes the contact between the tracks and the colon wall, enhancing maneuverability. The tracks are constructed from de-molded polydimethylsiloxane (PDMS) and incorporate micro-patterns on their outer surfaces. We have proposed a straightforward strategy for detecting haustral folds using TSER’s stereo camera, which allows for precise identification of their position and depth. The TSER achieves an average motion speed of 9.8 mm/s in a bellows tube that contains silicone oil and a speed of 5.2 mm/s in an exvivo porcine intestinal segment. Impressively, the TSER boasts an 88.11% accuracy rate in haustral fold depth estimation, surpassing the performance of existing geometric shape fitting methods. These results demonstrate that the TSER holds great potential for effective and efficient movement and inspection within the colon, offering a promising solution for improved colon cancer screening.

Capsule endoscopic robot holds great promise for the early diagnosis of gastrointestinal diseases without causing discomfort to patients. However, currently available active capsule endoscopic robots suffer from issues such as complex structure, poor mobility, large size, and high cost, which have hindered their widespread adoption and resulted in a lower screening rate for gastrointestinal diseases. To address these challenges, this paper proposes a highly integrated propeller-driven capsule endoscopic robot (PCER) system that integrates STM32 processor, magnetic sensor, IMU, RF communication unit, and motor drive. The micro propeller of the PCER has been analyzed through finite element simulation to ensure its efficiency. FLUENT software has been utilized to simulate the fluid force acting on the PCER as it moves through a liquid medium. The results of the simulation are then used to determine the optimal pitch angle for the robot's movement. The thrust generated by the capsule robot propellers has been measured using a lever mechanism to investigate the relationship between the thrust and voltage applied to the motors. The experiments confirmed that the PCER is capable of performing flexible motions within fluid environments, such as changing pitch angle during movement, passing circular obstacles, horizontal motion, and spiral ascent. These findings demonstrate the feasibility of the proposed PCER as an effective tool for non-invasive early screening of gastrointestinal diseases.

ICSI (Intracytoplasmic sperm injection) is one of the most effective treatments for severe male infertility. During the implementation of the ICSI, it is necessary to perform the three-dimensional positioning of the tip of the glass injection micropipette. At present, the process is mainly controlled by skilled operators. Such manual operation is time-consuming and likely to cause micropipette damage or even serious damage to the oocyte membrane due to the inaccurate positioning. In this paper, an automatic system was developed for micropipette positioning. The microscopic positioning is carried out by autofocusing and planar positioning. A search strategy based on Dynamic Curve Fitting (DCF) was proposed to avoid local extremum issues due to the noises during the autofocusing. The proposed search strategy is adaptive to different focus algorisms, making the design of autofocusing becomes simpler. Besides, we proposed a planar positioning algorithm that can quickly and accurately obtain the tip position of the micropipette at the focusing plane. Finally, the visual servo control is employed to move the micropipette to the center of the vision. The experimental results demonstrated that the DCF method can accurately (mean error: 3 μm) find the focusing plane when applying random Gaussian noise with a mean value of 0 and a variance of 1 to the focus measure. By contrast, both the hill-climbing search method and the Fibonacci search method cannot work properly. The tip positioning algorithm provides a real-time in-plane tip positioning at a frame rate of 40 Hz with an average accuracy of 11 μm.

Magnetic manipulation provides a versatile, remote, noninvasive, and cost-effective strategy in a variety of applications. Till now, many different configurations of magnetic manipulation systems have been developed to address different needs on force, torque, accuracy, and accessibilities. Magnetic field mapping can help to explore the exact map of the magnetic field in the working space and guarantee the homogeneity of the magnetic field. In this paper, a flexible mapping method is employed to solve the scalar potential of the magnetic source by using the separation of variables in Cartesian coordinates. Levenberg-Marquardt Algorithm (LMA) and Whale Optimization Algorithm (WOA) are set to the solver of the model. The work is evaluated in the mapping of an eight-pole magnetic manipulation system. The result of numerical simulation shows that the coefficient of determination R2of the model reaches99.81%, and the actual system mapping obtains R2value of 99.57%. This technique can directly be used to calculate the magnetic flux density and gradient field in a short period (≈1ms). Finally, the manipulation of a permanent magnet under the control magnetic field mapping and PID controller demonstrates the effectiveness of the proposed method.

Particle tracking has emerged as a powerful tool for investigating the swarm control of microrobots and the dynamic biological processes in the life sciences. In seed plants, pollen tubes, a part of the male gametophyte, are excellent models for understanding plant growth and cellular behavior, because vesicle motion within pollen tubes reveals important information about vesicle function and interactions. Conventional vesicle tracking is based on spatiotemporal image analysis, which requires high-quality images and vesicles with constant velocity. For in vivo tracking, vesicles may disappear in some frames, and image sequences may have spatial and temporal distortions, which hamper vesicle tracking for broader applications. In this paper, we studied intracellular motion during pollen tube growth with an optical flow method. Streaming images from confocal and optical microscopes were recorded to study the intracellular motion of vesicles of different size. Local motion for each vesicle was detected using a local displacement vector field. The displacement from two adjacent frames was then calculated. The flow field shows information such as the dynamics of vesicle secretion, endocytosis, exocytosis, and cytoskeletal stability. Vesicles from different regions inside the tube were tracked simultaneously with a Kanade-Lucas-Tomasi (KLT) feature matching algorithm. The spatial and temporal characteristics of intracellular vesicles were evaluated. The proposed methods can be of great use for studying the dynamics of fluorescently tagged particles in biological systems.

Helical structure assembled by hydrogel microfibers is significant for culture of smooth muscle cells. However, the helical structure is only fabricated at the macroscale, while the fabrication of helical microstructure is still a challenge due to the lack of assembly method. In this paper, we propose a robotics-based assembly method to handle such challenge. An electromagnetic needle (EMN) is employed as end-effector to magnetically reel the microfiber encapsulating magnetic nanoparticles around a micropillar, and a dual-ring structure is designed to keep the microfiber being attracted at the EMN tip. For enhancing the stability of tip attraction, the manipulation mode of anticlockwise pushing microfiber is established. Moreover, the interaction mechanism between EMN tip and microfiber is analyzed by developing a static force model, and then the key condition of stably reeling microfiber is concluded. Furthermore, a robotics-based motion trajectory of EMN tip is planned to achieve a smooth reeling process. Based on such planning, the size of dual-ring structure is further optimized to improve the success rate of reeling. Finally, the helical microstructure with there-turn coils is successfully fabricated.

3D tailor-made biodegradable scaffold integrated with biological cells or molecules is of great importance for tissue engineering. This paper addresses an improved method for exploring magnetic tweezers in patterning and aligning magnetic hydrogel fiber to fabricate large-scale engineered cell-hydrogel constructs. Magnetic hydrogel fibers were fabricated based on microfluidic device. The fabricated hydrogel fiber is made of alginic acid sodium and with a diameter of 34 μm. Magnetic nanoparticles is added into the alginic acid sodium solution to append magnetic material inside the fibers. The magnetic material inside the hydrogel fiber is regulated by the microfluidic device. Magnetic tweezers system based on solenoid electromagnet is utilized to evaluate the magnetic response of the magnetic hydrogel fiber. Evaluation results show the hydrogel fiber can be maneuvered by the proposed system with a positioning resolution of sub-micro level. The cultivation results of hydrogel fiber with C2C12 cells shows the potential for real applications of the proposed method in tissue engineering.

Recently the research about constructing 3 dimensional cell structures is very important for its great potential applications in tissue engineering. In this paper, we report a novel method of constructing multi-layered microstructures embedding cells via microfluidic devices. The on-chip fabrication of movable microstructures embedding fibroblasts (NIH/3T3) based on Poly (ethylene glycol) Diacrylate (PEGDA) was reported. Two approaches for assembling these movable microstructures were presented. One was a manual assembly method based on micromanipulation system and the other one was a self-assembly method based on microfluidic channel. Several manual assembly ways were demonstrated and a tube-shaped microstructure with 17 layers was assembled by an efficient assembly method. A novel microfluidic channel was presented for conducting self-assembly method and a 2-layered experimental microfluidic device was fabricated by Polydimethylsiloxane (PDMS). The self-assembly process of fabricated microstructures via this device was preliminarily demonstrated.

Magnetic field has been used for manipulation of micro-robots; in this research we use it for scaffold fabrication. Magnetic sugar particle (MSP) was used as porogen to control pore size, pore structure and pore density in the scaffold. We studied the influence of the strength of magnetic fields for controlling the coating thickness of unmagnetized MSPs during the fabrication of sheet-like scaffolds. The experimental relationship between magnetic flux density and the thickness of MSP layer was illustrated. Furthermore we investigated the infiltration capacity of poly(L-lactide-co-ε-caprolactone) (PLCL) which was used as scaffold material on the MSP clusters. 5% and 10% PLCL solutions were employed in the experiments. After polymer casting and removal of the sugar template, spherical pores were generated inside scaffold, the thickness evaluation of a single layer scaffold was carried out and the cytocompatibility experiment of NdFeB powder used in fabrication of MSP was confirmed with human umbilical vein endothelial cells.

This paper reports a magnetic steering method for controlled porogen fabrication of scaffold for blood vessel regeneration. The method described involves generating gradient magnetic field by a combination structure of Helmholtz coils and Maxwell coils to propel particularly prepared magnetic sugar particles (MSPs) moving in fluid environment at desired trajectory and forming a specified 3D shape as templates for particulate leaching. The movement properties of MSP are theoretically analyzed and the corresponding dynamic mechanics model is established. Further the magnetic field distributions inside the combined coils are calculated and optimal control parameters of coils configuration are obtained. Preliminary motion control experiment is also conducted to prove the feasibility of proposed method. The result demonstrates that MSP cluster can be manipulated with average speed of 0.25 mm/s for a cluster of 12 MSPs and 0.116 mm/s for a cluster of 37 MSPs with the proposed coil system and used for improving interconnection of MSP dot patterns.