Spatio-temporal alignment is crucial for temporal modeling of end-to-end (E2E) perception in autonomous driving (AD), providing valuable structural and textural prior information. Existing methods typically rely on the attention mechanism to align objects across frames, simplifying the motion model with a unified explicit physical model (constant velocity, etc.). These approaches prefer semantic features for implicit alignment, challenging the importance of explicit motion modeling in the traditional perception paradigm. However, variations in motion states and object features across categories and frames render this alignment suboptimal. To address this, we propose HAT, a spatio-temporal alignment module that allows each object to adaptively decode the optimal alignment proposal from multiple hypotheses without direct supervision. Specifically, HAT first utilizes multiple explicit motion models to generate spatial anchors and motion-aware feature proposals for historical instances. It then performs multi-hypothesis decoding by incorporating semantic and motion cues embedded in cached object queries, ultimately providing the optimal alignment proposal for the target frame. On nuScenes, HAT consistently improves 3D temporal detectors and trackers across diverse baselines. It achieves state-of-the-art tracking results with 46.0% AMOTA on the test set when paired with the DETR3D detector. In an object-centric E2E AD method, HAT enhances perception accuracy (+1.3% mAP, +3.1% AMOTA) and reduces the collision rate by 32%. When semantics are corrupted (nuScenes-C), the enhancement of motion modeling by HAT enables more robust perception and planning in the E2E AD.

Operating in inherently dynamic and unstructured environments, mobile robots must routinely navigate substantial uncertainties—particularly in crowded spaces or multi-door configurations. Yet mainstream navigation approaches often disregard such environmental unpredictability and underutilize historical task experience, leading to recurrent obstacle blockages that degrade both efficiency and safety. This work proposes an adaptive navigation method for uncertain environments using probabilistic topological mapping (PTM). We develop a Gaussian mixture model (GMM)-based PTM construction approach that extracts environmental feature nodes through path similarity and node density filtering, clusters nodes via GMM, and establishes topological connections to efficiently encode environmental characteristics. For adaptive navigation, we introduce: 1) a posterior heuristic algorithm for planning under uncertainty; 2) a topological edge analysis with re-planning and escape mechanisms for robustness; and 3) a dynamic particle filter for online probability updates. This framework enables robots to autonomously analyze environmental information and progressively optimize navigation strategies through probabilistic self-updating. Simulations and real-world experimental studies reveal that our method exhibits higher efficiency and safety compared to current mainstream navigation approaches.

To address the challenges posed by periodic gait-induced disturbances in legged robotic systems, we propose an enhanced visual SLAM framework that systematically incorporates gait-phase awareness into the mapping optimization process. Building upon the ORB-SLAM2 architecture, our methodology introduces two principal innovations: 1) a gait-synchronized pose prediction model that dynamically adjusts motion estimation parameters according to cyclic locomotion patterns and 2) an adaptive keyframe selection strategy that optimizes temporal sampling based on phase-dependent stability characteristics. By integrating real-time gait-phase detection with visual-inertial data fusion, our system demonstrates improved robustness against transient sensor perturbations and high-frequency vibrations inherent in dynamic locomotion. Quantitative evaluations reveal that this gait-adaptive optimization framework achieves superior tracking performance compared with conventional SLAM implementations, with lower absolute trajectory error, higher loop closure detection accuracy, and enhanced mapping consistency with less drift and overlap. The proposed architecture effectively decouples locomotion-induced sensor noise from true environmental observations, maintaining relatively high tracking precision even under severe gait disturbances (peak acceleration > 13m/s2). This advancement enables reliable long-term autonomous operation in unstructured environments where traditional vision-based SLAM systems typically suffer from error accumulation and mapping degradation.

SLAM (Simultaneous Localization and Mapping) has a wide application prospect in navigation of autonomous endoscopic minimally invasive surgery. However, traditional methods like ORB-SLAM2 often struggle in dynamic environments and exhibit significant modeling inaccuracies, particularly in bronchial scenes where respiratory motion induces continuous deformation of the airway structure. To address this challenge, we propose a novel monocular SLAM-based framework for dynamic bronchial environment reconstruction tailored to robotic lung invasive surgery. Firstly, a pseudo-static processing method that builds map sequences at identical respiratory phases across cycles was built. Furthermore, we present a voxel model optimization technique using curvature-consistent graph interpolation to refine the bronchial lumen surface, eliminating pores and redundancies in the voxel map. Experiments conducted on real patient demonstrate that our method has good accuracy and robustness. Compared to existing state-of-the-art methods, our framework achieves superior reconstruction completeness, showing strong potential for clinical use in robotic bronchoscopy.

Monocular endoscope-based reconstruction of dynamic 3D surgical fields is beneficial for both intraoperative manual/robotic manipulation and post-operative surgical skills training. However, the natural characteristics of tissue deformations with instruments and blood obscuration bring great challenges to 3D scene awareness, especially under sparse viewpoints limited by laparoscopic movements. In this work, we propose Endo-4SRF, an effective Neural Radiance Field (NeRF)-based method that can reconstruct deformable tissues with instruments stealth by solely relying on monocular endoscopic image flows from a single viewpoint. Specifically, to enhance the 3D reconstruction accuracy under the deficiency of depth ground truth, we devised a dynamic Gaussian-based neural sampling strategy, leveraging the depth inherently obtained from NeRF and the conjunction information inferred by a prior learning-based depth estimation network. Besides, we integrated the Signed Distance Function (SDF) and resolved its singularity problem by furnishing additional geometric constraints for the neural radiance field, thereby achieving precise reconstruction of dynamic scenes devoid of depth ground truth supervision. Furthermore, adopting spherical harmonic functions for color fitting has significantly improved our model's computational efficiency and rendering quality. We extensively performed cross validation experiments to verify the performance using public and in-house datasets. Our quantitative and qualitative results demonstrate remarkable superiority over the state-of-the-art (SOTA) approaches concerning depth prediction accuracy, image rendering quality, model training efficiency, and 3D reconstruction outcomes.

Robotic endoluminal surgery has gained tremendous attention for its enhanced treatments in gastrointestinal intervention, where navigating surgeons with monocular camera-based metric depth estimation is a vital sector. However, existing methods either rely on external sensors or perform poorly in terms of visual navigation. In this work, we present our M$^{3}$-Degrees Net, a novel monocular vision-guided and graph learning-based network tailored for accurate metric marching depth (MD) estimation. We first leverage a generative model to output a scale-free depth map, providing a depth basis in a coarse granularity. To achieve an optimized and metric MD prediction, a relational graph convolutional network with multi-modal visual knowledge fusion is devised. It utilizes shared salient features between keyframes and encodes their pixel differences on the depth basis as the main node, while a projection length-based node that predicts the MD on a proportional relationship basis is introduced, aiming to enable the network with explicit depth awareness. Moreover, to compensate for rotation-induced MD estimation bias, we model the endoscope's orientation changes as image-level feature shifts, formulating an ego-motion correction node for MD optimization. Lastly, a multi-layer regression network for the metric MD estimation with finer granularity is devised. We validate our network on both public and in-house datasets, and the quantitative results reveal that it can limit the overall MD error under 27.3%, which vastly outperforms the existing methods. Besides, our M$^{3}$-Degrees Net is qualitatively tested on the in-house clinical gastrointestinal endoscopy data, demonstrating its satisfactory performance even under cavity mucus with varying reflections, indicating promising clinical potentials.

The 6-degree-of-freedom (DoF) grasp method based on point clouds has shown significant potential in enabling robots to grasp target objects. However, most existing methods are based on the point clouds (2.5-D points) generated from single-view depth images. These point clouds only have one surface side of the object, providing incomplete geometry information, which misleads the grasping algorithm to judge the shape of the target object, resulting in low grasping accuracy. Humans can accurately grasp objects from a single view by leveraging their geometry experience to estimate object shapes. Inspired by humans, we propose a novel 6-DoF grasping framework that converts the point completion results as object shape features to train the 6-DoF grasp network. Here, point completion can generate approximately complete points from the 2.5-D points similar to the human geometry experience, and converting them into shape features is the way to utilize it to improve grasp efficiency. Furthermore, due to the gap between the network generation and actual execution, we integrate a score filter into our framework to select more executable grasp proposals for the real robot. This enables our method to maintain a high grasp quality in any camera viewpoint. Extensive experiments demonstrate that utilizing complete point features enables the generation of significantly more accurate grasp proposals, and the inclusion of a score filter greatly enhances the credibility of real-world robot grasping. Our method achieves a 17.8% success rate, higher than the state-of-the-art method in real-world experiments. Code and videos are available at https://github.com/ChengYaofeng/PCF-Grasp

In this work, a multimodal point cloud registration method using CT and video frames is proposed to optimize the modeling of the bronchial cavity environment. Preoperative CT data improve the quality of point clouds acquired from intraoperative video frames. Initially, preoperative CT scans are used to obtain bronchial point clouds and airway centerlines, while intraoperative bronchial point clouds and endoscope trajectories are captured in real-time using SLAM. Given that intraoperative frame-by-frame mapping cannot be directly globally registered, multi-modal point clouds undergo local segmentation. Subsequently, the preoperative bronchial airway centerlines guide iterative scaling and adjustment of the preoperative CT point clouds, achieving precise registration between the CT and the video frame point clouds. Experimental results demonstrate a rapid and accurate enhancement in the quality of the intraoperative bronchial point cloud, providing more precise maps of the cavity environment for surgical robots. The method is validated and evaluated using CT and video frame data collected from ex vivo pig lungs, achieving intraoperative mapping accuracy of 0.5 millimeter, respectively. These results surpass those of methods relying solely on SLAM for intraoperative mapping.

The motion of magnetic microrobots relies on an external magnetic field. Current electromagnetic propulsion systems are often complex in structural design, challenging in control strategies, and associated with high energy consumption. In this study, we introduce a mechano-electromagnetic hybrid actuation system that employs only a pair of Helmholtz coils to generate an oscillating uniform magnetic field, combined with a rotating platform to modify the field’s direction, thereby enabling the control of microrobot speed and direction. The properties of this hybrid actuation system were modeled and validated to analyze the magnetic field distribution within the workspace. The design and actuation of two microrobots are described in detail. Furthermore, a visual feedback closed-loop control strategy for the system was developed. A series of navigation control experiments were conducted, demonstrating that the proposed hybrid actuation system achieves effective microrobot actuation and precise control while reducing manufacturing costs and simplifying control strategies. The proposed novel mechano-electromagnetic hybrid structure contributes to the field of electromagnetically driven systems and magnetically controlled microrobots.

Over the past decade, significant advancements have been made in the research and industrialization of robotic systems for endovascular procedures, yet their clinical application remains relatively limited. Physicians commonly report that these robots lack certain intelligent assistive capabilities during procedures. There has been increasing interest and attempts to apply learning-centered algorithms to the training and enhancement of surgical robot skills. This paper proposes an autonomous navigation algorithm for interventional guidewires that is initially trained solely in a virtual simulation environment and subsequently deployed to a real-world robot. Experimental results demonstrate the feasibility of this approach for real-world applications. The proposed approach can help physicians reduce the learning curve for guidewire manipulation and elevate the robot to a higher level of autonomous operation, thereby breaking through the current bottleneck in the level of intelligence for clinical applications of interventional robots. It also holds promise for bringing intelligent transformation to future interventional procedures. Note to Practitioners—This work is motivated by the emerging need to increase the level of autonomy in robotic-assisted endovascular procedures, which has the potential to improve procedural efficiency, standardize procedures, and broaden the adoption of robotic systems in clinical practice. The proposed simulation-based reinforcement learning provides a safe and efficient method for training robotic systems, enabling them to master complex tasks in simulation environments prior to real-world application. The successful deployment of models trained in simulation onto physical robotic platforms demonstrates the feasibility of this method for real-world applications. The proposed simulation-based reinforcement learning method offers a promising and viable pathway for enhancing skill acquisition in endovascular interventional robots.

Docking is a common step in magnetic helical microrobot operations. However, the boundary effect causes helical microrobots to drift laterally near the container bottom. The mismatch between orientation and movement direction further complicates manual steering. In this paper, an automatic control method for on-plane docking of magnetic helical microrobots is proposed, which divides the docking process into preparation, alignment, and insertion stages. First, the docking path is planned according to the position and orientation of the helix and the target. Second, a steering-based path-following controller guides the helix swim to the ready position for alignment. Third, another path-following controller realizes the position alignment through rotational-frequency-controlled movement direction while locking the helix orientation aligned with the target. Finally, the docking is completed in the insertion stage. Experiments verified the automatic docking of helical microrobots with static targets, including inserting into a slender gap/tube, assembling with a micro-shaft axially, and leaning against a rod radially. This method can eliminate the negative impact of operator’s fatigue and inexperience on docking operations and has the potential to be applied in biomedical and microoperations.Note to Practitioners—This paper is motivated by the problem of docking magnetically driven helical-shaped swimming microrobots with target objects on Petri dishes, which is a common step in capture, loading, assembly, and delivery operations. Here, two closed-loop path-following controllers based on steering and rotational frequency control guide the microrobot to approach and align with the target respectively along a calculated path. In alignment, the non-ideal lateral drift of helical microrobots is controlled by rotational frequency for position alignment. Experiments verified the automatic docking of micro-helixes with static targets, including inserting into a slender gap/tube, connecting with a micro-shaft, and leaning against a rod.

In many intelligent edge applications involving Autonomous Mobile Robots (AMRs), efficient and real-time localization and mapping is a fundamental issue. Extended Kalman Filter Simultaneous Localization and Mapping (EKFSLAM) algorithm is a classic and successful solution to realize localization and mapping, while it is computationally intensive and poses a challenge for real-time tasks in small and micro robots. To address this issue, this work proposes an energy-efficient, highframe-rate, and reconfigurable EKF-SLAM processor. Firstly, a heterogeneous dual-core architecture is proposed to enable full acceleration of both matrix operations and nonlinear calculations in EKF-SLAM at the hardware architecture level. Secondly, a Reconfigurable Matrix Accelerator (RMA) and Reconfigurable Nonlinear Accelerator (RNA) are proposed to maximize data reuse and support diverse nonlinear functions at the data flow level. Thirdly, a data property-aware strategy is proposed at the data property level, which exploits matrix symmetry, sparsity, and dependency to reduce storage significantly and eliminate redundant computations. FPGA validation results show that the proposed design can achieve a frame rate of 774 fps and an energy efficiency of 0.66 mJ/frame, when performing mapping processes involving 60 landmarks at 100 MHz.

Online Continual Learning (OCL), as a core step towards achieving human-level intelligence, aims to incrementally learn and accumulate novel concepts from streaming data that can be seen only once, while alleviating catastrophic forgetting on previously acquired knowledge. Under this mode, the model needs to learn new classes or tasks in an online manner, and the data distribution may change over time. Moreover, task boundaries and identities are not available during training and evaluation. To balance the stability and plasticity of networks, in this work, we propose a replay-based framework for OCL, named Contrastive Correlation Preserving Replay (CCPR), which focuses on not only instances but also correlations between multiple instances. Specifically, besides the previous raw samples, the corresponding representations are stored in the memory and used to construct correlations for the past and the current model. To better capture correlation and higher-order dependencies, we maximize the low bound of mutual information between the past correlation and the current correlation by leveraging contrastive objectives. Furthermore, to improve the performance, we propose a new memory update strategy, which simultaneously encourages the balance and diversity of samples within the memory. With limited memory slots, it allows less redundant and more representative samples for later replay. We conduct extensive evaluations on several popular CL datasets, and experiments show that our method consistently outperforms the state-of-the-art methods and can effectively consolidate knowledge to alleviate forgetting.

This paper focuses on developing modern, efficient, lightweight object detection a method for sweeping robots while trading off parameters, FLOPs and performance. In order to comply with the real-time requirements of sweeping robots, a method combining layer pruning and channel pruning is used to compress the initial model before optimization. First, a C2f module based on shunt gradient is introduced to lighten the network. Second, a$\mathrm{C}3_{-}\mathbf{DCN}$(Deformable convolutional networks) module is used to fit the shape and size of the object when sam-pling. Finally, a Convolutional Block Attention Module(CBAM) is added behind the backbone network to enhance the extraction of important features. In the experimental studies, we compare our method with state-of-the-art methods, and the results reveal that the new model accuracy can reach 88.5 % with only 2.27M parameters and 5.5G FLOPs. Furthermore, the experiment results show that our proposed method achieves a better balance between model size and accuracy.

The problems of a 7-degree of freedom (DOF) manipulator with rapid and continuous response to uncertain fast-flying objects are addressed: 1) how to effectively solve trajectory planning of the 7-DOF manipulator with multiple criteria; and 2) how to make the 7-DOF manipulator realize the rapid and continuous response to uncertain fast-flying objects. In the proposed approach, based on the trajectory parameterization of the 7-DOF manipulator, a multiobjective teaching-learning-based optimization (MOTLBO) algorithm is adopted to find a close representation of the Pareto optimal set rather than a single solution. As such, an optimal solution can be chosen as digital knowledge information. A new methodology based on a knowledge base representing and learning the operation environment, that is, skill digitization, is presented, which enables the 7-DOF manipulator to realize the rapid and continuous response skill. Simulation and practical testing results of a ping-pong robot validate the feasibility and effectiveness of the proposed approach, in which the online trajectory generation spends only around 1 ms.

Monocular depth estimation has been actively studied in fields such as robot vision, autonomous driving, and 3D scene understanding. Given a sequence of color images, unsupervised learning methods based on the framework of Structure-From-Motion (SfM) simultaneously predict depth and camera relative pose. However, dynamically moving objects in the scene violate the static world assumption, resulting in inaccurate depths of dynamic objects. In this work, we propose a new method to address such dynamic object movements through monocular 3D object detection. Specifically, we first detect 3D objects in the images and build the per-pixel correspondence of the dynamic pixels with the detected object pose while leaving the static pixels corresponding to the rigid background to be modeled with camera motion. In this way, the depth of every pixel can be learned via a meaningful geometry model. Besides, objects are detected as cuboids with absolute scale, which is used to eliminate the scale ambiguity problem inherent in monocular vision. Experiments on the KITTI depth dataset show that our method achieves State-of-The-Art performance for depth estimation. Furthermore, joint training of depth, camera motion and object pose also improves monocular 3D object detection performance. To the best of our knowledge, this is the first work that allows a monocular 3D object detection network to be fine-tuned in a self-supervised manner.

The sampling-based partial motion planning algorithm has come into widespread application in dynamic mobile robot navigation due to its low calculation costs and excellent performance in avoiding obstacles. However, when confronted with complicated scenarios, the motion planning algorithms are easily caught in traps. In order to solve this problem, this paper proposes a knowledge-based fast motion planning algorithm based on Risk-RRT, which guides motion planning by constructing a topological feature tree and generating a heuristic path from the tree. Firstly, an online topological feature learning method is proposed to simultaneously extract the features during the motion of the robot by means of the dual-channel scale filter and the secondary distance fusion. The learning process is completed until the feature points can represent arbitrary obstacle-free grid points of the whole map. Secondly, the topological feature tree is constructed with environmental feature points and the heuristic motion planning can be carried out on the feature tree. For one map, once the construction of the feature tree finishes, it can be reused as a prior knowledge in the following heuristic motion planning process, which will further improve the efficiency of searching feasible paths. The experimental results demonstrate that our proposed method can remarkably reduce the time taken to find a heuristic path and enhance the success rate of navigation in trapped environments.

This paper presents a method based on frequency shift feedback for living cell manipulation and in situ nano-injection with a cantilevered micropipette probe (CMP). This method can detect tip-cell interactive forces at the piconewton level by measuring the frequency shift of the rigid CMP oscillated in the first bending eigenmode (amplitude: ~10 nm). Different interaction states throughout the process of manipulation and injection, including the contact, gripping, detaching, and cell (nuclear) membrane penetration, can be well detected and thereby controlled. In addition, the cell adhesion can be quantified by the integral of the frequency shift during the detachment process. Manipulation and nanoinjection are automatically performed using two types of CMPs with different apex aperture diameters (manipulation: Ø4 μm; nanoinjection: Ø200 nm), under the control of the dynamic force and microscope vision feedback. The proposed method can control the contact force of ~ 300 pn for nondestructive cell manipulation and can detect cell membrane (~250 pn) and nuclear membrane (~400 pn) penetration forces in nanoinjection. The versatility and robustness of the proposed method are further demonstrated by quantifying the cellsubstrate adhesion, the building of cell patterns, and automated cell nanoinjection.

This article presents a novel adaptive slicing method for additive manufacturing, it is based on extracted the number of candidate feature points for the different areas of the models to decide the thickness of each slicing layer. The slicing method is more systematic and robust than other approaches. It could achieve balanced optimization of multi-objectives in additive manufacturing, i.e., geometrical accuracy and build efficiency, for complex models and geometries. The approach is capable of deciding the thickness of each slicing layer for a complex model adaptively to balance these two conflicting objectives of geometrical accuracy with build efficiency according to users' needs.

The zona pellucida (ZP) micro-dissection technology has played a key role in the field of artificial assisted reproduction (such as ZP thinning, preimplantation genetic diagnosis (PGD), etc.). Currently, laser ZP micro-cutting technology using far-infrared beam has the disadvantages of high price, thermal damage, and low degree of freedom. Piezoelectric ultrasonic micro-cutting technology has matured at the biological tissue level, but ultrasonic cutting at the single-cell level is still difficult to achieve. In this paper, based on piezoelectric ultrasonic tissue micro-cutting theory, the mechanism of ultrasonic cutting of the oocyte ZP was researched, and a cutting method for the ZP was proposed. Based on the design formula of flexure's stiffness, this paper analyzes the effects of dimension parameters on the vibration condition of micro-needle. An ultrasonic cell surgery instrument based on a three-dimensional stereoscopic flexure-guided structure was designed. The modal analysis and the harmonic response analysis of the structure were performed using finite element software. The experimental results show that the lateral amplitude of the newly designed device's needle tip is smaller than the traditional one in the specified range of frequency. Among them, when the operating frequency is around 22.1 kHz, the lateral amplitude of the needle tip is reduced to 0.021μm. Finally, the theoretical method of piezoelectric ultrasonic cutting on single cell layer is presented for the first time, which has a broad prospect and significance for artificial assisted reproduction.

In the all-season indoor and outdoor background, facing the complicated environment formed by different lighting, partial blocked, pseudo-object interference, noise and other factors, the recognition and positioning of the charging port of an electric vehicle cannot be conventionally partitioned into a difficult problem. This paper studies the method for charging port recognition in a complex environment based on CNN, which not only ensures the accuracy and robustness of the recognition, but also provides a solution for accurately locating the charging port. The overall goal of the charging port recognition in this paper is to identify the category of the current image, and then identify the intensity of light for the image with the charging port. We built a sample set of charging port after the denoising of median filter, which is divided into four categories: complete, none, fake, and incomplete; In order to improve the generalization ability of the model, we add the number of LeNet-5 model and use the Relu activation function; Use the above two sample sets to train the models separately, save the models and parameters, and finally actually test. The experimental results show that the method uses the deep learning ability of convolutional neural network to automatically extract the features in the image, the recognition accuracy of the charging port is 99%, and the recognition accuracy of different light intensity is 100%. The integrity information and light intensity information are feedback to the automatic charging system, in order to accurately position the charging port subsequently, the camera position and the subtraction light strategy are adaptively adjusted to obtain a clearer image.

Nanomanipulation system based on scanning electron microscope(SEM) with good real-time visual feedback and nanoscale observation resolution had high operability in a vacuum working environment. Different nanomanipulation tasks of carbon nanotubes (CNTs) are carried out through the nanomanipulation system in SEM. Nanomanipulation platform existed inherent manufacture errors, installation errors and other errors, and imprecise nanomanipulation system were also time-consuming and laborious for operators. This paper presentes a method of combining the visual feedback and feedforward control to detect and compensate the motion error of the multi-dimensional SmarAct nanomanipulation platform in the nanomanipulation system in SEM. This method reduces the motion error in the X-Y direction and achieved higher operating accuracy. At the different step speed, the motion error in the X direction and Y direction is 135.7nm and 112.9nm respectively. After the feedforward compensation, the motion error in the X direction and Y direction reduces to 61.3nm and 54.1nm respectively.

Camera can sensor the environment on the lane by extracting the lane lines, but such detection is limited to a short distance with effect of illumination and other factors; radar can detect objects a long distance away but cannot detect the lane conditions. This paper combined machine vision with millimeter-wave radar and extracted the nearby distinct lane line through images; at the same time, the radar obtained the motion trajectory information of distant vehicles, then the least-square method was used to make curve fitting on those motion trajectory information in order to reconstruct the lane line information. Finally, in the stage of fusing two segments of lane lines, the goodness of fit was applied to complete the matching of corresponding lane lines. While, for areas between two segments of lane lines that neither camera or radar can detect, we established a lane model, utilized probabilistic neural network to select the corresponding lane model for matching, and then used approximate mathematics expression according to the selected lane model, thus obtaining the final front road information of current vehicle.

This paper presents a research on classifying walking speed and step length simultaneously by using cerebral hemoglobin information. Nine healthy subjects performed walking task spontaneously in three levels of speed and three levels of step length. Brain information of the subjects was measured by using functional near-infrared spectroscopy (fNIRS) technology. The differences between the oxygenated hemoglobin (oxyHb) and deoxygenated hemoglobin (deoxyHb) were decomposed by wavelet packet. Feature vectors were extracted in both the time domain and frequency domain. Walking speed and step length was identified by applying support vector machine (SVM) method. The preliminary identification accuracy was 62.97%. This finding puts forward a new method for identifying two-dimensional state of lower limbs in level walking. And it lays a foundation for realizing autonomous control of walking-assistive equipment.

In this paper, we propose a new method of salient region detection for indoor robots, which integrate the background distribution into the primary saliency. Region roundness is proposed to describe the compactness of a region to measure background distribution more robustly. In order to validate the proposed method, several influential ones are compared on the DSD dataset. The results demonstrate that the proposed approach outperforms existing methods and is useful for indoor robots.

It is of great importance to pick up a single carbon nanotube (CNT) from a bulk of CNTs for nanodevice fabrication. In this study, we have proposed a nanorobotic manipulation system allowing automated pick-up of CNTs based on visual feedback. We utilize histogram normalization for automatic binarization, and it achieves to clearly distinguish CNTs from substrate and other impurities under different image brightness. Furthermore, we develop the gradient orientation inversion (GOI) algorithm to recognize CNT tip and atomic force microscopy (AFM) cantilever. Taking full advantages of the geometrical characteristics of CNT and AFM cantilever, GOI is proved to be quite robust. We have designed segment detection method (SDM) to successfully separate the AFM cantilever and CNT, whereas the contact detection between them is achieved by analyzing the straightness variation. Preliminary experimental results imply that our method shows high promise in realistic fabrication of nanodevices.

The redundant humanoid manipulator has characteristics of multiple degrees of freedom and complex joint structure, and it is not easy to obtain its inverse kinematics solution. The inverse kinematics problem of a humanoid manipulator can be formulated as an equivalent minimization problem, and thus it can be solved using some numerical optimization methods. Biogeography-based optimization (BBO) is a new biogeography inspired optimization algorithm, and it can be adopted to solve the inverse kinematics problem of a humanoid manipulator. The standard BBO algorithm that uses traditional migration and mutation operators suffers from slow convergence and prematurity. A hybrid biogeography-based optimization (HBBO) algorithm, which is based on BBO and differential evolution (DE), is presented. In this hybrid algorithm, new habitats in the ecosystem are produced through a hybrid migration operator, that is, the BBO migration strategy and DE/best/1/bin differential strategy, to alleviate slow convergence at the later evolution stage of the algorithm. In addition, a Gaussian mutation operator is adopted to enhance the exploration ability and improve the diversity of the population. Based on these, an 8-DOF (degree of freedom) redundant humanoid manipulator is employed as an example. The end-effector error (position and orientation) and the ‘away limitation level’ value of the 8-DOF humanoid manipulator constitute the fitness function of HBBO. The proposed HBBO algorithm has been used to solve the inverse kinematics problem of the 8-DOF redundant humanoid manipulator. Numerical simulation results demonstrate the effectiveness of this method.

In this paper, a joint control algorism was proposed to implement on our musculoskeletal robot. The joints are all actuated by pneumatic muscles and have antagonistic structure. In order to gain a better performance, we first modeled the pneumatic muscle, and then considering the dynamics two control method was discussed and compared. As a tradeoff of rapidity and accuracy, the combined joint control algorism was implemented on the robot system. The algorism could tune the stiffness of the joint automatically to fit the compliance need for joint control. The experiments on our robot showed that the joint control algorism could operate the robot with a fast and accurate response. There is nearly not lagging behind, and with a small overshot during the step testing, and a complex leg's trajectory was also achieved through our joint controller.

The high speed running of legged robot raise higher requirements on the design and development of the robot and the control method. The mammals' excellent performance, especially the cheetah, provides us much inspiration on control, mechanical design and etc. In this paper, a model of the cheetah robot is developed for researching the locomotory behaviors of fast running. Under the biological knowledge on neural mechanism, a new bio-inspired control method was proposed via controlling the muscles flexion and extension movements. Then, a bio-musculoskeletal hindlimb prototype was designed in this paper. The muscle-like actuator and the cheetah-like structure combining the bio-control approach provide more natural fashion for the new robot. Finally, the simulation of the bounding gait establishes good biological properties, cheetah-like leg's trajectory. Experiment with the real hindlimb prototype validates the design and the development of the structure and the controller. In the near future, a final prototype will be manufactured with onboard power and air source.

In this paper, we introduce a quadruped robot designed for bounding gait with only one actuator per compliant leg. Under the analysis of the dynamics model of the robot, a new simple linear running controller using the energy control idea, which requires minimal task level feedback and only controls both the leg torque and ending impact angle, is proposed. It successfully executes fast running from rest till a constant speed and hi-speed turning, both in the prototype simulation and robot experiment. These results contribute to that complex dynamically dexterous tasks may be controlled via simple energy control method and delayed task feedback, which is closer to the animal's actual locomotion conditions. In the future, we plan to modify this method for reducing the energy expending and make the robot running fast.

In the die bonding process of MEMS microphone chip, the exact center and rotate angle of MEMS microphone chip is corrected after the vacuum nozzle has adsorbed it. In order to achieve high speed and precision placement function, the flying vision system based on rotating mirror is proposed, the mirror is driven to rotate by the power drawn from the nozzle rise and fall movement through a rack and pinion mechanism, the nozzle pick and place functions is not interfered by mirror through designing the rack and pinion mechanism parameters. The vision system can capture an static clear MEMS chip image when the mirror rotated to a 45 degrees position referenced with horizontal position, the center position and rotating angle of MEMS chip is fast calculated by the fine and coarse Hough Transform Algorithm, the center location and angle will be corrected by the control system during the mounted head is flying to placement position. The experimental results show that the system can obtain high-quality MEMS microphone chip image, the chip's center position and rotating angle correction algorithm takes no more than 0.1s, so it can meet the high-speed placement requirements. In this proposed flying vision system, mechanical structure is simple and compact, MEMS chip's image is clear without distortion, and there is no restriction on the mounted head movement trajectory. The flying vision system significantly enhances the quality and efficiency of SMD Chip; it can be widely used in the surface mounted system.

Due to scale effects, the releasing of micro objects has been a long-standing challenge in micromanipulation applications. In this paper a micromanipulation system is presented based on the adhesion control with compound vibration. This adhesion control technique employs inertia force to overcome adhesion force achieving 100% repeatability with releasing accuracy of 4±0.5μm, which was experimentally quantified through the manipulation of 20-100μm polystyrene spheres under an optical microscope. The micromanipulation system consists of a microgripper and a piezoelectric ceramics module. The compound vibration comes from the electrostatic actuator and the piezoelectrically driven actuator. Surface and bulk micromachining technology is employed to fabricate the microgripper used in the system from a single crystal silicon wafer. Experimental results confirmed that this adhesion control technique is independent of substrate. Theoretical analyses were conducted to understand the releasing mechanism. Based on this preliminary study, the micromanipulation system proves to be an effective solution for active releasing of micromanipulation.

The liquid dosing system has been studied widely as it had been used in many technical and medical applications. And in recent years, the miniaturization of the liquid dosing system as well as the accuracy of liquid transfer has gradually become a hot research field. In this paper, a MEMS (Micro-electromechanical system) flow sensor was integrated with a PZT(Piezoelectric Transducer) pump to make a closed-loop drug dosing system. Benefiting from the feedback of sensor information, the system can self-adjust the driving voltage of the PZT pump to precisely dispensing desired liquid volume. First, the structure and principle of the PZT pump will be introduced briefly. Secondly, the MEMS flow sensor is presented. Finally, the fuzzy PID (Proportional-Integral-Differential) closed-loop control strategy is proposed to calculate and adjust the driving voltage in real-time. Finally, experiment results show that the drug delivery system could precisely delivery drug volume with error smaller than 0.01µL when desired volume is 100µL.

This paper deals with the dynamics and motion control of a spherical robot designed for reconnaissance and unstructured hostile environment exploration. The robot in this paper has three DOFs and two inputs, of which the nature is a nonlinear and underactuated system with nonholonomic dynamic constraints. The improved construction of two pendulums offers novel motion principle of spherical robot, which is moving simultaneously actuated by both eccentric moment and inertial moment generated by the two pendulums. Meanwhile the mobility is enhanced when the robot behaves dynamically. The emphasis is placed on the linear motion and turning in place motion control. The dynamic model of linear motion is formulated on the basis of Lagrange equation, and a smooth trajectory planning method is proposed for linear motion. A feedback controller is constructed to ensure the accurate trajectory planning. Turning in place motion is an indispensable element of omni directional locomotion which can enhance the mobility of spherical robots. The dynamic model of turning in place motion is derived on the theory of moment of momentum, and a stick-slip principle is analyzed. The two motion control methods are validated by both simulations and prototype experiments.

This paper presents the design, fabrication, and application of an piezoelectrically driven microgripper integrated sidewall piezoresistive force sensor for measuring the gripping force. Surface and bulk micromachining technology is employed to fabricate end-effectors and sensor of the microgripper from a single crystal silicon wafer. Vertical sidewall surface piezoresistor etching technique is used to form the side direction sensors. The end-effectors of the gripper are four-bar structures. Two fixed cantilever beams integrated with piezoresistive sensor are designed to sense the gripping force, and a piezoelectrically driven microactuator is designed to provide the force to operate the other two movable bars. The piezoelectrically driver adhered with the silicon end-effectors generates a linear horizontal motion 9 mum, which is amplified to 30 mum at the bar tip of the microgripper. Then the range of the operation is 25 mum - 140 mum. Testing results verify that the vertical sidewall surface piezoresistor etching technique is effective. The sensitivity of the piezoresistive sensors is better than 72 V/N, and the resolution is better than 3 muN.

In biomedical areas, such as protein crystallization, drug discovery and medical diagnostics, thousands of reagents with different viscosities need to be transferred or dispensed in submicroliter range. Most of the commercial automated liquid dispensing systems need to be calibrated through experiment to accurately dispense liquids when the reagents viscosities change, which is time consuming and less accurate. In this paper, an intelligent precise liquid handling system was developed, in which a high speed MEMS flow sensor was used. Benefiting from the feedback of sensor information, the system can self-adjust the open time of the solenoid vale to accurately dispensing desired reagent volume without pre-calibration. Firstly, the system construction was introduced in detail. Secondly, a novel closed-loop control strategy was proposed to calculate valve open-time for each dispensing cycle, which makes the system immune to liquid viscosity, pressure fluctuation and some other disturbances. Finally, experiments results are presented with different dispensing volumes, coefficient of variance (CV) has been shown to be below 3% at 1ldquol and approach 4% at 100 nl.

The paper proposes a real-time tracking algorithm for a moving object with mobile robot based on vision using adaptive color matching and Kalman filter. The adaptive color matching can limit the region containing moving object on vision image plane. It can adjust color matching threshold to reduce the influence of lighting variations in the scene. Kalman filter is used as our prediction module to calculate motion vectors of moving object in the robot coordinate system. A view window containing the position of moving object estimated by Kalman filter is determined on image plane to reduce the image processing area. Color matching threshold can adjust itself adaptively in view window, which is used as an updating module. Experimental results show that the algorithm can adapt to lighting variations and has good tracking precision. It can also be implemented in real time.

Microscopic computer vision differs significantly from macroscale computer vision. Autofocusing techniques is of fundamental importance to automated micromanipulation in providing high level task understanding, task planning and real time control. Depth-from-Focus (DFF) based autofocusing method is widely used in many microscope systems, while the low efficiency limited its application in micromanipulation system which needs fast autofocus for real time control. In this paper, Depth-From-Defocus (DFD) algorithms are proposed to improve autofocusing performance and robustness for microscopic optics. Two different schemes based on look-up table and function relation are investigated and evaluated through actual experiments. Experimental results validate the performances of the two proposed autofocusing methods.

Recently, model-based and autocode technology has become mature and brings many advantages in automotive software development. In order to take advantage of these changes, organization must adjust development process. This paper proposes a "V+v" method, in which the process of code design and implementation are divided into two development processes (controller algorithms and software platform) in contrast to traditional "V" development process. Control engineers develop algorithm based on model, generate codes with autocoding tools and carry out tests with the same model via different kinds of simulations. Platform engineers design software development platform based on OSEK-compliant RTOS according to results of system functional analysis. During software integration & test phase, algorithm codes are "installed" into the platform and then they are linked and built together to create whole ECU software. The method has been used for diesel ECU software development in China FAW Group Corporation and the results demonstrate benefits.

Calibration is an essential issue to use a robot to accomplish some tasks with high accuracy requirements. An important part of calibration process is measuring the actual pose of some parts of the robot. Many sensors have been used for measuring, for some of them, extra objects need installing on the robot, so it's possible to implement extra accuracy problems. While choosing coordinate measure machine as the measure equipment, a new pose measure method is proposed. The major advantage of this method is neither extra objects nor definite points are needed, so it's more feasible to realize the coordinate measure machine's strength. The planes for calibration can be directly machined on the end-effector, so the assembly errors are avoided completely. An experiment to calibrate a parallel robot's geometrical parameters is carried out and the results are provided.

A redundant robot configured with a parallel base and a series manipulator is proposed in this paper. The position and orientation of end manipulator are considered as the result of the joints' ordinal motion according to the equivalence principle, so the close analytic solution for self-motion of this redundant robot is obtained via geometrical method. The Jacobian matrices of the parallel base and the whole robot system are solved respectively. Based on this, the solution for self-motion in two classes of singular configuration is also obtained. Finally a self-motion simulation of the redundant robot is performed via software MATLAB and ADAMS to validate the algorithm

Pixel-based auto-focusing is a long-standing topic in the literatures. It involves three main parameters: Region of Interest (ROI), image sharpness function and global maximum searching algorithm. As the mathematical description of image sharpness, sharpness function is the core issue for realizing robust auto-focusing. In this paper, the existing sharpness functions are summarized and grouped firstly, then a new space domain SUSAN (Smallest Univalue Segment Assimilating Nucleus) based sharpness function is proposed. The key problem in proposed algorithm is selecting a suitable similarity function to measure the similarities of the sampling pixel and its neighbors. In experiments, 8 similarity functions are analyzed and evaluated. Based on the evaluation results, a rough/fine two grades global maximum searching strategy is designed to realize fast and robust auto-focusing. At last, experiments verify the validity of the proposed auto-focusing algorithm.

Micro operation assisted by miniature mobile robot system has become a research hot in robotics and micro operation domain. In this paper, a newly developed micro operation oriented five D.O.F. (degrees of freedom) miniature mobile robot system is presented. By comparing it with other miniature robot systems, the main improvements of this design are macro/micro dual driven, embedded hardware, wireless power supply and wireless communication Hereinto, development of a mobile robot with dual motions is the first challenge and the main original contribution. Two miniaturized brushless DC motors drive the macro motion, which realizes faster motion of the robot. Micro motion, which realizes precision positioning of the robot, is driven by four piezoelectric-actuators and four electromagnetic foots based on inchworm principle. To guide the robot accomplishing micro operation automatically, an external visual sensor system with CCD cameras and optical zoom microscopes is designed. And the details of robot control architecture are explained. Experiments show that the proposed robot system succeeds in providing micro operation task such as micro part gripping, convey and assembly.

Full parallel manipulator systems with flexure hinges are widely applied in many fields demanding high precision, whose dynamic performances influence the system positioning accuracy, so it is very necessary to analyze the structural dynamics in detail. The dynamic analyses of this kind of systems are relatively complex for they belong to spatial multi-loops mechanisms. In this paper, a simplified modeling is proposed which can estimates the natural characters. This method can provide an effective generalized model to estimate lower natural frequencies of the parallel manipulators with flexure hinges needless to deduce the complicated dynamics model. Firstly, the system kinematics model is established, based on which the flexibility matrix is deduced. And then the dynamics equation can be formulated, via which the lower natural frequencies can be calculated. It is validated that the method proposes an effective and concise estimating model for generalized parallel manipulator systems' dynamics characters

Batch microassembly of MEMS sensors is limited by the manual manipulation required specially trained technicians. To reduce the production costs and simultaneously obtain high production quality, a flexible microassembly system for automated bonding of MEMS sensors is developed. The system consists of a set of autonomous modules that can adapt their structures and functions to various sizes of MEMS sensors, including positioning stages, a microscopy imaging system, a flexible micromanipulator, a heater, a fixture, a supply station and auxiliary systems. Optomechatronic design is essential to the development of integrated systems due to the basic importance of microscope optics to microassembly. In this paper, major methodology issues in optomechatronic design of this system are introduced. A wavelet-based microscopic focus measure and a control scheme with a modified Smith predicator to decrease the inherent time delay of vision system are presented. A smart force sensor with one dimension is employed to sense and control the interactive force. To perform manipulations automatically, a control system, including a task planning level and a real-time execution level, is developed. The productivity of the flexible microassembly system is validated by further experiments

Microscopy imaging can not achieve both high resolution and wide image space simultaneously. Image fusion and 3-D surface reconstruction is of fundamental importance to micromanipulation in providing high lever task understanding, task planning and real time control. Purpose of image fusion is to combine those images into one single image with an extended depth-of-field. Previously, real valued wavelet transforms based image fusion suffers from shift variance, poor directionality, and lack of phase information. One promising fusion method based on complex wavelet transforms to overcome these problems is proposed. Combined 2-D position data of "in focus" pixels with a height map obtained from the proposed image fusion method, a 3-D surface reconstruction algorithm of microparts is developed. Experimental results validate performances of the proposed image fusion and 3-D surface reconstruction methods.

In robot-assisted surgery projects researchers should be able to make fast 3D reconstruction. Usually 2D images acquired with common diagnostic equipments such as UT, CT and MRI are not enough and complete for an accurate 3D reconstruction. There are some interpolation methods for approximating non value voxels which consume large execution time. We introduce a novel algorithm based on generalized regression neural network (GRNN) which can interpolate unknown volxes fast and reliable. The GRNN interpolation is used to produce new 2D images between each two succeeding ultrasonic images. It is shown that the composition of GRNN with image distance transformation can produce higher quality 3D shapes. The results of this method are compared with other interpolation methods practically. It shows this method can decrease overall time consumption and conserve the quality on online 3D reconstruction

This paper proposes time-optimal trajectory generation method for industrial robots. The proposed time-optimal trajectory generation method studies the relationship between the maximum acceleration and the maximum velocity, and presents an algorithm for deriving the optimal trajectory based on this relationship. In order to verify this method, a new planar parallel robotic manipulator with fast motion and high precision is designed. Finally, the proposed method is applied to the designed robot, and the simulation results prove that the method is effective for the industrial robot

In order to exactly evaluate the robot stability and stability margin while walking, one general criterion is proposed, which adopts the minimal distance from supporting edge to center of pressure on ground to estimate robot walking stability. This general criterion take many condition terms in consider such as disturb force and moment, height of center of gravity, obliquity of supporting plane and mass of robot. For convenience in judge, the conception of minimal stability distance (MSD) is introduced and based on the general criterion, the shortest distance from supporting edge to the vertical projection point of robot center of gravity on ground is used to embody the general criterion mentioned above indirectly. By simulation, the relations between stability and parameters such as disturb elements, height of center of gravity, obliquity of supporting plane and mass of robot are discussed. At last, the experiment proved the effectiveness of the general criterion

Microscopy imaging can not achieve both high resolution and wide image space simultaneously. Autofocusing and 3-D surface reconstruction techniques are of fundamental importance to automated micromanipulation in providing high lever task understanding, task planning and real time control. In this paper, a new wavelet-based focus measure is developed, which provides significantly better depth resolution accuracy, and robustness than previous ones. A complex valued wavelet-based microscopic image fusion method and 3-D surface reconstruction scheme were proposed. Purpose of image fusion is to combine those multi-focus images into one single clear composite image with an extended depth-of-field. Combined 2-D position data of "in focus" pixels with a height map obtained from the proposed image fusion method, a 3-D surface reconstruction algorithm of microparts is developed. Experimental results validate the performances of the proposed image fusion and 3-D surface reconstruction methods

Although micro-motion parallel manipulators have been used widely, their special architectures make it impossible to be applied in the condition demanding relative large workspace. In this paper, a novel large workspace flexure hinge-based parallel manipulator is presented, which can attain the sub-micron scale accuracy over the cubic centimeter workspace. This manipulator system is a 6-PSS parallel mechanism, in which piezoceramic motors actuate the prismatic joints, precision linear encoders detect the actuation displacements, and as the key technology, a kind of novel wide-range flexure hinges are utilized as passive spherical joints. Because of the adoption of wide-range flexure hinges, the whole system features large workspace differing from the conventional micro-motion flexure hinge-based parallel manipulators. The kinematics analysis of the whole mechanism is performed based on the stiffness model of the flexure hinges via FEM theory. The wide-range flexure hinges move over large space during the course of self-deformation, so the kinematics model is a typical geometrical nonlinear problem. In this paper, a Newton-Raphson increment iterative scheme for the kinematics model solution procedure of the whole mechanism is presented based on Updated Lagrange Formulation. And finally, the numerical calculation results about the theoretical model are given.

This paper presents a Task-reconfigurable system developed for efficient and reliable MEMS assembly. The system consists of a set of autonomous modules that can adapt their shape and function to various assembly tasks, including a distributed 6 DOF coarse positioning module, a distributed 5 DOF fine positioning module, grippers designed for different manipulation tasks with same mechanical interface, a multi-view imaging system, and control software. In order to achieve high precision and dexterity in microassembly, a hybrid vision-force control method is proposed. For easier reconfiguration, a semi-automated calibration method is applied to quick calibrating of the new reconfigurations. Analysis and 3 cases of microassembly are presented to demonstrate that this system has high reconfigurability and adaptability.

This paper presents the automatic microassembly of miniaturized gear system by hybrid vision-force control with the vision system and the 3-DOF force sensor feedback data. The assembly process consists of three phases, including visual positioning, searching meshing state, and inserting. Visual feedback is used to achieve the coarse positioning and guide the grasping and transporting of microparts during the visual positioning phase. During the searching phase, a fuzzy PID controller is used to control the contact force on z axis and fuzzy logic strategy is developed to search the meshing state by the force feedback on x and y axes. The tolerance compensations movement is used to support the inserting task to avoid the case of blocking and prevent the micro parts from damaging. The assembly experiment of three planetary gears validates the hybrid force control strategy.

This paper presents a novel 3-DOF hybrid kinematics robot composed of planar parallel mechanism including parallelogram linkages and linear drive, which are used for wire bonding of microelectronic device. Combining the advantages of parallel mechanism and direct drive, the robot can achieve high precision and high acceleration. The structure of the robot is optimized from the view of both kinematics and dynamics respectively. The experimental results demonstrated that the repeatability, acceleration and settling time of the robot can satisfy the proposed application.

Parallel-structure flexure mechanisms are increasingly designed due to their superior characteristics. This paper explores a novel six degree-of-freedom large workspace flexure parallel mechanism based on the concept of wide-range flexure hinge, which can attain sub-micron scale accuracy over cubic centimeter motion range. The geometric dimensions of the flexure hinges utilized in this mechanism as passive joints will influence the system stiffness directly and other properties indirectly such as the workspace, load-carrying capacity, and driving-load capacity etc. In this paper, the stiffness model of individual flexure hinge is established firstly, and then the stiffness of the whole flexure mechanism is modeled via assembling stiffness matrices and formulating constraint equations. Based on the system stiffness model of the whole mechanism, the stiffness atlases' analysis is presented which provides theoretical principles for designing and developing this kind of flexure parallel mechanism in further. Finally, a 6-PSS large workspace flexure parallel mechanism prototype is proposed according to the analysis results, which will be utilized in the precision positioning.

This paper reports the development of a robotic system to assist surgeons to perform bonesetting operation, such as femur or tibia fracture, in which interlocking intramedullary nail are widespread to combine bones. As for ordinary procedures, sometimes the nails are distorted, so it is difficult to lock the screw in the bone. And in such a situation, surgeons have to operate under X-ray directly. Consequently the harm to the surgeon is very serious, and the operation effect is disillusionary. A novel robot-assisted bonesetting system is being developed to solve these problems. The system can assist surgeon to reposition fracted long bone and lock the interlocking intramedullary nail. It can be executed in three modes: telesurgery, semi-autonomous surgery and autonomous surgery. The destination is autonomous surgery., i.e. once configured by surgeons, the system can fulfill the part of bone repositioning and interlocking intramedullary nail locking. In order to accomplish interlocking intramedullary nail procedure, the system consists of automatic operating table, exact fluoroscopy unit, fracture reposition robot, guiding robot, computer control system and surgeon console, as well as some function units, which are image acquisition & processing and calibration system, tele-operation system, virtual operation simulation system and human-machine interface. All mentioned above are presented in detail in this paper.