Intraocular surgery, one of the most challenging discipline of microsurgery, requires sensory and motor skills at the limits of human physiological capabilities combined with tremendously difficult requirements for accuracy and steadiness. Nowadays, robotics combined with advanced imaging has opened conspicuous and significant directions in advancing the field of intraocular microsurgery. Having patient treatment with greater safety and efficiency as the final goal, similar to other medical applications, robotics has a real potential to fundamentally change microsurgery by combining human strengths with computer and sensor-based technology in an information-driven environment. Still in its early stages, robotic assistance for intraocular microsurgery has been accepted with precaution in the operating room and successfully tested in a limited number of clinical trials. However, owing to its demonstrated capabilities including hand tremor reduction, haptic feedback, steadiness, enhanced dexterity, micrometer-scale accuracy, and others, microsurgery robotics has evolved as a very promising trend in advancing retinal surgery. This paper will analyze the advances in retinal robotic microsurgery, its current drawbacks and limitations, as well as the possible new directions to expand retinal microsurgery to techniques currently beyond human boundaries or infeasible without robotics.

Surgical robots are usually controlled using a priori models based on the robots’ geometric parameters, which are calibrated before the surgical procedure. One of the challenges in using robots in real surgical settings is that those parameters can change over time, consequently deteriorating control accuracy. In this context, our group has been investigating online calibration strategies without added sensors. In one step toward that goal, we have developed an algorithm to estimate the pose of the instruments’ shafts in endoscopic images. In this study, we build upon that earlier work and propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data. Our experiments demonstrated an improvement of 21 % for translation error and 26 % for orientation error on synthetic test data with respect to our previous work. Results with real test data provide a baseline for further research.

Surgical robots are used to perform minimally invasive surgery and alleviate much of the burden imposed on surgeons. Our group has developed a surgical robot to aid in the removal of tumors at the base of the skull via access through the nostrils. To avoid injuring the patients, a collision-avoidance algorithm that depends on having an accurate model for the poses of the instruments' shafts is used. Given that the model's parameters can change over time owing to interactions between instruments and other disturbances, the online estimation of the poses of the instrument's shaft is essential. In this work, we propose a new method to estimate the pose of the surgical instruments' shafts using a monocular endoscope. Our method is based on the use of an automatically annotated training dataset and an improved pose-estimation deep-learning architecture. In preliminary experiments, we show that our method can surpass state of the art vision-based marker-less pose estimation techniques (providing an error decrease of 55% in position estimation, 64% in pitch, and 69% in yaw) by using artificial images.

Minimally Invasive Surgery (MIS) is one of the most successful applications of surgical robots. Although the introduction of robotic technology has brought a number of benefits, further advancements in MIS are limited by the size and bending radius of instruments. In this paper, we present a compliant four degree-of-freedom manipulator that consists of elastic elements with partly thinner structures. The proposed mechanism allows the elastic element to deform locally, thus minimizing its bending radius while the low number of mechanical parts greatly contributes to its compactness. This paper describes the design strategy, optimization method using FEA, prototype implementation, and evaluations. The evaluations reveal high accuracy and repeat accuracy, which are key elements for robotic instruments in MIS. Further, the prototype is able to exert sufficient force and it is possible to perform a simulated needle insertion task using the manipulator, demonstrating the feasibility of the proposed mechanism.

In adult laparoscopy, robot-aided surgery is a reality in thousands of operating rooms worldwide, owing to the increased dexterity provided by the robotic tools. Many robots and robot control techniques have been developed to aid in more challenging scenarios, such as pediatric surgery and microsurgery. However, the prevalence of case-specific solutions, particularly those focused on non-redundant robots, reduces the reproducibility of the initial results in more challenging scenarios. In this paper, we propose a general framework for the control of surgical robotics in constrained workspaces under teleoperation, regardless of the robot geometry. Our technique is divided into a slave-side constrained optimization algorithm, which provides virtual fixtures, and with Cartesian impedance on the master side to provide force feedback. Experiments with two robotic systems, one redundant and one non-redundant, show that smooth teleoperation can be achieved in adult laparoscopy and infant surgery.

Robotic assistance allows surgeons to perform dexterous and tremor-free procedures, but robotic aid is still under-represented in procedures with constrained workspaces, such as deep brain neurosurgery and endonasal surgery. In these procedures, surgeons have restricted vision to areas near the surgical tooltips, which increases the risk of unexpected collisions between the shafts of the instruments and their surroundings. In this paper, our vector-field-inequalities method is extended to provide dynamic active-constraints to any number of robots and moving objects sharing the same workspace. The method is evaluated with experiments and simulations in which robot tools have to avoid collisions autonomously and in real-time, in a constrained endonasal surgical environment. Simulations show that with our method the combined trajectory error of two robotic systems is optimal. Experiments using a real robotic system show that the method can autonomously prevent collisions between the moving robots themselves and between the robots and the environment. Moreover, the framework is also successfully verified under teleoperation with tool-tissue interactions.

Robotic assistance allows surgeons to perform dexterous and tremor-free procedures, but is still underrepresented in deep brain neurosurgery and endonasal surgery where the workspace is constrained. In these conditions, the vision of surgeons is restricted to areas near the surgical tool tips, which increases the risk of unexpected collisions between the shafts of the instruments and their surroundings, in particular in areas outside the surgical field-of-view. Active constraints can be used to prevent the tools from entering restricted zones and thus avoid collisions. In this paper, a vector field inequality is proposed that guarantees that tools do not enter restricted zones. Moreover, in contrast with early techniques, the proposed method limits the tool approach velocity in the direction of the forbidden zone boundary, guaranteeing a smooth behavior and that tangential velocities will not be disturbed. The proposed method is evaluated in simulations featuring two eight degrees-of-freedom manipulators that were custom-designed for deep neurosurgery. The results show that both manipulator-manipulator and manipulator-boundary collisions can be avoided using the vector field inequalities.

Automation of surgical tasks is expected to improve the quality of surgery. In this paper, we address two issues that must be resolved for automation of robotic surgery: online trajectory planning and force control under dynamic conditions. By leveraging demonstrations under various conditions, we model the conditional distribution of the trajectories given the task condition. This scheme enables generalization of the trajectories of spatial motion and contact force to new conditions in real time. In addition, we propose a force tracking controller that robustly and stably tracks the planned profile of the contact force by learning the spatial motion and contact force simultaneously. The proposed scheme was tested with bimanual tasks emulating surgical tasks that require online trajectory planning and force tracking control, such as tying knots and cutting soft tissues. Experimental results show that the proposed scheme enables planning of the task trajectory under dynamic conditions in real time. In addition, the performance of the force control schemes was verified in the experiments.

The authors have investigated a construction methodology for a non-invasive ultrasound theragnostic system, called NIUTS, to track and follow moving body targets, such as tumors / stones located in moving organs (kidneys, livers, etc.), robustly and precisely, so as to compensate for nearly 90 percent of target motions in ultrasound images, while irradiating therapeutic ultrasound to focal lesions. In the present report, we propose a novel servo recovery algorithm for NIUTS. We confirm the effectiveness of the implemented servo recovery algorithm for a real human kidney, which, as far as we know, is the first successful report of semi-autonomous servo recovery systems for lost body targets. This robust servo recovery method could provide an indispensable tool for precise, safe, and efficient treatments of tumors and stones.

The authors have developed a bed-type non-invasive ultrasound theragnostic system (NIUTS) that compensates for movement by tracking and following the area to be treated by stereo ultrasound imaging while irradiating high-intensity focused ultrasound (HIFU) onto the affected area. In this paper, we propose a novel robust template matching method to track and follow body targets, which include tumors and stones for the NIUTS. The proposed novel robust template matching method could be applied to a motion tracking of the real human kidney based on the ultrasound images, which is the first successful report as far as I know. This robust visual servoing method could be a great tool to treat the tumors and stones precisely and safely.

In orthopedic surgery, bone-cutting procedures are frequently performed. However, bone-cutting procedures are very risky in cases where vital organs or nerves exist beneath the target bones. In such cases, surgeons are required to determine the depth of the penetration into the bone by using only their haptic senses. Thus, we developed a handheld bone-cutting-tool system that detects the penetration of the cutting material. The developed system autonomously detects the penetration before total penetration and stops the actuation of the cutting tool, leaving a very thin remnant of work material. The developed system estimates the cutting resistance by using its motor's current and rotational speed. On the basis of data collected preoperatively, the system estimates the cutting state by using a support vector machine (SVM). According to the SVM outputs, the system detects the penetration of the work material and autonomously stops the actuation of the cutting tool. The proposed method was verified through experiments, and the results showed that the developed system successfully detected the penetrations of work materials and stopped autonomously immediately before total penetration. This study showed that the autonomous detection of bone penetration with a hand-held bone-cutting tool is feasible by using the proposed scheme.

The automation of surgical tasks has great potential for improving the performance of robotic surgery. The learning-from-demonstration approach has thus far been employed by many researchers when planning the trajectories of robotic instruments for automated surgical tasks. However, previous methods are applicable only when the demonstrations and trajectory generation are performed under the same initial conditions. In this paper, we propose an algorithm that learns through demonstrations and generates a trajectory regardless of the initial conditions. The variance of the demonstrated trajectories over the initial conditions is modeled and learned using a statistical method, and the learned trajectories are generalized and used to generate a trajectory. As an example, the bi-manual looping task of a surgical thread was demonstrated by changing the initial positions of the robot arms, and a trajectory was then planned given these initial arbitrary positions. The proposed algorithm was verified through simulations and experiments using an actual robotic surgical system.

The authors have developed a non-invasive ultrasound theragnostic system (NIUTS) with a focal lesion servo (FLS) function based on stereo ultrasound imaging in order to compensate for movement while irradiating high-intensity focused ultrasound (HIFU) onto a focal lesion. In the present paper, we propose a redundant motion control mechanism of the HIFU focus, for therapeutics, that is independent of the ultrasound probes for diagnostics. Using the proposed redundant motion control mechanism, an arbitrary designated position in the focal lesion can be treated without altering the viewpoint of the ultrasound probes. The proposed mechanism enables (i) noise factors, which deteriorate the image quality (IQ) for FLS, to be reduced, thereby enhancing the FLS performance, and (ii) enables the destruction of a focal lesion (kidney stone) with a preoperatively designated locus that is moving due to respiration/heartbeat.

Remote surgery has attracted great interest in recent decades because it has the potential to improve equality of access to health care. However, remote surgery has not been approved in any country, since patient safety cannot be guaranteed. In this paper, we present a hybrid control system for master-slave velocity control and admittance control for safe remote robotic surgery. The proposed controller switches between these two controls adaptively and stably and avoids excessive contact force between the robotic surgical instrument and the patient's organs. Through the proposed scheme, the contact force between the robotic instrument and objects at the slave site can be bounded by a set upper limit regardless of the motion the operator inputs at the master site. The performance of the proposed system was verified analytically. The proposed control scheme was implemented in a robotic surgical system and its performance was verified through simulations and experiments. The results of the simulations and experiments showed that the system autonomously switches stably between master-slave velocity control and admittance control and avoids excessive contact force.

Advanced robotic assistance in microsurgery, such as automation, requires an accurate estimation of the state of the robotic forceps. In this paper, we propose a robust and accurate forceps tracking method to estimate the full state of the forceps (i.e., the position, posture, and grip parameters) using visual information obtained from stereo microscopic images and kinematic information obtained from the robotic sensory information, forward kinematics, and hand-eye coordination. An online method for updating the hand-eye coordination was also developed using an extended Kalman filter to cancel the hand-eye coordination errors caused by the repositioning of the microscope. The experimental results showed that the proposed method could accurately and robustly estimate the state of the robotic forceps even after the repositioning of the microscope.

Loss of haptic sensation in a master-slave system is one of the open problems in robotic surgery, and recognition of surgical situations through haptic sensation is a challenge. In this paper we propose an autonomous risk-detection system for a master-slave surgical robotic system in order to estimate a property of an object (i.e., contact impedance) using a force sensor mounted on a surgical robotic instrument. The system autonomously detects the risk based on the estimated contact impedance and accordingly activates the motion at the slave unit as well as the force feedback at the master unit. We implemented the proposed method in a teleoperated master-slave system to detect the perforation risk of a membranous object. The performance of the system was evaluated through experiments. The classification accuracy for perforation risk was about 98.5 % in fourfold cross-validation. The experiments verified that the risk detection system accurately detected the perforation risk and improved the safety of the master-slave system.

Forceps tracking is an important element of high-level surgical assistance such as visual servoing and surgical motion analysis. In many computer vision algorithms, artificial markers are used to enable robust tracking; however, markerless tracking methods are more appropriate in surgical applications due to their sterilizability. This paper describes a robust, efficient tracking algorithm capable of estimating the full state parameters of a robotic surgical instrument on the basis of projective contour modeling using a 3-D CAD model of the forceps. Thus, the proposed method does not require any artificial markers. The likelihood of the contour model was measured using edge distance transformation to evaluate the similarity of the projected CAD model to the microscopic image, followed by particle filtering to estimate the full state of the forceps. Experimental results in simulated surgical environments indicate that the proposed method is robust and time-efficient, and fulfills real-time processing requirements.

In this paper, we present the development of an intuitive user interface for hand-held multi-DOF forceps for laparoscopic surgery. Interface design is crucial to the development of hand-held medical instruments. Surgeons may not use medical devices owing to their poor usability, regardless of their performance. In addition, most surgeons do not have the time to learn the use of robotic instruments. We have listed all possible input mechanisms based on the motions of the fingers, and we have designed sample interfaces consisting of effective input mechanisms. The usability of the designed interfaces was evaluated by surgeons via experiments performed using the sample interfaces and a simulator. In summary, we propose a method for designing the most intuitive user interface for hand-held multi-DOF forceps.

We propose a non-invasive ultrasound theragnos tic system that tracks movement in an affected area (kidney stones, in the present study) by irradiating the area with high intensity focused ultrasound (HIFU). In the present paper, the concept behind a novel medical support system that integrates therapy and diagnostics (theragnostics) is illustrated. The re quired functions for the proposed system are discussed and an overview of the constructed system configuration is illustrated. The problems associated with kidney stone motion tracking by ultrasonography are described. In order to overcome these problems, we consider two approaches. The first approach is to minimize the servoing error so as to enhance both the efficiency of the therapy and the safety of the patient. The second approach is to reduce the effect of the servoing error. With respect to the first approach, we propose a robust detection method of the stone position based on shape information. With respect to the second approach, we propose a solution for controlling the HIFU irradiation power in accordance with the servoing error, primarily in order to enhance the safety of the patient.

In this paper, we discuss the issue of safety in robot-assisted fracture reduction. We define the hazards of robot-assisted fracture reduction and design safety control methods. Although a large reduction force is required to reduce femoral neck fractures, an unexpectedly large force produced by a robot may cause injury to the patient. We have designed two mechanical failsafe units and a software force limiter; this along with velocity control can guarantee a safe operation in reduction force. In addition, to reduce the movement of bone fragments as much as possible, we devised spatially constrained control methods for fracture-reduction robots. The fracture-reduction system was evaluated using simulated fracture reductions.

Force feedback systems have been developed to improve the user-friendliness of robotic surgery systems. In teleoperative surgery, one of the major problems is the delay of the transmission of the information from the operation site to the surgery site. The transmission delays of the control signal and force information are less than the that of the image signal. In many cases, the visual information is synchronized with the force information in order to avoid a time gap between them. In this paper, we propose that force information be presented earlier than the visual information. This approach reduces the damage to the organs in surgical operations even though there is a time gap between the delay of force information and visual information. In addition, we propose a method to present predicted force information to avoid damage to the organs due to time delays in teleoperative surgery. The liver can be treated with a contact force of less than 0.8 N and a grasping force of less than 1.1 N by applying the proposed method. The method can realize a teleoperative surgery without damaging the patient's organs.

The non-invasive ultrasound theragnostic system, we propose, tracks and follows movement in an affected area -kidney stones here-, while High-Intensity Focused Ultrasound (HIFU) is irradiated onto the area. In this paper, the concept of the novel medical support system, which integrates the therapy and diagnostics, is illustrated at first. Secondly, structuring the required functions for the proposed system is discussed. Third, the overview of the constructed system configuration is illustrated. Fourth, the problem of the stone motion tracking by ultrasonography is clarified. To cope with this problem, the respiratory motion of a human kidney is analyzed and a controller, by utilizing the quasi-periodical motion of the respiratory kidney motion, is proposed. Finally, the result of the servoing and HIFU irradiation experiments of the model stone, which moves based on the real human kidney motion data, is reported to confirm the effectiveness of the proposed controller and the constructed system.

In the present paper, we describe a method for constructing a remote ultrasound diagnostic system. Remote diagnosis can be realized using a communication network. We have developed a master-slave type remote medical system to diagnose shoulder diseases, such as dialysis-related amyloid arthropathy (DRAA), by ultrasonographic images. Proper positioning, orientation, and contact force between the ultrasound probe and the affected area of the patient are required in order to acquire proper diagnostic images. Safety and manipulability are also required when operating the remote medical system through a communication network. Therefore, the system has impedance control capability for positioning of the master and slave manipulators in order to convey the contact force and enhance manipulability. In addition, the system has continuous-path control capability for the orientation of the slave manipulator in order to realize smooth and accurate motion of the ultrasound probe, even if the sampling rate of the transmission of the orientation data of the master manipulator is not sufficient. The results of remote diagnostic experiments demonstrated that a healthcare professional could diagnose real patients through a communication network using the constructed system.

The increase of precision and minimal invasiveness is an important current issue in orthopedic surgery. The femur and the tibia must be shaped to fit an artificial joint for successful knee arthroplasty. The recent trend towards MIS (Minimally Invasive Surgery) to decrease the length of the required incision has increased surgical difficulty, since the open access area is small. In this paper, registration and cutting error were analyzed with a robotic surgery system being developed first as an example, and a method of active compensation of robot deformation by gravity and cutting force was proposed and tried based on the error map with the expectation that the precision will increase.

The master and slave surgical assistant systems have been studied actively. However, regarding the master system, the manipulator should be designed for each surgical field because the target workspace and precision are different. Furthermore, operability is important for safety reasons. Therefore, the authors analyzed the motion of surgeon first and then developed a master manipulator suitable for the operation of micro neuro surgery. Some experiments were conducted to evaluate the control method and show the effectiveness of the proposed manipulator control method.

Remote surgery is one of the most desired applications in the context of recent advanced medical technologies. For a future expansion of remote surgery, it is important to use conventional network infrastructures such as Internet. However, using such conventional network infrastructures, we are confronting time-delay problems of data transmission. In this paper, a remote surgery experiment between Japan and Thailand using a research and development Internet is presented. In the experiment, the image and audio information was transmitted by a newly developed low latency CODEC system to shorten the time-delay. By introducing the low latency CODEC system, the time-delay was shortened compared with the past remote surgery experiments despite the longer distance. We also conducted several network measurements such as a comparison between TCP/IP and UDP/IP about the control signal transmission.

Adaptive control of the machining process in orthopedic surgery will increase not only productivity but also the safety issues of tool usage. The authors have developed a technology for an adaptive control system. The system has two modes, the "predictive mode" and the "adaptive mode." The predictive mode is used to shorten the air cutting time. In the adaptive mode, the system monitors the cutting force and the cutting temperature, and controls the feed override according to the difference between the real and the desired cutting force. The software is implemented on the robot controller and its effectiveness is evaluated with a urethane bone.

Toolpath generation and optimization is considered as a challenging problem in the minimally invasive orthopedic surgery with a milling robot. The objective of this paper is to minimize the collision of the cutting tool with the soft tissues. A novel approach of toolpath generation and optimization is proposed. A redundant axis is implemented to avoid the collision in the robot. Some important components are modeled based on the physical requirements. A geometric optimization approach based on the model is proposed to improve the toolpath. Case studies show the validity of this approach. Software is developed for this application and the effectiveness is evaluated with a cadaveric bone.

Recently, lithotripsy (kidney stone treatment) using HIFU (high intensity focused ultrasound) was developed by researchers in therapeutic ultrasound field. The lithotripsy crushes kidney stones powder, therefore, do not harm to the surrounding tissues of the kidney stones. However, it is necessary to continuously emit high intensity ultrasound waves on a target kidney stone during treatment. Therefore, HIFU transducers should follow a target kidney stone which moves due to respiration and heartbeat. In this paper, ultrasound-based visual servoing system is described. Two ultrasound probes and a HIFU transducer are mounted on the end effector of a xyz stage machine. The two ultrasound probes visually keep track of the target kidney stone within the body, and servo the xyz stage machine. The required techniques, such as conversion of a frame of radio frequency (RF) echo signals into an ultrasound bright mode image, visual tracking, pose estimation and control, are explained.

Several robotic surgical systems have been developed for MIS (minimally invasive surgery) including commercialized products such as da Vinci and ZEUS. We have developed a minimally invasive surgical system, which have carried out remote surgery experiments for five times at the writing time. In this paper, a remote surgery experiment, which was conducted between Japan and Korea by using the developed minimally invasive surgical system is described. Research & Development (R & D) Internet testbet, APII (Asia-Pacific Information Infrastructure), which consists of an optical submarine cable network KJCN (Korea-Japan Cable Network), was used. In the experiment, a laparoscopic cholecystectomy was successfully carried out on a pig. The network time-delays of control signal and images were 6.5 msec and 435.5 msec respectively. A comparison of remote surgery experiments using ISDN and the Internet was studied

Fracture reduction of carpal bones is rather difficult, because of their tight formation. Therefore, it is difficult to recognize their posture and position as well as their segmentation. This paper deals with a novel robotic system for a K-wire guiding using CT and fluoroscopic images for scaphoid fracture reduction. The robot is designed to consider issues such as sterilization, X-ray halation and easy setup. Template matching in registration is investigated focusing on shape and structure of the carpal bones as well as scaphoid itself. Experiments show validity of the developed system in terms of insertion accuracy and operation time

We have been studying the technologizing of skill of the medical doctor in the medical diagnosis such as dialysis related amyloid arthropathy (DRAA) by ultrasonographic images. Proper position, orientation, and contact force between the ultrasound probe and the affected part of the patient are required to acquire the proper diagnostic image. In this paper, we propose a navigation system to support the medical doctor to realize proper position of the ultrasound probe by utilizing the constructed shoulder model based on the knowledge of the medical doctor and the analysis of the diagnosis

In this paper, a method of force feedback augmentation to improve the force perception is proposed. Reaction force is amplified naturally according to the forceps action. In particular, the principal force components corresponding to the ideal reaction force directions of the current forceps action will be amplified. The other components are displayed without amplification. As a result, only the simple but significant reaction force patterns are emphasized. Complex behavior of the reaction force, which may distract or deteriorate the intended action, is attenuated. In addition, the PI gain scheduling local force feedback controller compensates for the inertial and gravitational force of the master manipulator. Thus the true reaction force at the forceps can be felt more realistic

Several robotic surgical systems have been developed for minimally-invasive surgery including commercialized products such as da Vinci and ZEUS. We have developed a minimally-invasive surgical system, which had performed remote surgery experiments for four times until now (performed on pigs over 150 km in Japan). In this paper, two approaches that extend the system from the perspective of extensions of surgeon's dexterity are described. The first approach is a development of complex robotic forceps, which have multiple d.o.f. on their tip. A complex rigid link mechanism has been applied to the robotic forceps to realize 3 d.o.f. motions in a patient's abdominal cavity. With the rigid link mechanism, surgeons have no longer need to be concerned a cable cutting problem, which often can be happened on conventional cable driven systems. The second approach is a force feedback capability. The developed rigid link mechanism is preferable for an implementation of a force feedback capability. The force feedback capability is important not only for extending surgeon's dexterity but also for preventing unexpected damage on internal organs. An evaluation experiment of the force feedback capability on a simulated surgical procedure was conducted. An augmented force presentation method was developed that displays contact information between forceps and soft tissues. In an evaluation experiment, the load applied to soft organ tissues was reduced by approximately 30% compared with a conventional force feedback method.

This paper describes a novel robot system for fracture reduction of a femur with a reduction path navigation system based on 3D CT image modeling. The robot has 6-degrees-of-freedom to realize fracture reduction motion with a mechanical failsafe mechanism as well as a software failsafe. Furthermore, one 6-axis force sensor is installed to realize power assist capability for a doctor as well as to measure the force applying on the patient's femur. Three of the operation modes necessary for fracture reduction operation are introduced, i.e., a JOG operation mode, a power assist mode and an automatic operation mode. The control method of the power assist mode by means of one force sensor is proposed and explained in detail. Furthermore, the control method of the automatic operation mode is described to correspond with a real-time reduction path modification by the navigation system. Experiments are conducted to verify the failsafe function and accuracy of the motion control.

In total knee joint replacement, the femur and the tibia must be precisely shaped to ensure that they firmly mate with the artificial joint. The normal procedure is to manually cut the bones using a bone saw and mechanical jigs. A 9-axis bone cutting machine tool has been developed by the authors to increase cutting accuracy and reduce the hospital stay required for recuperation. As developed, the cut plane is determined by a 5 degree-of-freedom mechanism and the femur and tibia are cut by moving two translational and one rotational degrees of freedom, a system that provides a cutter path which avoids cutting the ligament. A preoperative planning system based on CAD/CAM principles and an overview of a system for total knee arthroplasty is also discussed. Experimental results showed the effectiveness of the developed robot and the system.

A master-slave type remote ultrasound diagnostic system was developed. This paper presents and evaluates its control strategy. There is only one impedance controller that is located in the master site or the slave site or the relay point in the conventional impedance controller. Therefore, the effect of the time lag in transmitting force and position data becomes large in the conventional system. Then we propose an impedance controller to cope with this problem. In the proposed system, the position controller for the slave manipulator is also constructed by means of a velocity-control-based impedance controller. This feature makes it possible to shorten the dead time taken by the slave manipulator to respond for the contact force input. This paper also reports the result of a clinical use of the remote ultrasound diagnostic system. The result of the experiments demonstrates that a medical doctor can diagnose patients as well as the conventional diagnosis.

This paper describes a tele-endoscopic surgical system with force feedback capability. The system consists of multi-media cockpit, surgical site and communication link. The multimedia cockpit is equipped with force feedback type master manipulators, visual and auditory information presentation apparatuses, and foot switches. A slave manipulator with 3 arms is located at the surgical site; the left and right arms hold forceps or a radio knife and the center arm holds an endoscope. The slave manipulator has a special safety mechanism. Vital signs can be also transmitted to the cockpit. In the experiment, a cholecystectomy was successfully performed on a pig at the distance of 150 km.

A master-slave type remote ultrasound diagnostic system was developed. The controller has impedance control capability for the master and slave manipulators' positions and has a continuous path (CP) control capability for the slave manipulator's orientation. The paper presents the controller of the slave manipulator's orientation. The remote ultrasound diagnostic system is one of remote medical systems through a communication network. Therefore it is difficult to transfer information for control at high sampling rate. To cope with this problem, we introduce a CP controller to this remote medical system. CP control realizes a smooth and less vibrated motion of the slave manipulator even if it is available to use a lower sampling rate of transmitting orientation data of the master manipulator. As a result, it allows the slave manipulator to improve safety and to decrease the volume of transmitting data. The experimental results demonstrate the accuracy of the path control of the slave manipulator during CP control in the master-slave manipulation system. The experimental results also show that a medical doctor can perform diagnosis properly using the developed system.

A master-slave type remote ultrasound diagnostic system was developed. This paper presents its controller. The controller has impedance control capability for the master and slave manipulators' positions. And it has also continuous path (CP) control capability for the slave manipulator's orientation. The impedance control allows the master and slave manipulators to move according to the motion control law, and allows the medical doctor to feel feedback force. Manipulability changes according to the impedance parameters. This paper makes clear the relationship of impedance parameters with manipulability and response speed, and proposes a method to determine appropriate impedance parameters. Experimental results demonstrate that the medical doctor can perform diagnosis successfully using the proposed system.

This paper discusses a robotic surgical system for total knee joint replacement. The system construction from preoperative assist system to bone-cutting robot is proposed. According to the examined system overview, a development of the bone-cutting system is presented as a main issue of this paper. The characteristic feature of the bone-cutting robot is to establish high rigidity, high cutting accuracy, high operability and high safety. These performances can be achieved by well-designed hardware mechanisms and degree-of-freedom configuration of the robot. In the last part, model bone cutting experiments prove the validity of the system compared with the conventional method using a bone saw. To be specific, the waviness of the machined surface and the error of the edge angle which affect fitness to the artificial knee have been improved dramatically.

This paper presents a master-slave type remote ultrasound diagnostic system. In the system, the motion of the master manipulator is controlled by a medical doctor and is reproduced by a slave manipulator carrying an ultrasound probe. The contact force between the probe and a patient is fed back to the master manipulator. High rigidity was achieved by adopting radius-guides in the slave manipulator. These allow precise position and orientation control of the ultrasound probe. In the master manipulator, a special mechanism keeps the orientation of the force sensor constant. The mechanism separates the functions determining the orientation and position of the probe. The control algorithm implements impedance control to reflect force, with the viscosity of the system varied depending on the measured force at the slave manipulator. Furthermore, a control strategy to enhance the safety of the system is proposed. Using the developed system, remote ultrasound diagnosis experiments through ISDN lines were successfully executed.

When designing systems that support remote instruction on physical tasks, one must consider four requirements: 1) participants should be able to use non-verbal expressions, 2) they must be able to take an appropriate body arrangement to see and show gestures, 3) the instructor should be able to monitor operators and objects, 4) they must be able to organize the arrangement of bodies and tools and gestural expression sequentially and interactively. GestureMan was developed to satisfy these four requirements by using a mobile robot that embodies a remote instructors actions. The mobile robot mounts a camera and a remote control laser pointer on it. Based on the experiments with the system we discuss the advantage and disadvantage of the current implementation. Also, some implications to improve the system are described.

No abstract available.

This paper describes a master-slave type tele-micro-surgical system with an intelligent user interface. To enhance the accuracy and the rigidity of the slave-manipulator, multicircular guides were adopted for the rotational degrees of freedom. To increase the operability of the system, an "antishadow" technique was used and the 3D relative information was presented on a 2D display. Furthermore, the anti-shadow was processed using a real-time tracking vision system to guide the slave-manipulator to the target position automatically. It took approximately 25 minutes to suture a 1 mm diameter artificial micro-blood-vessel using the system. The carotid artery of a rat with a diameter of 0.3 mm was also successfully sutured.

A system which supports remote instruction for physical tasks in the real world should support following requirements: 1) flexibility of the body arrangement; 2) awareness of each other's orientations; and 3) communication of gestural information. To satisfy these requirements, we developed a system named "GestureMan". GestureMan is a mobile robot mounted with three cameras and a remotely controlled laser pointer on it. An instructor controls the robot remotely with a joystick and gives instruction to an operator. Two preliminary experiments using a low-speed Satellite link and high-speed ATM network respectively show that GestureMan has an ability to support the remote instruction. Furthermore, gigabit network enabled the natural interaction between the instructor and the operator by alleviating limitations such as time delay and narrow field of view.

The motion of the operator's fingers has been tracked by a real-time tracking vision system to provide a user interface to a machine tool. The desired cutter position is determined from the finger position and the machine tool path is controlled to follow. The system provides an environment where a machining center can be controlled remotely without any special apparatus, such as a master manipulator. Thus, the user interface described in this paper is one of the crucial technologies for "technology transfer with real feeling". Experimental results show the effectiveness and the performance of the developed system.

Describes a tele-micro-surgical system and an experiment in suturing an artificial blood vessel with diameter approximately 1 mm. An experiment was performed on September 25th and 26th, 1997 at an "operation room" and "surgery site" separated by a distance of approximately 700 km. In the experiment the vessel was successfully sutured by a surgeon.

This paper describes a tele-machining and tele-handling system which enables multiple operators to machine and handle an object in a remote environment. Engineers will be able to work cooperatively using the system, even if they are located remotely. The system can also be used for tele-education. The paper describes key technologies for tele-machining predictive display of geometrical information, real-time machining surface display using multi-axis force information, predictive auditory information presentation as well as tactile presentation of machining state. Implementation of the system for multiple operators and results of experiments are also discussed.

This paper describes a tele-micro machining system which works under a stereo SEM and has operational environment transmission capability. The system includes a 3-axis force sensing table, the signal from which is converted into auditory information to enhance the operability of the system. Furthermore, the cutting state in micro-cutting is reflected as force feedback to the master joystick using a newly introduced index which represents the cutting state. Finally, the cutting force in the micro-world is analyzed and compared with that in the macro-world.

In tele-micro-surgery, as in most tele-micro-handling systems, it is necessary to view the working space from various view angles and magnification ratios without disturbing the operation. This paper describes a human interface which enables selection of a natural view direction and magnification ratio through recognition of the posture change of the operator. The system observes the operator and understands his or her intention. Experimental results show the effectiveness of the proposed method and system.

This paper describes the construction of a telehandling/machining system with predictive and augmented information display. In particular, we propose a method for predictive information display and methods for information transformation of force to sound and visual information and illustrate their application through the use of a machine tool as an example of a tele-operated machine.

This paper describes the necessary functions and the basic structure of a tele-micro-surgery system. In particular, the system is comprised of a vision system which is focused at the end of a surgical tool and a slave manipulator, the rotational axes of which intersect at the same focal point. These mechanisms realize high positionability that is independent of the approach angle from the microscope and the manipulator to the small object that is being operated on. The system also incorporates the transformation of multi-axis force information to sounds that can be easily interpreted by the operator to indicate the contact state between the tool and the object. Furthermore, an inter-world tele-micro-handling experiment has been executed successfully from a remote site using the Internet and two artificial satellites.

This paper describes the construction of a tele-handling and tele-machining system at the macro and micro scales that uses the Internet for communication. The necessity of transmitting information concerning the operational environment requires an information compression method based on the real-time judgment of the machining state, the transmission of the compressed information and the transformation and augmentation of the information for presentation to the operator. A method to compensate for transmission time delays using the physical model of the object is also described. Results of tele-handling and tele-machining experiments connecting Washington DC and the University of Tokyo in Japan are discussed.

This paper describes the necessary functions and the construction of a micro-surgery system which can be operated from a separate place. In particular, a vision system which has a fixed visual point and a slave manipulator, which has the same fixed point at the end of the tool, are proposed. In the vision system, the axes of all rotational degrees of freedom intersect at the focal point of the microscope. In the slave manipulator, the axes of all rotational degrees of freedom also intersect at this point. These mechanisms provide high accessibility to small objects independent of the approach angle from the microscope and the manipulator to the object. Another feature of the system is the transformation of multi-axis force to auditory information to enhance the operability of the system by indicating the contact state of the end-effector and object. Furthermore, the visual information acquisition and display systems are controlled in accordance with the inferred intention of the operator. Finally, a tele-micro-handling experiment was performed using the Internet and two artificial satellites.

This paper describes a new micro-teleoperation system that is inherently suitable for micro-object handling since it mitigates several drawbacks associated with human interface devices. The system uses an upward-facing CRT display monitor mounted beneath a table. An operator performs micro-tasks in a sitting posture at the table while looking down at the monitor with a touch sensor screen situated between the operator's arms. To execute these tasks, microscope-observed images of the target objects are utilized in conjunction with the touch sensor screen and a specially designed, pencil-shaped master manipulator having a force-feedback function. The system incorporates the following characteristic features: (1) Intuitive teleoperation is possible because coincidence of the operating and monitoring point is provided. (2) The pencil-shaped master manipulator enables sensitive bilateral micro teleoperation because of its light weight and small inertia for movement, and (3) With the side of the right hand fixed in a pen-holder grip, the operator uses the master manipulator to generate commands with a wide dynamic range of force feedback, i.e., the operator can simultaneously feel force feedback during operation from both pressure/slip sensory receptors on the fingertips and sensory receptors which detects stretching of the finger muscles. The stated features are proved experimentally by the constructed micro teleoperation prototype system.>

This paper describes the construction of tele-macro/micro handling/machining system with a predictive information display. Applications of the system are tele-machining, where the operator controls a remotely located machining machine, a micro-handling system, which operates on micro-scale objects under the microscope, and a micro surgery system. In the case of constructing a teleoperated handling/machining system, displaying information appropriately to the operator is essential to improve the operability of the system. The authors selected a machine tool as a tele-operated machine and, in particular, discuss the method of predictive force and sound information display.>

Describes the construction of a tele-operated handling/machining system which operates in the micrometer to millimeter range. The system is equipped with a pair of three dimensional force sensors and can be controlled remotely using a three dimensional joystick. The paper presents a transformation method for converting multi-axis force information to auditory information. Such a transformation conveys a realistic sense of the state of the operation and allows high operability of the system. Experimental results are presented which demonstrate the effectiveness of the implemented system.

The authors describe a manufacturing system that allows the natural manipulation of tools of production and raw materials in environments which cannot be accessed directly by normal human senses. The term 'hyper-environment' connotes such an environment as remote locations, the micrometer world, or outer space. The authors discuss the functions necessary to realize the system, the software and hardware employed in the system, and the 'reality communication' techniques needed to allow the remote human operator to monitor the operation effectively. Attention is given to a technique for transforming cutting force information into auditory information that allows the operator to determine the cutting conditions. Tests in which a machining center in Tokyo was controlled by teleoperators in Washington are described.>