The vague interpretation of myoelectrical signals on the residual limb end makes restoring dexterous hand function in amputees still impossible. Understanding motor control between human motion intention and synaptic inputs to motor neurons also remains a significant challenge. The neural decoding methods of surface EMG signals remains challenging, which limit the application of robot hand in real life. Herein, we propose and substantiate a human-machine interface for motor control that introduces neural information of motor neurons in conjunction with the combination mechanism of muscle contraction. The interface firstly introduces a new concept of motor unit (MU) spike trains, which combines decoupling of the electrical activations on motor neuron axons with extraction of motion patterns from the discharge timings of the motor neuron pools. We realized a real-time implementation of the EMG decomposition algorithm on our developed prosthesis hand control system. The control scheme provides an accurate classification of intuitive hand motions, enabling the amputee to perform versatile finger movements of the prosthesis hand. The concept of motor neuron discharge timings was evaluated through experiments on one amputee participant and six able-bodied participants. The results show that the neuroprosthesis hand control scheme based on MU spike trains has the capacity of generating accurate and intuitive hand movements for amputees in a physical environment.

Gesture classification and motion speed regression are always two major issues in myoelectrical prosthetic hand research. However, there is little research considering these two issues in conjunction. Some shared EMG feature information in these two processing tasks is promising to improve the performance of prosthetic hand control. In this study, a joint-loss (JL) neural network architecture is proposed to implement gesture classification and motion speed regression problems in parallel by sharing the hidden neural units in the training process and optimizing the joint loss function. We evaluated the proposed control system through motion experiments performed on six participants. The experiment result shows that the classification and regression models can successfully reproduce smooth movement based on EMG measurement with high accuracy. Furthermore, the possibility of clinical application is demonstrated through the online movement of the real prosthetic hand.

Automation of surgical tasks is expected to improve the quality of surgery. In this paper, we address two issues that must be resolved for automation of robotic surgery: online trajectory planning and force control under dynamic conditions. By leveraging demonstrations under various conditions, we model the conditional distribution of the trajectories given the task condition. This scheme enables generalization of the trajectories of spatial motion and contact force to new conditions in real time. In addition, we propose a force tracking controller that robustly and stably tracks the planned profile of the contact force by learning the spatial motion and contact force simultaneously. The proposed scheme was tested with bimanual tasks emulating surgical tasks that require online trajectory planning and force tracking control, such as tying knots and cutting soft tissues. Experimental results show that the proposed scheme enables planning of the task trajectory under dynamic conditions in real time. In addition, the performance of the force control schemes was verified in the experiments.

The authors have investigated a construction methodology for a non-invasive ultrasound theragnostic system, called NIUTS, to track and follow moving body targets, such as tumors / stones located in moving organs (kidneys, livers, etc.), robustly and precisely, so as to compensate for nearly 90 percent of target motions in ultrasound images, while irradiating therapeutic ultrasound to focal lesions. In the present report, we propose a novel servo recovery algorithm for NIUTS. We confirm the effectiveness of the implemented servo recovery algorithm for a real human kidney, which, as far as we know, is the first successful report of semi-autonomous servo recovery systems for lost body targets. This robust servo recovery method could provide an indispensable tool for precise, safe, and efficient treatments of tumors and stones.

The authors have developed a bed-type non-invasive ultrasound theragnostic system (NIUTS) that compensates for movement by tracking and following the area to be treated by stereo ultrasound imaging while irradiating high-intensity focused ultrasound (HIFU) onto the affected area. In this paper, we propose a novel robust template matching method to track and follow body targets, which include tumors and stones for the NIUTS. The proposed novel robust template matching method could be applied to a motion tracking of the real human kidney based on the ultrasound images, which is the first successful report as far as I know. This robust visual servoing method could be a great tool to treat the tumors and stones precisely and safely.

In orthopedic surgery, bone-cutting procedures are frequently performed. However, bone-cutting procedures are very risky in cases where vital organs or nerves exist beneath the target bones. In such cases, surgeons are required to determine the depth of the penetration into the bone by using only their haptic senses. Thus, we developed a handheld bone-cutting-tool system that detects the penetration of the cutting material. The developed system autonomously detects the penetration before total penetration and stops the actuation of the cutting tool, leaving a very thin remnant of work material. The developed system estimates the cutting resistance by using its motor's current and rotational speed. On the basis of data collected preoperatively, the system estimates the cutting state by using a support vector machine (SVM). According to the SVM outputs, the system detects the penetration of the work material and autonomously stops the actuation of the cutting tool. The proposed method was verified through experiments, and the results showed that the developed system successfully detected the penetrations of work materials and stopped autonomously immediately before total penetration. This study showed that the autonomous detection of bone penetration with a hand-held bone-cutting tool is feasible by using the proposed scheme.

The automation of surgical tasks has great potential for improving the performance of robotic surgery. The learning-from-demonstration approach has thus far been employed by many researchers when planning the trajectories of robotic instruments for automated surgical tasks. However, previous methods are applicable only when the demonstrations and trajectory generation are performed under the same initial conditions. In this paper, we propose an algorithm that learns through demonstrations and generates a trajectory regardless of the initial conditions. The variance of the demonstrated trajectories over the initial conditions is modeled and learned using a statistical method, and the learned trajectories are generalized and used to generate a trajectory. As an example, the bi-manual looping task of a surgical thread was demonstrated by changing the initial positions of the robot arms, and a trajectory was then planned given these initial arbitrary positions. The proposed algorithm was verified through simulations and experiments using an actual robotic surgical system.

The authors have developed a non-invasive ultrasound theragnostic system (NIUTS) with a focal lesion servo (FLS) function based on stereo ultrasound imaging in order to compensate for movement while irradiating high-intensity focused ultrasound (HIFU) onto a focal lesion. In the present paper, we propose a redundant motion control mechanism of the HIFU focus, for therapeutics, that is independent of the ultrasound probes for diagnostics. Using the proposed redundant motion control mechanism, an arbitrary designated position in the focal lesion can be treated without altering the viewpoint of the ultrasound probes. The proposed mechanism enables (i) noise factors, which deteriorate the image quality (IQ) for FLS, to be reduced, thereby enhancing the FLS performance, and (ii) enables the destruction of a focal lesion (kidney stone) with a preoperatively designated locus that is moving due to respiration/heartbeat.

Remote surgery has attracted great interest in recent decades because it has the potential to improve equality of access to health care. However, remote surgery has not been approved in any country, since patient safety cannot be guaranteed. In this paper, we present a hybrid control system for master-slave velocity control and admittance control for safe remote robotic surgery. The proposed controller switches between these two controls adaptively and stably and avoids excessive contact force between the robotic surgical instrument and the patient's organs. Through the proposed scheme, the contact force between the robotic instrument and objects at the slave site can be bounded by a set upper limit regardless of the motion the operator inputs at the master site. The performance of the proposed system was verified analytically. The proposed control scheme was implemented in a robotic surgical system and its performance was verified through simulations and experiments. The results of the simulations and experiments showed that the system autonomously switches stably between master-slave velocity control and admittance control and avoids excessive contact force.

Advanced robotic assistance in microsurgery, such as automation, requires an accurate estimation of the state of the robotic forceps. In this paper, we propose a robust and accurate forceps tracking method to estimate the full state of the forceps (i.e., the position, posture, and grip parameters) using visual information obtained from stereo microscopic images and kinematic information obtained from the robotic sensory information, forward kinematics, and hand-eye coordination. An online method for updating the hand-eye coordination was also developed using an extended Kalman filter to cancel the hand-eye coordination errors caused by the repositioning of the microscope. The experimental results showed that the proposed method could accurately and robustly estimate the state of the robotic forceps even after the repositioning of the microscope.

Loss of haptic sensation in a master-slave system is one of the open problems in robotic surgery, and recognition of surgical situations through haptic sensation is a challenge. In this paper we propose an autonomous risk-detection system for a master-slave surgical robotic system in order to estimate a property of an object (i.e., contact impedance) using a force sensor mounted on a surgical robotic instrument. The system autonomously detects the risk based on the estimated contact impedance and accordingly activates the motion at the slave unit as well as the force feedback at the master unit. We implemented the proposed method in a teleoperated master-slave system to detect the perforation risk of a membranous object. The performance of the system was evaluated through experiments. The classification accuracy for perforation risk was about 98.5 % in fourfold cross-validation. The experiments verified that the risk detection system accurately detected the perforation risk and improved the safety of the master-slave system.

Forceps tracking is an important element of high-level surgical assistance such as visual servoing and surgical motion analysis. In many computer vision algorithms, artificial markers are used to enable robust tracking; however, markerless tracking methods are more appropriate in surgical applications due to their sterilizability. This paper describes a robust, efficient tracking algorithm capable of estimating the full state parameters of a robotic surgical instrument on the basis of projective contour modeling using a 3-D CAD model of the forceps. Thus, the proposed method does not require any artificial markers. The likelihood of the contour model was measured using edge distance transformation to evaluate the similarity of the projected CAD model to the microscopic image, followed by particle filtering to estimate the full state of the forceps. Experimental results in simulated surgical environments indicate that the proposed method is robust and time-efficient, and fulfills real-time processing requirements.

In this paper, we present the development of an intuitive user interface for hand-held multi-DOF forceps for laparoscopic surgery. Interface design is crucial to the development of hand-held medical instruments. Surgeons may not use medical devices owing to their poor usability, regardless of their performance. In addition, most surgeons do not have the time to learn the use of robotic instruments. We have listed all possible input mechanisms based on the motions of the fingers, and we have designed sample interfaces consisting of effective input mechanisms. The usability of the designed interfaces was evaluated by surgeons via experiments performed using the sample interfaces and a simulator. In summary, we propose a method for designing the most intuitive user interface for hand-held multi-DOF forceps.

We propose a non-invasive ultrasound theragnos tic system that tracks movement in an affected area (kidney stones, in the present study) by irradiating the area with high intensity focused ultrasound (HIFU). In the present paper, the concept behind a novel medical support system that integrates therapy and diagnostics (theragnostics) is illustrated. The re quired functions for the proposed system are discussed and an overview of the constructed system configuration is illustrated. The problems associated with kidney stone motion tracking by ultrasonography are described. In order to overcome these problems, we consider two approaches. The first approach is to minimize the servoing error so as to enhance both the efficiency of the therapy and the safety of the patient. The second approach is to reduce the effect of the servoing error. With respect to the first approach, we propose a robust detection method of the stone position based on shape information. With respect to the second approach, we propose a solution for controlling the HIFU irradiation power in accordance with the servoing error, primarily in order to enhance the safety of the patient.

Force feedback systems have been developed to improve the user-friendliness of robotic surgery systems. In teleoperative surgery, one of the major problems is the delay of the transmission of the information from the operation site to the surgery site. The transmission delays of the control signal and force information are less than the that of the image signal. In many cases, the visual information is synchronized with the force information in order to avoid a time gap between them. In this paper, we propose that force information be presented earlier than the visual information. This approach reduces the damage to the organs in surgical operations even though there is a time gap between the delay of force information and visual information. In addition, we propose a method to present predicted force information to avoid damage to the organs due to time delays in teleoperative surgery. The liver can be treated with a contact force of less than 0.8 N and a grasping force of less than 1.1 N by applying the proposed method. The method can realize a teleoperative surgery without damaging the patient's organs.

The non-invasive ultrasound theragnostic system, we propose, tracks and follows movement in an affected area -kidney stones here-, while High-Intensity Focused Ultrasound (HIFU) is irradiated onto the area. In this paper, the concept of the novel medical support system, which integrates the therapy and diagnostics, is illustrated at first. Secondly, structuring the required functions for the proposed system is discussed. Third, the overview of the constructed system configuration is illustrated. Fourth, the problem of the stone motion tracking by ultrasonography is clarified. To cope with this problem, the respiratory motion of a human kidney is analyzed and a controller, by utilizing the quasi-periodical motion of the respiratory kidney motion, is proposed. Finally, the result of the servoing and HIFU irradiation experiments of the model stone, which moves based on the real human kidney motion data, is reported to confirm the effectiveness of the proposed controller and the constructed system.

The increase of precision and minimal invasiveness is an important current issue in orthopedic surgery. The femur and the tibia must be shaped to fit an artificial joint for successful knee arthroplasty. The recent trend towards MIS (Minimally Invasive Surgery) to decrease the length of the required incision has increased surgical difficulty, since the open access area is small. In this paper, registration and cutting error were analyzed with a robotic surgery system being developed first as an example, and a method of active compensation of robot deformation by gravity and cutting force was proposed and tried based on the error map with the expectation that the precision will increase.

The master and slave surgical assistant systems have been studied actively. However, regarding the master system, the manipulator should be designed for each surgical field because the target workspace and precision are different. Furthermore, operability is important for safety reasons. Therefore, the authors analyzed the motion of surgeon first and then developed a master manipulator suitable for the operation of micro neuro surgery. Some experiments were conducted to evaluate the control method and show the effectiveness of the proposed manipulator control method.

Adaptive control of the machining process in orthopedic surgery will increase not only productivity but also the safety issues of tool usage. The authors have developed a technology for an adaptive control system. The system has two modes, the "predictive mode" and the "adaptive mode." The predictive mode is used to shorten the air cutting time. In the adaptive mode, the system monitors the cutting force and the cutting temperature, and controls the feed override according to the difference between the real and the desired cutting force. The software is implemented on the robot controller and its effectiveness is evaluated with a urethane bone.

Toolpath generation and optimization is considered as a challenging problem in the minimally invasive orthopedic surgery with a milling robot. The objective of this paper is to minimize the collision of the cutting tool with the soft tissues. A novel approach of toolpath generation and optimization is proposed. A redundant axis is implemented to avoid the collision in the robot. Some important components are modeled based on the physical requirements. A geometric optimization approach based on the model is proposed to improve the toolpath. Case studies show the validity of this approach. Software is developed for this application and the effectiveness is evaluated with a cadaveric bone.

In total knee joint replacement, the femur and the tibia must be precisely shaped to ensure that they firmly mate with the artificial joint. The normal procedure is to manually cut the bones using a bone saw and mechanical jigs. A 9-axis bone cutting machine tool has been developed by the authors to increase cutting accuracy and reduce the hospital stay required for recuperation. As developed, the cut plane is determined by a 5 degree-of-freedom mechanism and the femur and tibia are cut by moving two translational and one rotational degrees of freedom, a system that provides a cutter path which avoids cutting the ligament. A preoperative planning system based on CAD/CAM principles and an overview of a system for total knee arthroplasty is also discussed. Experimental results showed the effectiveness of the developed robot and the system.

This paper describes a tele-micro machining system which works under a stereo SEM and has operational environment transmission capability. The system includes a 3-axis force sensing table, the signal from which is converted into auditory information to enhance the operability of the system. Furthermore, the cutting state in micro-cutting is reflected as force feedback to the master joystick using a newly introduced index which represents the cutting state. Finally, the cutting force in the micro-world is analyzed and compared with that in the macro-world.