Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/taylor-francis/a-sensorless-force-feedback-system-for-robot-assisted-laparoscopic-2Erk0D4GDA
References
References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.
- Publisher
- Taylor & Francis
- Copyright
- © 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
- ISSN
- 2469-9322
- DOI
- 10.1080/24699322.2018.1557887
- Publisher site
- See Article on Publisher Site
Abstract
COMPUTER ASSISTED SURGERY 2019, VOL. 24, NO. S1, 36–43 https://doi.org/10.1080/24699322.2018.1557887 RESEARCH ARTICLE a b Baoliang Zhao and Carl A. Nelson Shenzhen Key Laboratory of Minimally Invasive Surgical Robotics and System, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China; Department of Mechanical and Materials Engineering, University of Nebraska-Lincoln, Lincoln, USA KEYWORDS ABSTRACT Surgical robot; sensorless The existing surgical robots for laparoscopic surgery offer no or limited force feedback, and force feedback; there are many problems for the traditional sensor-based solutions. This paper builds a teleoper- teleoperation system; ation surgical system and validates the effectiveness of sensorless force feedback. The tool-tissue laparoscopic surgery interaction force at the surgical grasper tip is estimated using the driving motor’s current, and fed back to the master robot with position-force bilateral control algorithm. The stiffness differ- entiation experiment and tumor detection experiment were conducted. In the stiffness differen- tiation experiment, 43 out of 45 pairs of ranking relationships were identified correctly, yielding a success rate of 96%. In the tumor detection experiment, 4 out of 5 participants identified the correct tumor location with force feedback, yielding a success rate of 80%. The proposed sensor- less force-feedback system for robot-assisted laparoscopic surgery can help surgeons regain tact- ile information and distinguish between the healthy and cancerous tissue. Introduction the robot-assisted laparoscopic surgery, the surgeon can sit far away from the patient and do the surgery In the laparoscopic surgery, surgeons use long rigid remotely, also the control system can filter the hand tools to operate on tissues through several small inci- tremor, which will help reduce the surgeon fatigue sions in the abdominal wall. This allows less bleeding, and improve the operation accuracy. However, less pain, shorter recovery time and improved cosmetic because surgeons cannot touch the surgical site dir- outcomes to the patients. However, the operation com- ectly, they are prone to exert larger forces than neces- plexity is greatly increased in this kind of minimally sary and cause tissue damage [7]. The loss of force invasive surgery, due to the non-intuitive tool control feedback is regarded as a main concern in the existing together with limited dexterity and surgical vision [1, robot-assisted laparoscopic surgery [8]. 2]. To increase the operability of surgical instruments Force feedback plays a very important role in sur- and get better visual access to the surgical site, robot- gery. It enables surgeons to perceive the mechanical assisted laparoscopic surgery has become popular. properties of tissue, evaluate its anatomical structures, The da Vinci Surgical System (Intuitive Surgical, and apply appropriate force control actions for safe California, USA), perhaps the most commercially suc- tissue manipulation [9, 10]. To prove the feasibility cessful surgical robot for laparoscopic surgery, offers and effectiveness of force feedback in laparoscopic surgeons magnified 3D HD vision, various surgical surgery, researchers have developed several surgical instruments with dexterity comparable to that of the systems. Moradi attached strain gauges on the tool human hand and enhanced ergonomics. It consists of shaft of a surgical grasper to measure the sideway a master-side robot and a slave-side robot, and runs manipulation forces, and conducted tissue character- in a teleoperation mode. In the surgery, the surgeon ization experiment to identify the stiffness difference manipulates the master robot and the slave robot fol- of three artificial tissue samples [11]. Sarmah attached lows the motion and operates on tissues [3, 4]. The piezoresistive force sensors on the jaw and strain ZEUS surgical system and RAVEN surgical system take gauges on the shaft of a laparoscopic grasper to a similar construction and operation modes [5, 6]. In measure the grasping force and sideway manipulation CONTACT Baoliang Zhao bl.zhao@siat.ac.cn Shenzhen Key Laboratory of Minimally Invasive Surgical Robotics and System, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China; Carl A. Nelson cnelson5@unl.edu Department of Mechanical and Materials Engineering, University of Nebraska-Lincoln, Lincoln, USA 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. COMPUTER ASSISTED SURGERY 37 force, and built a master-slave teleoperation system to investigate the role of force feedback in the robot- assisted surgery [12]. Wagner and Semere attached a commercial force/torque sensor at the tip of the surgi- cal instrument to measure the tool-tissue interaction force, and fed back the measured force to the master robot (PHANTOM haptic device), they conducted blunt dissection task with the teleoperation system and showed that the force feedback can reduce the applied force by 50% and the number of errors that damage tissues by 66% [13, 14]. Shi built a customized master-slave surgical system with force feedback func- tion, and attached a self-designed force/torque sensor at the tip of tool to measure the applied force, the master robot ran in a current control mode to reflect Figure 1. The surgical grasper (a) CAD model, (b) prototype. force from the tool tip, and the force feedback tests tumor detection experiment with freshly harvested had been conducted by pressing-down, pulling-up, porcine liver are conducted. lateral touching, and knotting on a freshly harvested porcine liver [15]. Culjat attached a customized sensor array at the tip of the surgical tool to measure the Materials and methods grasp force, and fed back the forcer to the da Vinci Surgical grasper prototype development master manipulator through arrayed pneumatic bal- loon actuators, the force feedback system can provide The steerable surgical tools used in robot-assisted sur- five distinct levels of effective tactile information to gery usually have multiple degrees of freedom to the surgeon’s finger [16]. enable dexterous tool tip motion, and are driven by The existing force-feedback robotic systems for lap- cables due to space limitation. For the cable-driven aroscopic surgery are using sensors to measure the tool tip, there exists the coupling problem [19], a applied force at the tool tip, however, due to the planetary gear-based decoupling mechanism proposed steam sterilization requirement of surgical tools, which in [20] is adopted in our design to decouple the yaw is high-temperature, high-pressure and high-moisture, motion and pitch motion. A 3-DOF surgical grasper the harsh environment may damage the sensors, or prototype is fabricated by 3D printing, with motorized requires sensor recalibration at least [8, 17]. Also, the grasper jaws and yaw joint. Each joint is equipped surgical tools for laparoscopic surgery are usually small with journal bearing to reduce friction, and driven by in size, with diameters less than 10 mm, this makes it a DC motor (Faulhaber 2224U012S DC motor in com- extremely difficult to integrate sensors on them, and bination with a 66:1 planetary gearhead) through the sensor attachment may also hinder the normal braided polyethylene cable. All the motors are fixed at function of the tools [18]. Generally, current solutions the base, and several idler pulleys are used to regulate based on extra sensors increase the cost while the cable route, as shown in Figure 1(a). The tool shaft decrease the robustness of the surgical systems. has a diameter of 15 mm. Figure 1(b) shows a picture In this paper, to address the aforementioned short- of the grasper prototype. Since each of its joints is comings, a sensorless force-feedback system for robot- driven independently by only one motor, the external assisted laparoscopic surgery is built, which consists of force at the tool tip on each DOF can be estimated by a 3-DOF motorized surgical grasper and a 3-DOF the corresponding motor’s driving current, which is force-reflecting robot, and runs in a master-slave tele- acquired from the motor driver at 2kHz and processed operation mode. Both the surgical grasper and force- with a low-pass filter [21]. Equation (1) shows the lin- reflecting robot are cable-driven, and the applied force ear relation between a joint’s external force and its at the tool tip is estimated by the driving motors’ cur- driving motor’s current. rent, the estimated force is fed back to the force- F ¼ NKI=L (1) reflecting robot which runs in torque control mode. To show the feasibility and effectiveness of sensorless where F represents the estimated reaction force, N force feedback in laparoscopic surgery, stiffness differ- represents the reduction ratio of the gearhead which entiation experiment with different materials and is 66, K represents the torque constant of the driving 38 BAOLIANG ZHAO AND CARL A. NELSON Figure 3. Force-reflecting robot prototype, (a) overall view, (b) cable-capstan transmission, (c) decoupling mechanism. Figure 2. Comparison between force estimation and force measurement on grasp DOF. motor which is 14.5 mNm, I represents the filtered motor current, and L represents the distance between the force applied location and the joint axis. A force- sensitive resistor (FlexiForce A201 with 4.4 N force range, Tekscan, Massachusetts, USA) is used to meas- ure the contact force. The force estimation method is tested on grasp DOF, and the comparison between estimated force and measured force is shown in Figure 2. It is noticed that the force estimation has an overshoot at the beginning of the loading process, which is caused by dynamic effects, then the ampli- tude decreases and settles to a steady state, which is slightly larger than the force measurement, with esti- Figure 4. Master-slave teleoperation system. mation error about 0.24 N (averaged for the four groups of data). 180 and 270 separately. Similar to the grasper proto- type, the force-reflecting robot has three motorized DOFs including a yaw joint and two grasp bars, the Force-reflecting robot prototype development upper bar of the force-reflecting robot controls the A force-reflecting robot is 3D printed with a similar upperjaw of thegrasper,the lowerbar of theforce- design (decoupling mechanism) as the surgical reflecting controls the lower jaw of the grasper, and the grasper, but with larger scale for convenient human- yaw joint of the force-reflecting robot controls the yaw machine interaction (Figure 3). All the joints are driven joint of grasper. This structure makes both the position by DC motors (Faulhaber 2642012CXR DC motor) and control and force reflection easy; furthermore, since all equipped with ball bearings to reduce friction. Cable- the motors can be fixed on the frame, the inertia of the capstan transmission, rather than gear head, is moving components remains low. All the motors run in adopted on the force-reflecting robot due to its low- the torque control mode to reflect forces at each joint. friction and zero-backlash properties as a speed reducer/torque amplifier. The capstan joint consists of a pre-tensioned cable clamped at two ends of the Master-slave teleoperation system capstan pulley and wrapped several times around the With the surgical grasper and force-reflecting robot threaded shaft of the DC motor. The cable-capstan introduced above, a master-slave teleoperation system transmission ratio is chosen as 10:1. To keep the is built, as shown in Figure 4. The surgeon manipula- cable-capstan transmission tensioned during the joint’s motion range (±45 for yaw joint, ±90 for grasp joint), tes the force-reflecting robot and receives force feed- the arc of capstan pulley for yaw and grasp joint is back at the same time, while the surgical grasper COMPUTER ASSISTED SURGERY 39 Figure 6. Contact instability. Figure 5. Block diagram of a 2-channel position-force bilateral teleoperation system. Table 1. Specification of the block diagram given in Figure 5. Symbol Description h , h position command from master robot and m s position output of slave robot v velocity of master robot C damping coefficient of master robot K environment stiffness F ,F human input force and environment reaction force h e K ,K position feed forward gain and force feedback gain p-p f-f M ,M mass of master robot and slave robot m s K ,K,K PID parameter of slave robot position controller p i d follows the motions of force-reflecting robot and per- forms on tissues. Figure 5 shows a 2-channel position- Figure 7. Performance of master robot after adding damp- force bilateral teleoperation block diagram, with terms ing logic. explained in Table 1. The surgical grasper (slave robot) runs a close-loop PID position controller, with the contact with the environment, which causes high-fre- force-reflecting robot position command as the input quency oscillation around the contact position. This and tool-tissue interaction force as output. The force- phenomenon is also observed in the force-reflecting reflecting robot (master robot) runs an open-loop master robot when the surgical grasper contacts tis- force controller, with tool-tissue interaction force as sues, as shown in Figure 6. To solve this problem, input and position command as the output. The bilat- damping logic is added to the master robot control, eral control structure is adopted in the master-slave as shown in Figure 5. Motors on the master robot will force reflection system introduced in this paper. produce damping forces proportional to their veloc- A common problem for force-reflecting devices is ities in opposite directions. The force reflection per- contact instability [22], which can be modeled by formance of the master robot is shown in Figure 7.It Equation (2). Z represents the environment (the object is noticed that, after the damping logic is added, the that the robot manipulates) impedance, x represents the contact instability phase with high-frequency motor distance between robot and environment, x represents oscillation does not happen, the master robot can the contacting point between robot and environment, switch between non-contact phase and stable contact and Z ðÞ x represents the impedance felt by the human: phase smoothly. It should also be noted that, in the non-contact phase, the motion of motor also causes a Z ðÞ x ¼ 1 þ sign x x (2) t ðÞ 0 slight force variation, which is due to the force estima- The equation demonstrates that Z ðÞ x will have a dis- tion error on the slave side (the motion of master continuity when the teleoperation system makes robot causes movement of slave robot, which leads to 40 BAOLIANG ZHAO AND CARL A. NELSON allowed to manipulate the different materials with the teleoperation system up to 1 min, to become acquainted with the different feelings when interact- ing with these materials. Then they were instructed to perform the manipulation tasks without watching (no visual feedback). One of these materials was randomly chosen and the participants were asked to operate the robotic grasper to grasp or press it; each material was tested only once, and after the participants fin- ished testing all three materials, they were asked to rank the stiffness of each. Table 2 shows the results for all five participants on the three DOFs. Figure 8. Experiment setup for stiffness differentiation on (a) The correct stiffness ranking of the three materials grasp DOF, (b) pitch DOF, (c) yaw DOF. is (1) wood, (2) foam, (3) sponge, and there are three pairs of relations in this ranking to be identified, wood motor current variation on the slave side, and the and foam, foam and sponge, wood and sponge. With force estimation algorithm regards it as external five participants testing on three DOFs, there are a force variation). total of 45 pairs of relations to be identified, and the results show that only two pairs of these relations are Results mistaken, yielding a success rate of 96%. Stiffness differentiation experiment Tumor detection experiment One of the benefits of force feedback is to help sur- geons explore the mechanical properties of tissue, so Burying gum or plastic lumps into tissue is a common that the surgeon can discriminate different tissues method to simulate tumors [23, 24], a polymeric cylin- such as fat, muscle and artery, or distinguish the exist- der (U7mm 3.5 mm and stiffness about 3 GPa) was ence of tumors. To show the stiffness differentiation embedded along the edge of a freshly harvested por- capability with the sensorless force-feedback system, cine liver to imitate the existence of an artificial tumor, three material samples with different stiffness were as shown in Figure 9. There was no visual cue to tell prepared, made of wood, foam and sponge. Made the location of the tumor, and our preliminary test with the same shape and size, these three samples showed that the stiffness of the tumor location was were tested separately on grasp, pitch and yaw DOFs, about two times higher than that of other locations as shown in Figure 8. [21], which is a typical case for the stiffness difference Five participants were asked to operate the master between healthy and cancerous tissue [25, 26]. robot controlling the slave grasper to interact with the Similar to the previous experiment, five participants three materials. Before the test, the participants were were asked to use the sensorless force-feedback Table 2. Stiffness differentiation result. COMPUTER ASSISTED SURGERY 41 Figure 9. The porcine liver with tumor phantom embedded. system to grasp the porcine liver at seven marked locations. Before the test, they were given up to 1 min to grasp the tissue locations with and without the tumor to get acquainted with the different sensations. Then they were instructed to grasp the seven loca- tions randomly (assisted by another person placing the tissue between the grasper jaws), with each loca- tion being grasped once. After the participant finished testing all the locations (blinded to their order), he/ she was asked to choose the location he/she believed to have the embedded tumor. To demonstrate the necessity of force feedback, the contrast experiment with visual feedback was followed; with the force feedback function off, the participants were asked to do this tumor detection task again. Figure 10(a) shows the results for all five partici- pants with force feedback only, with a success rate of 80%. Only 1 participant failed, but he indicated that he was not sure among locations 4, 6, 7 (he chose 6 at the end); after given a second chance to test the Figure 10. Tumor detection result with (a) force feedback, (b) three locations, he identified the correct visual feedback. tumor location. Figure 10(b) shows the results for all five partici- since they don’t know how much grasp force they are pants with visual feedback only, with a success rate of applying, thus causing tissue bleeding. 20%. All the participants said they felt the same when grasping all the locations; the only cue they used to Discussion judge the tumor location was the tissue deformation when the tissue was being grasped. They indicated To address the force feedback problem of the robot- that this was kind of a guessing process and it was assisted laparoscopic surgery, the current solution is much harder to tell the tumor location without force to attach sensors on the shaft or at the tip of surgical feedback. Location 6 got more votes because the tis- instruments, to measure the tool-tissue interaction sue is a little thinner there, so the tissue deformation forces. Due to the size problem and sterilization pro- is slightly less significant than at the other locations cess, the sensor-based solution has difficulty on its when tissue is grasped, which is an indicator of the practical application. In this paper, a master-slave tele- presence of the tumor. operation system has been built to explore the feasi- The comparison between the test with and without bility and effectiveness of sensorless force feedback. It force feedback shows that the force feedback really consists of a 3-DOF motorized surgical grasper and a plays a significant role in the tumor detection process. 3-DOF active force-reflecting robot, and the tool-tissue It was also observed that when the force feedback interaction forces at the grasper tip is estimated with function is off, participants are prone to apply larger driving motors’ current, then the estimated forces are forces, and the participants squeeze the tissue hard fed back to the force-reflecting robot to give the 42 BAOLIANG ZHAO AND CARL A. NELSON [8] Okamura A M. Haptic feedback in robot-assisted min- surgeon a sensation of the tissue that is being manip- imally invasive surgery. Curr Opin Urol. 2009;19(1): ulated. With this system, the stiffness of different 102–107. materials has been distinguished, and the location of [9] Talasaz A, Patel RV. Integration of force reflection an embedded tumor in a porcine liver has been with tactile sensing for minimally invasive robotics- clearly identified, which proves the feasibility of sen- assisted tumor localization. IEEE Trans Haptics. 2013; sorless force feedback in robot-assisted laparoscopic 6(2):217–228. [10] Moradi D M, Shirinzadeh B, Nahavandi S, et al. Effects surgery. Furthermore, the comparison of tumor detec- of realistic force feedback in a robotic assisted minim- tion result with and without force feedback shows ally invasive surgery system. Minim Invasive Ther that the force feedback does help surgeons regain Allied Technol. 2014;23(3):127–135. tactile information and distinguish between the [11] Moradi DM, Shirinzadeh B, Shamdani AH, et al. An healthy and cancerous tissue, which expands the cap- actuated force feedback-enabled laparoscopic instru- ability of surgeons in the robot-assisted laparoscopic ment for robotic-assisted surgery. Int J Med Robotics surgery. For future work, the force estimation algo- Comput Assist Surg. 2014;10(1):11–21. [12] Sarmah A, Gulhane UD. Surgical robot Q with rithm will be improved to obtain higher accuracy, and haptics feedback system. International Conference the surgical tool (slave side) will be attached on a on Emerging Trends in Robotics and robotic arm to explore the performance of this force Communication Technologies. Chennai, India: IEEE; feedback system in more complex, clinically represen- 2010. p. 288–291. tative tasks. [13] Wagner CR, Howe RD, Stylopoulos N. The role of force feedback in surgery: analysis of blunt dissection. Haptic Interfaces for Virtual Environment and Funding Teleoperator Systems, 2002. Haptics 2002. Proceedings. Symposium on. Orlando (FL): IEEE; 2002. This work was supported by the US National Institute of p. 68–74. Biomedical Imaging and Bioengineering [award 5 R21 [14] Semere W, Kitagawa M, Okamura AM. Teleoperation EB015017-02], Key Fundamental Research Program of with sensor/actuator asymmetry: task performance Shenzhen [No. JCYJ20170413162256793, No. with partial force feedback. International Conference JCYJ20160608153218487] and in part by Shenzhen Peacock on Haptic Interfaces for Virtual Environment and Plan [No. KQTD2016113010571019] and Shenzhen Key Teleoperator Systems. Washington, DC: IEEE Laboratory Project [No. ZDSYS201707271637577]. Computer Society; 2004. p. 121–127. [15] Shi Y, Zhou C, Xie L, et al. Research of the References master–slave robot surgical system with the function of force feedback. Int J Med Robot, 2017(4). [1] Kuo CH, Dai JS, Dasgupta P. Kinematic design consid- [16] Culjat M, King C H, Franco M, et al. Pneumatic balloon erations for minimally invasive surgical robots: an actuators for tactile feedback in robotic surgery. overview. Int J Med Robotics Comput Assist Surg. Industrial Robot. 2008;35(5):449–455. 2012;8(2):127–145. [17] Puangmali P, Althoefer K, Seneviratne L D, et al. [2] Roh HF, Nam SH, Kim JM. Robot-assisted laparoscopic State-of-the-art in force and tactile sensing for minim- surgery versus conventional laparoscopic surgery in ally invasive surgery. IEEE Sens J. 2008;8(4):371–381. randomized controlled trials: a systematic review and [18] Min CL, Chi YK, Yao B, et al. Reaction force estimation meta-analysis. Plos One. 2018;13(1):e0191628. of surgical robot instrument using perturbation obser- [3] Senapati S, Advincula AP. Surgical techniques: robot- ver with SMCSPO algorithm. IEEE/ASME International assisted laparoscopic myomectomy with the da Conference on Advanced Intelligent Mechatronics. V R Vinci surgical system. Int J Med Robotics Comput Montreal, Canada: IEEE. 2010:181–186. Assist Surg. 2007;1(1):69–74. [19] Nishizawa K, Kishi K. Development of interference- [4] Yu J, Wang Y, Li Y, et al. The safety and effectiveness free wire-driven joint mechanism for surgical manipu- of Da Vinci surgical system compared with open sur- lator systems. J Robotics Mechatronics. 2004;16(2): gery and laparoscopic surgery: a rapid assessment. 116–121. J Evid-Based Med. 2014;7(2):121–134. [20] Zhao B, Nelson CA. Decoupled cable-driven grasper [5] Kakeji Y, Konishi K, Ieiri S, et al. Robotic laparoscopic design based on planetary gear theory. J Med Dev. distal gastrectomy: a comparison of the da Vinci and 2013;7(2):020918. Zeus systems. Int J Med Robotics Comput Assist Surg. [21] Zhao B, Nelson CA. Estimating tool-tissue forces using 2006;2(4):299–304. a 3-DOF robotic surgical tool. J Mech Robotics. 2016, [6] Hannaford B, Rosen J, Friedman D W, et al. Raven-II: 8(5):0510151–05101510. an open platform for surgical robotics research. IEEE [22] Hannaford B, Anderson R. Experimental and simula- Trans Biomed Eng. 2013;60(4):954–959. tion studies of hard contact in force reflecting tele- [7] Lim SC, Lee HK, Park J. Role of combined tactile and operation. IEEE International Conference on Robotics kinesthetic feedback in minimally invasive surgery. Int J Med Robotics Comput Assist Surg. 2015;11(3): and Automation, 1988. Proceedings. Piscataway: IEEE; 360–374. 1988. p. 584–589 (vol. 1). COMPUTER ASSISTED SURGERY 43 [23] Beccani M, Natali CD, Benjamin CE, et al. Wireless tis- [25] Hoyt K, Castaneda B, Zhang M, et al. Tissue elasticity sue palpation: head characterization to improve properties as biomarkers for prostate cancer. Cancer tumor detection in soft tissue. Sens Actuat A Phys. Biomarkers. 2008;4(4–5):213. 2015;223(1):180–190. [26] Stewart DC, Rubiano A, Dyson K, et al. Mechanical [24] Perri MT, Trejos AL, Naish MD, et al. Initial evaluation characterization of human brain tumors from patients of a tactile/kinesthetic force feedback system for min- and comparison to potential surgical phantoms. Plos imally invasive tumor localization. IEEE/ASME Trans One. 2017;12(6):e0177561. Mechatronics. 2010;15(6):925–931.
Journal
Computer Assisted Surgery – Taylor & Francis
Published: Oct 1, 2019
Keywords: Surgical robot; sensorless force feedback; teleoperation system; laparoscopic surgery