JP2009183733A - Minimally-invasive surgical operation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system which controls a slave manipulator to conduct minimally-invasive surgical operation from a remote control operator station to a worksite. <P>SOLUTION: The system comprises a manually-operated master link mechanism and a servomechanism which is composed so as to link the master link mechanism with a slave manipulator arm, a slave wrist member and a slave end effector. The servomechanism is composed to link a slave manipulator arm 34R with a master manipulator arm 76R so that the slave manipulator arm acts according to manually-operated motion of the master manipulator arm, to link the slave wrist member with a master wrist member so that the slave wrist member is jointed according to manually-operated motion of the master wrist member, and to link an slave end effector 40R so that the end effector acts according to manually-operated motion of a master end effector. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention generally relates to the teleoperator method and apparatus, particularly including means for providing the remote operator with the same sensation as felt directly by the operator's hand working at the work site.

  Tele-operating, as is well known, involves a human performing work at a remote location using a manipulator.

  Telepresence involves providing the teleoperator with the same feedback control that is performed with his own hands once he is at the work site where the operator actually operates. Telepresence operations generally involve the use of stationary visual displays, particularly stereoscopic display at remote workplaces. Stereoscopic television systems are well known, as shown, for example, in U.S. Pat.

  Remote manipulators using stereoscopic televisions that project with feedback power are also described, for example, in “Control of remote manipulators through kinematic coupling”, Beczy et al., Computers in Mechanical Engineering, 1983. July, pp. 48-60 and “Stereo Advantage for a Peg-In-Hole Task Using a Force-Feedback Manipulator”, Spain (E.H. Spain, Stereoscopic Displays and Applications (Stereoscopic Displays and Applications) s) The 1256 Volume, 1990, is well known as shown in the article pages 244-254. The Bexie et al paper includes a force-torque feedback. U.S. Pat. No. 3,921,445 shows a manipulator that includes power, torque and a slip sensor of the type that may be used in the present invention.

The operator of the conventional manipulator is sent a fixed three-dimensional image at the work site, and the manipulator is manually feedback-controlled, but the operator is not given a sense of actually existing at the work site. . The present invention is directed to an apparatus for use in a remote control system that substantially adds the presence of an operator at the remote manipulator site.
U.S. Pat. No. 4,562,463 US Pat. No. 4,583,117 UK patent application GB2,040,134

  One of the objects of the present invention is to provide an improved teleoperator system and method, including an improved monitoring system, that enhances the operator's sense of presence with a remote manipulator controlled from a remote location by the operator. .

  One object of the present invention is to provide an improved teleoperator system and method of the type described above, where the image of the end effector of the manipulator monitored by the operator is used by the operator to control the end effector. It is perceived by the operator as including an important part of the manual controller, and as a result, gives the operator a strong sense obtained at the work site.

  One object of the present invention is to provide an improved teleoperator system and method of the above type that is well suited for use in a wide variety of applications including military, industrial, biomedical, and the like.

  The present invention includes a manipulator installed at a work site and controlled by manual controller means at a remote operator control station. Manipulator end effectors are used at the work site to manipulate objects placed on the work site, and force torque feedback is used to transmit the mechanical resistance received by the end effector back to the operator. ing.

  A stereoscopic display provides the operator with an image of the work site. According to the present invention, the image is placed in proximity to the manual controller means, and the operator views the monitoring image in the direction of the manual controller means and close to the manual controller means. An image of the work site, either real or temporary, will be provided in close proximity to the manual controller means. A display means for projecting a real image is placed in close proximity to the manual controller means for direct monitoring of the real image by the operator. A mirror is placed between the operator's eyes and manual controller means to project a temporary sky image of the work site. In this case, the display means displays an actual image upside down, and the inverted image is projected through the mirror, which reverses the image and gives the operator a temporary sky image of the work site. And it appears that this work site is located close to the manual controller means.

  Despite the fact that the end effector is located at the work site and the manual controller means is located at the remote operator control station, by placing an image of the work site in close proximity to the manual controller means, the operator An effect is provided in which the effector and the manual controller means are substantially integrated. A stereophonic sound system may be included that provides the operator with stereophonic sound from the work site. The video camera means is used for monitoring the work site to obtain an image of the work site. Various other sensors and their associated responders are installed at the work site and operator control station, respectively, for the transmission of pressure, tactile, thermal, vibration and similar information for enhanced telepresence operations. May be.

Depending on the application, different magnification operations may be used in communicating information between the operator control station and the work site. For example, in operations such as microassembly, microsurgery, and small part manipulation, optical and / or video magnifiers may be used to provide magnified three-dimensional images that are monitored by the operator. Good. With a similar magnification operation between the manual controller means and the manipulator, the operator's sensation will be substantially the same as a reduced version of the operator would have if at the work site.
BRIEF DESCRIPTION OF THE DRAWINGS The present invention, together with other objects and advantages, will best be understood from the following description taken in conjunction with the accompanying drawings, in which: It will be understood that the drawings are for purposes of illustration and example only and that the description is not so limited. In each figure, like reference numerals indicate the same parts.

  1-3, the teleoperator system is shown as including an operator control station 20 (FIGS. 1 and 2) and a work site 22 (FIGS. 1 and 3). The operator 18 of the operator control station controls the manipulator 24 at the remote work site. The manipulator 24 consists of right and left manipulators 24R and 24L, and is used to manipulate an object such as an object 26 placed on a platform or base 28 in a work space 30 indicated by broken lines, respectively. For purposes of illustration only and not by way of limitation, the right manipulator 24R is shown as including a housing 32R attached to the base 28, from which a telescoping operating arm 34R extends. ing. The inner end portion (end portion on the base side) of the operation arm 34R is mounted so as to pivot in an arbitrary pivot direction using conventional mounting means. For example, the inner end of the operating arm 34R is mounted to pivot about a horizontal pivot axis 36, and the horizontal pivot axis is pivoted about a vertical axis 38.

  The operating arm 34R includes a telescoping inner portion 34R1 and an outer portion 34R2, the outer portion being axially moved inwardly of the inner portion 34R1, axially moved outwardly from the inner portion 34R1, and about its longitudinal axis. It is adapted to the rotational movement of An end effector 40R is supported on the outer end (terminal end) of the operating arm and is shown as including a gripper for purposes of illustration. The motor (not shown) includes a pivoting motion of the operating arm 34R about the pivot axes 36 and 38, an axial motion of the outer arm portion 34R2 along the longitudinal axis of the operating arm and a rotational motion of the outer arm portion 34R2 about the longitudinal axis, and a gripper. Control the opening and closing of 40R. The motor may be included in the housing 32R together with a motor control circuit for controlling the motor. The motor is connected through the right manipulator interface 44R and the motor control circuit under the control of the computer 42.

  The left manipulator 24L has substantially the same design as the right manipulator 24R, and the same reference numerals are used to indicate the same parts, but a subscript L is added instead of R. For illustrative purposes, the left end effector 40L shown in FIG. 3 is shown as including a cutting blade that operates in a pair of scissor blade manner.

  The work site is provided with a pair of video cameras 46R and 46L, and the work space 30 is observed from different angles in order to create a stereo signal output therefrom on lines 48R and 48L. The angle γ between the optical axes of both cameras shown in FIG. 3 is substantially equal to the viewing angle γ between the eyes of the operator in the image of the workspace as shown in FIG.

  The video camera output on lines 48R and 48L is supplied to an image memory 50 for instantaneous storage of the video field of right and left images from the camera. The field of view of the right and left images from the image memory 50 are alternately supplied via a left / right switch 52 to a visual display 54, such as a television monitor for alternate display of two images on the monitor surface 54A. . The timing and control means 56 includes elements included in the stereoscopic display system for timing and control of the system signal timing and control signals to the various elements of the system. If the digital storage means 50 is used, the analog signal is converted from the left / right switch to the analog signal format for conversion to the digital signal form of the camera signal output and display on the monitor 54 by the analog / digital converter prior to storage. Signal output conversion may be used.

  The electro-optic device 58 on the surface of the display means 54 controls the polarization received from the display means 54 under the control of the left / right synchronization signal from the timing and control unit 56. The left and right image fields are viewed by an operator 18 wearing a pair of passive polarizing glasses with right and left polarizing elements 62 and 64 polarized at right angles. The polarized light from the display 54 that has passed through the electro-optical device 58 is synchronized with the visual field such that the right visual field is shielded from the left eye and the left visual field is shielded from the right eye for stereoscopic viewing by the operator. It is. Other methods for viewing stereoscopic images in the left and right image fields, including those using active stereo glasses, can also be used in the practice of this invention to give the operator a stereoscopic view of the remote workspace. Means are well known.

  The vertical deflection coil connection for the monitor 54 is reversed, causing the monitor to scan from bottom to top, thereby producing a top-bottom inverted image 30I of the work space 30. The letters a, b, c and d are used to identify the corresponding corners of the work space 30 and the inverted work space image 30I. The inverted work space image 30I is observed by an operator through a mirror 66 on the upper surface of the table 68, and the mirror inverts the image 30I and reflects the image so that the operator can observe the upright position. When looking down in the direction of the mirror, the operator observes the virtual image 30V of the work space 30. According to one aspect of the present invention, in FIGS. 1-3, the image observed by the operator includes a virtual image and is located adjacent to the controller means 70 used by the operator for control of the manipulator means 24 at the work site. .

  The controller means 70 is disposed immediately below the table top surface 68 and includes right and left controllers 72R and 72L, and controls the right and left manipulators 24R and 24L, respectively. Since the right and left controllers are virtually identical in design, one explanation applies to both. As in the manipulator, the subscripts R and L are used to distinguish between the right controller element and the left controller element. For purposes of explanation and not limitation, the right controller 72R is shown to include a housing 74R secured to the bottom of the table top surface 68, from which a telescoping control arm or manual control arm 76R in the form of a stick is now shown. Extends.

  The right and left control arms 76R and 76L are each given the same degree of freedom as the associated operating arms 34R and 34L. For example, the inner end of the control arm 76R is pivoted about a horizontal axis 36 corresponding to the manipulator pivot axis 36, which is suitable for pivoting about a vertical axis corresponding to the manipulator axis 38. ing. The control arm 76R also includes an inner portion 76R1 and an outer portion 76R2, the outer portion being suitable for both axial movement into and out of the inner portion 76R1 and rotational movement about its longitudinal axis. Yes. It will be apparent that the control arm 76R is provided with four degrees of freedom similar to the associated operating arm 34R. Further, a sensor means 78R is disposed adjacent to the outer end of the outer portion 76R2 of the control arm and is used to control the gripping operation of the gripper 40R. Similarly, sensor means 78L is adjacent to the outer end of control arm 76L and is used to control the operation of scissor blade 40L.

  Right and left controllers 72R and 72L are included in the servo mechanism in which control arms 76R and 76L control the position of operating arms 34R and 34L, and the pressure on sensor means 78R and 78L is controlled by end effectors 40R and 40L. Control the opening and closing of each. In FIG. 1, right and left hand controller interfaces 80R and 80L are shown for connection of the controller to computer 42, respectively. Servo mechanisms for controlling mechanical motion at remote locations are known, which provide force and torque feedback from the manipulator to the manual controller means. Any suitable servomechanism can be used to practice the present invention, and a combination of force and torque feedback is particularly preferred for telepresence operation of the apparatus of the present invention.

  In the illustrated apparatus, right and left microphones are provided at the work site, and the outputs from these microphones are amplified by right and left amplifiers and fed to right and left speakers at the operator's control station to provide stereophonic sound to the operator in the work space. Provide stereo sound output.

  In FIG. 1, only the right channel of this stereo system is shown, which includes a right microphone 82R, a right amplifier 86R, and a right speaker 88R. Here, the left microphone and speaker are located behind the respective right microphone and speaker and operator control station at the work site (FIG. 1). Of course, earphones can be provided for the operator instead of these speakers, which can help to block external noise at the operator's control station. Further, in FIG. 1, a light shielding plate 54B in the monitor is shown to prevent direct observation of the monitor surface by the operator.

Reference is now made to FIG. 4, which shows a simplified diagram of the apparatus shown in FIGS. 1-3 and various lengths and angular positions are indicated by the figure number. In FIG. 4, the optical path length between the camera and the point F in the work space is indicated by a figure number L. The corresponding optical path length between the operator's eye and point F in the virtual image of the workspace is indicated by the distance a + b, where a is the distance from the operator's eye to the mirror 66, and b is the virtual image in the virtual image. The distance from the mirror to point F. Other dimensions shown here further include the camera height G on the pivot point of the operating arm 34R and the corresponding height g of the operator's eye on the pivot point of the control arm 76R. The operation arm 34R is adjusted to the length D by the control arm 76R having the length d. Similarly, with the control arm 76R at an angle β _A with respect to the vertical, the operating arm 34R is positioned at the same angle from the vertical line. The angle from the vertical line where the camera looks at the workspace and the angle from the vertical line where the operator's eye sees the virtual image of the workspace is denoted by α. The following relationships exist between the work position elements and the operator control station:
a + b = kL (1)
d = kD (2)
g = kG (3)
Here, k is a scale factor constant.

  When k is 1, a + b = L, d = D, and g = G, and it is not necessary to calculate the size of the work site.

  Any scale factor can be used here and the present invention is not limited to full-scale operation. For example, the work site can be small, including, for example, microscopic dimensions, in which case the optical parameters and the mechanical and dimensional parameters relating to the object, the distance between the eyes and the focus are applied appropriately. By using appropriate scaling and image magnification, force and torque feedback, and by positioning the image 30V of the workspace 30 adjacent to the manual control arms 76R and 76L, the operator It can have a strong sense of directly controlling 40R and 40L. The operator feels that the end effectors 40R and 40L and the respective control arms 76R and 76L are substantially united. This same sense of integration of the manual control arm and the end effector is not realized in a known art configuration in which the image viewed by the operator is not located in proximity to the manual control arm. Even if the known technique includes stereoscopic image observation and force and torque feedback, there is a sense of disconnection from the observation image object to be worked. The present invention overcomes this sense of disconnection by placing a working space image that appears to be the operator's hand directly controlling the end effector.

For small-scale operations, such as those required for surgical applications, it is desirable to reproduce the observation experience that a reduced observer would have if at the actual work site. FIG. 5 shows a state where the virtual eye 90 of the virtual reduced observer is observing the actual work space. The light from the light source at one point in the actual work space, X, Y, Z, gives a certain stimulus on the eye 90 of the reduced observer at the point represented by X '/ M. In FIG. 6, the actual operator's eye 92 is shown observing the enlarged virtual image generated by the video camera 94 used to observe this actual workspace. The camera shown here includes a solid-state imaging device such as a light receiving lens 96 and a CCD (Charge Coupled Device) array 98, in which the point light sources at X, Y, Z are denoted by X _i , Y _i , Z _i . It is.

On an accurate scale, the corresponding light source is generated in MX _i , MY _i , MZ _i either at the actual or apparent position of the surface of the viewing display, but the device system's stereo image manipulation actually does It appears that they are generated from points MX, MY, MZ corresponding to points X, Y, Z in the workspace. In the retina of the actual eye 92, a stimulus is generated at the point X ′ at the same position in proportion to the point X ′ / M in the virtual observer eye 90. This relationship selects the precisely sized camera distance and lens focal length such that the optical magnification M _O = M / M _V (where M is the desired overall magnification and M _V is the video magnification). Guaranteed by. Typically video magnification M _V is about 40, said magnification is equal to the ratio between the width and the display width of the CCD array 98.

  Here, FIGS. 7 to 9 show an embodiment of the present invention used in the medical field. Shown here are right and left manipulators 100R and 100L, which are shown controlled by right and left controllers 102R and 102L, respectively. Each element of the imaging apparatus system is the same as that used in the imaging system shown in FIGS. 1 to 3 except that an enlarged virtual image 104V of the actual work space 104 is provided for observation by an operator. Substantially the same. Further, the servomechanism elements for connecting the right and left controllers 102R and 102L to the respective manipulators 100R and 100L are also substantially the same as those described with respect to FIGS. In the apparatus shown here, the right and left manipulators are substantially identical in structure to the right and left controllers, i.e., the description of one manipulator and one controller is equally applicable to both. In addition, the suffixes R and L are used to distinguish the right and left elements.

  These manipulators include external control sections 100RA and 100LA and insertion sections 100RB and 100LB, where the insertion section is a portion suitable for insertion into a body cavity via a cylindrical tube or cannula. For illustration purposes, a manipulator is shown inserted through the abdominal wall 106 of the patient. As is widely understood, for laparoscopic surgery, the abdominal wall 106 is separated from the viscera by a blowing method, which is a method of introducing gas into the abdomen by suitable means, not shown. The manipulator motor and the motor control circuit combined therewith are placed in the external control sections 100RA and 100LA of the manipulator to control the insertion section. The manipulator is carried by a fixed rail that forms part of an operating table on which a patient is held together with a laparoscope 108 for observing organs in the abdominal cavity.

  Manipulator insertion sections 100RB and 100LB may be of substantially the same configuration as operating arms 34R and 34L described above with respect to the embodiment of FIGS. The insertion section has a relatively small size for use inside the body. Insert section 100RB includes a telescoping inner section 112R1 and an outer section 112R2, which is adapted for both axial movement into and out of inner section 112R1 and rotation about its longitudinal axis. . End effectors 114R and 114L are carried at the outer ends of the right and left outer sections 112R2 and 112L2, respectively, for manipulation of the organ 116. Inner section 112 </ b> R <b> 1 is adapted for pivoting movement about a crossing vertical axis substantially located at point P where the insertion section crosses wall 106. Except for the operation of the end effectors 114R and 114L, each operation arm is given four degrees of freedom as in the embodiment shown in FIGS. The end effectors 114R and 114L are essentially simply removed from the handle including retractors, electrosurgical cutters and coagulators, microtweezers, microneedle holders, dissecting scissors, blades, washers and sutures. May comprise a microsurgical instrument.

  A laparoscope 108 for viewing the working space 104 is shown comprising an outer operating section 108A and an insertion section 108B. The outer end section 120 of the insertion section 108B is axially and rotatably movable within its inner end section 122 and is provided with a pair of image transmission windows 124, 124 for viewing the working space 104 in three dimensions. Yes. The laparoscope is also provided with illuminating means (not shown) for illuminating the working space and liquid inlet and outlet means (not shown) for liquid to flow through the window. . The video camera means in section 108A is responsive to light received through the viewing window for the generation of left and right electronic images at output lines 48R and 48L for connection to image memory 50. An enlarged three-dimensional image 104I is created in the display means 54 for viewing by the interferometric polarization glasses 60 worn by the operator via the mirror 66. Similar to the embodiment shown in FIGS. 1-3, a working space 104 is created adjacent to the control arms 130R and 130L of the controllers 102R and 102L. Control arms 130R and 130L are of the same type as control arms 76R and 76L included in the embodiment of FIGS. 1-3 described above. They include nested inner and outer sections 132R1 and 132R2 and 132L1 and 132L2. Sensor means 134R and 134L located adjacent to the outer end of the control arm control the operation of the end effectors 114R and 114L, respectively, in the manner described above with reference to FIGS. 1-3. It will be noted here that the angle from the vertical line where the image is viewed by the operator need not be equal to the angle from the vertical line where the object is viewed by the camera. In the arrangement shown in FIGS. 7-9, the operator is shown to view the image 104V from the vertical at an angle θ, while the object 116 is shown to be viewed directly downward. In the absence of an external reference, the sense of vertical lines in the body is not particularly great, and there is no confusion in the operator's spirit as a result of the observer and camera looking at various angles to the vertical lines.

  In the case of the embodiment of FIGS. 7-9, not only is an enlarged virtual image 104V of the workspace provided for viewing by the operator, but control arms 130R and 130L of a length longer than the length of the manipulator insertion sections 100RB and 100LB are used. . Servo mechanism scaling of the axial movement of the telescoping control arm is provided so that its axial extension or retraction results in a smaller extension or retraction of the telescopic insertion section. Without scaling, the angular pivoting motion of the control arms 130R and 130L causes the same angular pivoting motion of the insertion sections 100RB and 100LB, and the rotational motion of the control arm end sections 132R2 and 132L2 is the insertion section of the right and left manipulators. Cause the same rotational movement of the end sections 112R2 and 112L2. This aspect of the invention with respect to magnified images is particularly useful in the field of microsurgery, especially when the surgeon cannot reach a given area by hand due to size limitations.

  The present invention is not limited to use with manipulators having any particular number of degrees of freedom. Manipulators with various degrees of freedom known in the art can be used in the practice of the present invention. Referring now to FIGS. 10 and 11, there are shown a controller 140 and a manipulator 142 that are each provided with a wrist joint to provide additional freedom of movement. The illustrated controller 140 includes a housing 144 attached to the bottom of the table top 68 on which the table mirror 66 is placed. An enlarged virtual image 146V of an actual work space (also referred to as a surgical space) 146 is provided adjacent to the operator's hand 148 so that it can be viewed by the operator when looking down on the mirror 66 in the manner described above.

  A control arm 150L, consisting of inner and outer sections 150L1 and 150L2, respectively, is installed in the housing 144 to pivot in any pivoting direction as shown by intersecting the double-headed arrows 152 and 154. The outer section 150L2 is adapted for axial movement in and out of the inner section 150L1 in the direction of the double-headed arrow 156. It is also adapted for rotation about its longitudinal axis in the direction of the double-headed arrow 158. In this embodiment, the control arm is pivotally attached to outer section 150L2 by wrist joint 162 for pivotal movement in the direction of double-headed arrow 164. End section 160 is comprised of axially aligned inner and outer sections 160A and 160B, which outer section 160B is rotatable about its longitudinal axis in the direction of double-headed arrow 166. As in the arrangement described above, sensor means (also referred to as a handle) 168 is installed adjacent to the free end of the control arm for operation of the end effector 170 in the manipulator 142 shown in FIG.

  Referring to FIG. 11, there is shown an end effector 170 comprising a pair of movable jaws attached to a wrist 172 comprising axially aligned links 172A and 172B. The outer link 172B is motor means (not shown) about a longitudinal axis relative to the inner link 172A in the direction of the double-headed arrow 166M in response to rotation of the outer section 160B of the manual controller end section 160 in the direction of the arrow 166. )). The wrist link 172A is pivotally attached to the manipulator forearm (also referred to as the operating arm) 174 to pivot in the direction of the double-headed arrow 164M in response to the pivoting movement of the manual controller end section 160 about the pivot axis 162. Has been. The forearm 174 can move axially in the longitudinal direction in the direction of the double-headed arrow 156M in response to the axial movement of the outer section 150L2 of the control arm 150L in the direction of the double-headed arrow 156. It can also rotate about the longitudinal axis in the direction of the double-headed arrow 158M in response to the rotation of the outer section 150L2 of the control arm 150L in the direction of the double-headed arrow 158. In addition, it can pivot about point 176 in the direction of double-headed arrows 152M and 154M in response to the pivoting movement of control arm 150L in the direction of double-headed arrows 152 and 154, respectively. For biomedical applications such as remote laparoscopic surgical surgery, the pivot point 176 is located substantially at the level of the abdominal wall 178 where the manipulator extends. In FIG. 11, the operating arm 174 is shown extending through a cannula 180 that penetrates the abdominal wall.

  The external operation end of the manipulator is designed to be attached to a support rail (not shown) of a surgical table on which the main body is supported. It includes an end effector drive motor for opening and closing the gripper 170. The wrist drive motor 184 controls the swing movement of the wrist 172 in the direction of the double-headed arrow 164M, and the extension drive motor 186 controls the axial movement of the operating arm 174 in the direction of the double-headed arrow 156M. The forearm pivot control motor and linkage are generally indicated by reference numeral 188 but provide pivoting movement of the operating arm 174 about the pivot point 176 in the direction of arrows 152M and 154M. The pivoting movement around the point 176 is provided by the lateral movement of the external operating end of the manipulator and the pivoting movement of the operating arm 174 simultaneously. The motion is coordinated so that the center of rotation of the forearm is fixed in the air at point 176 at the level of the abdominal wall.

  The controller 140 and manipulator 142 are included in a system such as that shown in FIGS. 7, 8 and 9, which includes a second controller and manipulator for use by the operator with the right hand, and manipulators with a manual controller. For remote operation, include an appropriate type of associated servomechanism (not shown). As shown in FIG. 9, the video camera means in the work place is shown in FIG. 7 together with the display means, but the image of the work space at a position adjacent to the control means operated by the operator with the left and right hands. Adopted to give. By using a manipulator with list binding, additional degrees of freedom are given for its increased operability and usability. However, as described above, the present invention is not limited to a manipulator having a certain degree of freedom.

Referring now to FIGS. 12 and 13, a variation of the present invention is shown, which provides a direct view of a three-dimensional image 240I of a workspace (not shown). 12 and 13, only the operator's workbench is shown, including left and right controllers 242R and 242L and associated left and right manual means 244R and 244L, which may be of the same type as the controller and control arm. The operator's workbench is designed for remote operation of a manipulator, which may be of the type described above. By sharing the visual display means 246 and the electro-optical device 58, the three-dimensional image of the work space is given by the display means and the plane of cross-polarized glasses worn by the operator. To this display means, the left and right video fields from the left and right video cameras viewing the work space are alternately supplied in the manner detailed above. End effector and destination images 248 and 250 are shown in the workspace as viewed from the video camera at the workplace, respectively. This display means 246 is positioned adjacent to the left and right manual means 244R and 244L for direct viewing by the operator. With this arrangement, the end effector and target images are visible simultaneously by the operator along with manual means 244R and 244L. Since the manual means are also visible, the operator is given a connection vision between the end effector means and the manual means so that they appear to be substantially united. Referring now to FIG. The distal end or tip 260 of the insert is shown, which is referred to as “Introduction to a New Project for National” described previously in “Introduction to a New Project for National Research and Development Program (Large Scale Project) in 1991” Substantially the same type as shown in the publication named Research and Development Program (Large-Scale Project) in FY 1991 ") The endoscope may be used in the practice of the present invention. The insertion end of the endoscope includes a pair of spaced apart viewing windows 262R and 262L and an illumination source for viewing and illuminating the viewing workspace. The light received at the window is focused by target lens means (not shown) and sent to the camera at the operating end of the endoscope (not shown) through an optical fiber bundle. The camera output is converted into a three-dimensional image of the workspace, which is located adjacent to manual means (not shown) on the operator's work table. Left and right steerable bodies (eg, catheters) 268R and 268L pass through auxiliary channels (passages) in the endoscope body. This body is designed for extension from the distal end, as shown. End effectors 270R and 270L are provided at the end of the main body, which may be a conventional endoscopic instrument. A force sensor (not shown) is also inserted through the endoscope channel. Steerable bodies (e.g., catheters) include control wires that control the bending of the body and the operation of end effectors suitable for use in the present invention, but are well known. Control motors for operation of the control wires are provided at the operating end of the endoscope, and these motors are described above for operation of the steerable catheter and associated end effector from the remote operator's workbench. Included in type servo mechanism. In another embodiment, the interfacing computer in the servomechanism system remaps the operator's hand movement to the end effector coordinate system and the end effector image is visible adjacent to the manual controller in the manner described above. It is. In this embodiment, the operator has the sensation of extending his hands to place both hands directly on the end effector for control through the endoscope. Different types of endoscopes may be employed in this embodiment of the invention. Provided that it includes one or more auxiliary channels for use in controlling the end effector means and appropriate visual means used to provide a visual representation of the workspace. For example, endoscopes such as a stomach endoscope and a colon endoscope are employed.

  Although the present invention has been described in detail according to the requirements of patent law, various other changes and modifications will be apparent to those skilled in the art. For example, as noted above, the present invention includes the use of tactile feedback that provides a subtle sensation for palpation and for manipulating tissue and instruments. To provide this feedback, a tactile sensor array may be included on the end effector coupled to the tactile sensor stimulation array on the manual control means, which reproduces the tactile sensation on the operator's hand. Various conversion techniques for teleoperator tactile sensing are known, including those of resistance / conduction, semiconductor, piezoelectric capacitance and optoelectronics. Different types of manual control means and manipulators may be employed, including, for example, gimbals, linkages, pulleys, cables, drive belts and bands, gears, optical or electromagnetic position encoders, and rotary and linear motors A very wide range of known mechanisms and electromechanical elements are used. Force feedback to the operator requires human contact with the manual control means. Both a handgrip type manual controller as illustrated and a control brace type manual controller can be fully used in the present invention for force feedback to the operator. Control brace-type manual controllers involve the use of structures with positive sensors attached to the operator's joints to measure joint angles. Force feedback can be applied to each joint. Similarly, fiber optic gloves with variable resistance sensors or fiber optic flex sensors attached to the joints to measure individual finger bending may be used. This type of glove may be provided with force feedback to provide telepresence interaction with real objects. Regardless of the type of manual control means employed, an image of the workspace is generated adjacent to it, giving the operator the feeling that the end effector means and the manual control means are substantially integral. Also, as noted above, many different types of servomechanisms are well known in robot and teleoperator system technology, and the present invention is not limited to any one type. Those including force and torque feedback to the operator are preferred and contribute to the sense of telepresence of operation. In addition, many different means for generating a stereoscopic image of a workspace are known. For example, instead of using two cameras, a single camera may be employed in the image receive path with a switched cross polarization element. In this case, a pair of spaced apart stereoscopic lenses are used to view the workspace from different angles and to give the camera its first and second images. In the configuration of FIG. 9, a laparoscope is shown here, but other types of endoscopes may be used to view the workspace. As noted above, the present invention is not limited to any particular application or use. In the biomedical field, use includes, for example, open surgery including surgery from a remote location, microsurgery, and minimally invasive surgery such as laparoscopic and endoscopic surgery. Also intended for laboratory use, including microscopic manipulation. Industrial uses of the present invention include, for example, hazardous material handling, remote control, microassembly and the like. The military and subsea use of the teleoperator system of the present invention is obvious.

  It is intended that the foregoing and other such changes and modifications fall within the spirit and scope of the invention as defined in the appended claims.

1 is a diagram of a teleoperator device embodying the present invention, including a side view of a work site and a remote control operator control station. FIG. FIG. 2 is an enlarged rear view of the operator's control station taken along line 2-2 of FIG. FIG. 3 is an enlarged rear view of the work site along line 3-3 in FIG. 1. FIG. 2 is a simplified side view similar to FIG. 1 showing the dimensional relationship between elements at a work site and elements at an operator control station. It is a diagram which shows the visual recognition by a miniature virtual eye. FIG. 6 is a diagram illustrating visual recognition by an operator when image magnification is used. FIG. 2 is a diagram similar to the diagram of FIG. 1 but showing a teleoperator system used for telepresence surgery. It is a rear view of the control station of the operator shown by FIG. FIG. 8 is a rear view of the work site shown in FIG. 7. It is a partial side view of a variant of the operator's control station with increased degrees of freedom. It is a partial side view of the modification of the manipulator which increased the degree of freedom. FIG. 4 is a side view of a variant of an operator control station in which display means are arranged for direct operator viewing. FIG. 13 is a side view of a variation of the operator control station shown in FIG. 12. It is a fragmentary view of the insertion part of the endoscope used with the present invention.

22 Work Site 24 Manipulator 26 Object 34 Operation Arm 72 Manual Controller

Claims (12)

A system for enabling a surgeon to control a surgical instrument inserted through a patient incision, wherein the incision is provided with a pivot point;
A manipulator arm for holding a surgical instrument, the manipulator arm having a driven joint and an actuator for rotating the surgical instrument;
A first input device for receiving an input command from a surgeon; and receiving the input command, calculating a motion of the arm based on the input command, giving an output command for operating the driven joint and the actuator, and turning A computer for moving a surgical instrument about a point, the surgical instrument having an elongated shaft, a wrist joint, and an end effector, the shaft having a proximal end adjacent to the arm. The wrist pivotally connects the distal end of the shaft to an end effector, and the computer pivots the forearm of the shaft about the pivot point and pivots the wrist within the patient. Moving the end effector by causing the computer to move;
The system comprising:   The system of claim 1, wherein an end effector has an actuation joint, and the computer actuates the end effector actuation joint. A system for controlling a slave manipulator to perform minimally invasive surgery at a work site from a remote control operator station, the slave manipulator comprising a slave end effector movably coupled to a slave wrist member, the slave manipulator The wrist member is movably coupled to the slave manipulator arm, the system comprising:
A manual master link mechanism including a plurality of master link mechanisms; and a slave manipulator arm moved in response to the movement of the master link mechanism to perform minimally invasive surgery in the patient's body at the work site from a remote control operator station; A servo mechanism configured to couple the master linkage to the slave manipulator arm, slave wrist member, and slave end effector to articulate the wrist member and actuate the slave end effector;
With
The master link mechanism can be operated with one hand to control the movement of the slave manipulator arm, slave wrist member, and slave end effector by operating the master link mechanism; and the master link mechanism is movably connected to the master manipulator arm. A master wrist member coupled to the master manipulator arm; and a master end effector movably coupled to the master wrist member, wherein the servo mechanism moves the slave manipulator arm in response to manual movement of the master manipulator arm. The slave manipulator arm is connected to the master manipulator arm, and the slave wrist member is connected to the master wrist member so that the slave wrist member is articulated according to the manual movement of the master wrist member. The slave end effector is coupled to the master end effector to activate the slave end effector in response to manual movement of the master end effector, and the slave manipulator arm is driven from outside the patient's body;
Said system.   The system according to claim 3, wherein the master link mechanism can be operated with one hand without releasing the master link mechanism.   The system of claim 3, wherein the master manipulator arm is pivotable relative to a pivot point.   The system of claim 3, wherein the master manipulator arm has an axis extending between a proximal portion and a distal portion, the master manipulator arm being rotatable about the axis.   The system of claim 3, wherein the master manipulator arm has an axis extending between a proximal portion and a distal portion, the master manipulator arm being movable along the axis.   4. The system of claim 3, wherein the master wrist member is connected to the master manipulator arm at a pivot joint and is pivotable relative to the pivot joint.   The system of claim 3, wherein the master wrist member has an axis extending between a proximal portion and a distal portion, the master wrist member being rotatable about the axis.   The master wrist member has an axis extending between a proximal portion and a distal portion, the master wrist member being movable relative to the master manipulator arm along the axis. The system described in.   The system of claim 3, wherein the master end effector is coupled to a master wrist member at a pivot joint and is pivotable relative to the pivot joint.   The system of claim 11, wherein the master end effector comprises a pair of open and close jaws that are pivotable relative to the pivot joint.