Nothing Special   »   [go: up one dir, main page]

WO2005074303A1 - Stereoscopic projection-based display for surgical robot - Google Patents

Stereoscopic projection-based display for surgical robot Download PDF

Info

Publication number
WO2005074303A1
WO2005074303A1 PCT/US2005/001964 US2005001964W WO2005074303A1 WO 2005074303 A1 WO2005074303 A1 WO 2005074303A1 US 2005001964 W US2005001964 W US 2005001964W WO 2005074303 A1 WO2005074303 A1 WO 2005074303A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
screen
multiplexer
video input
surgical robot
Prior art date
Application number
PCT/US2005/001964
Other languages
French (fr)
Inventor
Alok Shrivastava
Mani Menon
Original Assignee
Henry Ford Health System
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henry Ford Health System filed Critical Henry Ford Health System
Publication of WO2005074303A1 publication Critical patent/WO2005074303A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30Surgical robots
    • A61B34/35Surgical robots for telesurgery
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/363Image reproducers using image projection screens
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70Manipulators specially adapted for use in surgery
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/332Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/337Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using polarisation multiplexing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/398Synchronisation thereof; Control thereof
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37Surgical systems with images on a monitor during operation
    • A61B2090/371Surgical systems with images on a monitor during operation with simultaneous use of two cameras
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37Surgical systems with images on a monitor during operation
    • A61B2090/372Details of monitor hardware
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36Image-producing devices or illumination devices not otherwise provided for
    • A61B90/361Image-producing devices, e.g. surgical cameras
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37Surgical systems with images on a monitor during operation

Definitions

  • the present invention relates to surgical robots. More specifically, the present invention relates to a video display for -use in conjunction with surgical robots.
  • Minimally invasive medical techniques are aimed at reducing the amount of extraneous tissue that may be damaged during diagnostic or surgical procedures, thereby reducing patient recovery time, discomfort, and deleterious side effects. Many surgeries are performed each year in the United States. A significant amount of these surgeries can potentially be performed in a minimally invasive manner. However, only a relatively small percentage of surgeries currently use minimally invasive techniques due to limitations of minimally invasive surgical instruments and techniques currently used, the difficulty experienced in performing surgeries, and the difficulty in training surgeons using such traditional instruments and techniques. Advances in minimally invasive surgical technology could dramatically increase the number of surgeries performed in a minimally invasive manner. The average length of a hospital stay for a standard surgery is significantly longer than the average length for the equivalent surgery performed in a minimally invasive surgical manner.
  • MIS minimally ' invasive surgical
  • existing MIS instruments deny the surgeon the flexibility of tool placement found in open surgery. Difficulty is experienced in approaching the surgical site with straight instruments through the incisions on the abdominal wall.
  • the length and construction of many endoscopic instruments reduces the surgeon's ability to feel forces exerted by tissues and organs on the end effector of the associated instrument.
  • coordination of the movement of the end effector of the instrument as viewed in the image on the television monitor with actual end effector movement is particularly difficult, since the movement as perceived in the image normally does not correspond intuitively with the actual end effector movement. Accordingly, lack of intuitive response to surgical instrument movement input is often experienced. Such a lack of intuitiveness, dexterity and sensitivity of endoscopic tools has been found to be an impediment to the expansion of the use and training of minimally invasive surgery. Additionally, the 2D image viewed by the assisting surgeon limits the effectiveness of training.
  • Telesurgery is a general term for surgical systems where the surgeon uses some form of remote control, e.g., a servomechanism, or the like, to manipulate surgical instrument movements, rather than directly holding and moving the tools by hand.
  • the surgeon is typically provided with an image of the surgical site on a visual display at a location remote from the patient.
  • the surgeon can typically perform the surgical procedure at the location remote from the patient whilst viewing the end effector movement during the surgical procedure on the visual display. While viewing typically a three-dimensional image of the surgical site on the visual display, the surgeon performs the surgical procedures on the patient by manipulating master control devices at the remote location, which master control .devices control motion of the remotely controlled instruments.
  • such a telesurgery system can be provided with at least two master control devices (one for each of the surgeon's hands), which are normally operatively associated with two robotic arms on each of which a surgical instrument is' mounted. Operative communication between master control devices and associated robotic arm and instrument assemblies is typically achieved through a control system.
  • the control system typically includes at least one processor which relays input commands from the master control devices to the associated robotic arm and instrument assemblies and from the arm and instrument assemblies to the associated master control devices in the case of, e.g., force feedback, or the like.
  • the currently available systems for telesurgery typically use special endoscopes to provide a three dimensional (3D) or stereoscopic visualization to the operating surgeon through an arrangement of television monitor and mirror system. These systems rely on traditional 2D imagery for the assistants and other personnel observing the surgery. This limitation made assisting, training and mentoring difficult due to the different visual perspective between the surgeon (in 3D) and the others (in 2D).
  • Intuitive surgical of Sunnyvale, California provides a system under trademark of daVinci surgical system that allows a surgeon to perform minimally invasive procedure like coronary bypass grafting and robotic radical prostatectomy. The procedures are performed with instruments that are inserted through small incisions in the patient's chest or abdomen. The instruments are controlled by robotic arms.
  • Movement of the robotic arms and actuation of instrument end effectors are controlled by the surgeon through a pair of masters and foot pedal that are coupled to an electronic controller.
  • the master arms also control movement of an endoscopic camera.
  • the surgeon views the internal view on a display system made of two video monitors and mirror system.
  • the masters and the display system are typically integrated into a console that is operated by the surgeon to control the various robotic arms and medical instruments. Utilizing a robotic system to perform surgery requires a certain amount of training. It would be desirable to provide a system that would either allow a second surgeon to assist another surgeon in controlling a robotic medical system or to train other surgeons.
  • the second surgeon could both teach and assist a surgeon learning to perform a medical procedure with a daVinci system.
  • the primary problem with achieving this goal is that the existing equipment does not provide another surgeon with a three dimensional view. Instead the second surgeon is able to two- dimensionally view the surgery, but is not able to three-dimensionally view the surgery.
  • U.S. Patent No. 5,217,003 to Wilk discloses a surgical system that allows a surgeon to remotely operate robotically controlled medical instruments through a telecommunication link.
  • the system allows one surgeon to operate the robotic arms at a given time. It does not disclose or contemplate a system that allows two different surgeons to operate or three-dimensionally view the surgery.
  • U.S. Patent No. 5,149,270 to McKeown discloses an apparatus for practicing surgical endoscopic procedures. Simulators incorporate features to simulate visual and manipulation surgical conditions for training surgeons in surgical procedures such as laparoscopy and hysteroscopy.
  • the apparatus has a cavity in which an object simulating a human organ is mounted for performing the practice procedure. The cavity is closeable to outside view or access, thus forcing the individual to use and manipulate the instruments under conditions that mimic real life operating and diagnostic conditions.
  • U.S. Patent No. 5,273,038 to Beavin discloses a computer system receiving two dimensional slice data of a heart or other organ to be simulated in three dimensions.
  • the three dimensional image data and chemical composition data of the heart or other organ are stored in the computer memory.
  • a Voxel View or three-dimensional volume- rendering program forms images of the organ to be studied.
  • Diagnostic data obtained from a patient with electrical measurement signals including an electro-cardiogram, electro-myogram, electro-encephalogram, and other diagnostic measured electrical signals obtained from a patient are fed into the system and are placed in computer memory.
  • Physiological data of the patient, including the strength, weakness, and other parameters of the organ is also considered diagnostic data and is supplied into the system.
  • the data may be fed in black and white or in color to a device that shows the organ for visualization, operation simulation, or training.
  • the Beavin patent does not disclose a system that enables a second surgeon to view a surgical procedure three-dimensionally.
  • U.S. -Patent No, 5,385,474 to Brindle discloses a method for simulating anesthesiology and operating room conditions including the following six steps: displaying initial patient simulated vital sign information from a memory to signify an initial patient condition; randomly modifying the displayed patient vital, sign information according to a script matrix in a manner analogous to that in which a patient's vital signs would be effectedin the operating room by drugs or other external effects, thereby indicating a deteriorating condition; displaying user options; evaluating the timeliness and appropriateness of user input selections from the options in response to the changes in* patient vital sign information to improve its initial state or deteriorate to a critical state in accordance with the successive script blocks, in the script matrix depending upon the user's response and timeliness. While all of these conditions are beneficial for training purposes, the system does not enable the trainee to simultaneously, three- dimensionally view a surgical procedure that the trainee is not performing.
  • U.S. Patent No. 5,261 ,404 to Mick et al. discloses a three- dimensional mammal anatomy imaging system that provides images of the internal anatomy of a mammal. However, the system does not enable a second surgeon, or trainee, can experience the same three-dimensional view as that of the surgeon performing the surgery.
  • U.S. Patent No. 4,331 ,422 10, 1994 to Loftin et al. discloses a training system for use in a wide variety of training tasks and environments. Artificial intelligence is used to provide computer-aided training.
  • a robotic system for broadcasting or telecasting a three-dimensional image from a surgical robot to a screen including at least one video input operatively connected to a surgical robot, at least one projector for receiving and projecting the 3D image from the video input, and a screen for displaying the 3D image from the projector.
  • a method of training surgeons to use a surgical robot by performing a surgical procedure using a surgical robot, transmitting the 3D images seen by the surgeon during surgery to a screen, and viewing the 3D images, thereby training surgeons regarding surgery using the 3D imagery is provided.
  • a method of transmitting a three dimensional image from a surgical robot to a screen by obtaining a 3D image from a surgical robot, transmitting the 3D image to projector, and displaying the 3D image on a screen is also provided.
  • Figure 1 is a diagram of a rear projection system of the present invention
  • Figure 2 is a diagram of the projection system, including the screen, of the present invention
  • ⁇ and Figure 3 is a photograph of an operating room containing the system of the present invention.
  • the present invention provides a device that creates a three-dimensional (3D) image that can be viewed by individuals in the operating room or telecast to a geologically .different location in 3D. Additionally, the system of the present invention can be used for training individuals. More specifically, the present invention has two components a local viewing mode and a recording/telecasting mode. The device of the present invention can be used in connection with any surgical robot or other telesurgical device.
  • the local viewing mode is a local live display.
  • the display is a device that creates a 3D image viewable by individuals that are not able to view the surgeon's console of the telesurgical system but are present in the geographically same location of the surgical robot.
  • the recording/telecasting mode of the present invention enables individuals at geologically different location to view, in 3D, a recorded display or live display.
  • individuals at a remote location can view a surgery as though they were present at the site of the procedure.
  • the term "minimally invasive surgery" is intended to include endoscopy.
  • One of the more common forms of endoscopy is laparoscopy, which is minimally invasive inspection or surgery within the abdominal cavity.
  • Other endoscopic techniques include, but are not limited to, arthroscopy, retroperitoneoscopy, pelviscopy, nephroscopy, cystoscopy, cistemoscopy, sinoscopy, hysteroscopy, urethroscopy, and other similar procedures.
  • the system of the present invention can be used for local-live 3D display or for telecasting.
  • the system includes video inputs, projectors, and a screen.
  • the video inputs are connected to a surgical robot.
  • the robot includes video streams that are displayed for the surgeon.
  • the device of the present invention in connected to the video streams and channels the video stream to a projector.
  • the projector then displays the image that is seen by the surgeon to a screen, thus enabling individuals in the room to view the exact same image as
  • the vided inputs that are utilized can be any video input devices known to those of skill in the art.
  • the inputs can include, but are not limited to, S-video cables, BNC-RCA composite cables, KVM switches, component video inputs, coaxial cables, twisted pair cables, and fiber optic cables.
  • the projectors of the present invention are preferably projectors that are compatible with the video stream data.
  • An example of such a projector includes, but is not limited to, a DLP or LCD projector.
  • Examples, of such projectors include, but are not limited to, BenQ PB2220, HP mp3130, InFocus LP 70+, Mitsubishi XD50U Mini-Mits, NEC LT10, NEC LT170, Philips bCool XG1 , Olympus VP1 , Toshiba TLP-S10U and TLP-S41 U, Sharp Notevision PG-B10S, NEC VT465, Dell 3200MP, Epson PowerLite 73c, HP vp6120, Mitsubishi XD300U, NEC MT1065, NEC VT46, NEC VT460, Optoma EzPro 737, Panasonic PT-LC76U & PT-LC80U, PLUS V3- 131 , Sharp XR-1X, Toshiba TDP-P5-US, and Toshiba TLP-T501.
  • the system of the present invention utilizes two projectors.
  • the projectors are preferably mounted above one another on a frame with each projector including a polarizing filter.
  • a dual projection system is preferably used, as this enables the 3D image to be projected.
  • the projectors can project the 3D image without requiring the polarization filters or use an alternative method for creating the 3D image.
  • the screen of the present invention can be any screen capable to displaying the images.
  • the screen is preferably capable of maintaining the integrity of the polarization of the image. Individuals of skill in the art are familiar with such screens. Examples of such screens are a Silver matte screen from Dalite or Film screen 150 from Stewart.
  • the robot, or surgical device, for use in conjunction with the device of the present invention can be any surgical system with capability of binocular vision can be connected to the device of the present invention.
  • Examples of such robots include, but are not limited to, the da VinciTM surgical robot, EndoVia robot, surgical robot hand, computer assisted surgery, and other known surgical or surgery-related robots known to be useful in performing minimally invasive surgery.
  • the system of the present invention improves telesurgery display systems by providing similar 3D visual perspectives to the operating surgeon on the computer console and others in the operating room. In this manner the surgical procedure can typically be performed, learned and taught with greater confidence, safety, efficacy and in some cases greater accuracy.
  • the system of the present invention also provides means to telecast the telesurgical procedure live in 3D, to a geologically different location or to record, edit, and display the procedure later in 3D.
  • the system improves the training of surgeons as they can watch the educational videos of the surgery performed by the telesurgical system in 3D.
  • the individuals that view the projected images preferably wear special eyewear in order to visualize the projection to get a 3D perception.
  • the glasses can be either wired or wireless.
  • 3D eyeware include, but are not limited to, Polarized 3D glasses, Pulfrich 3D glasses, and anaglyph 3D glasses.
  • the glasses can be clip-on lenses or can be self-contained glasses. Preferably, the glasses are wireless as this makes use significantly easier.
  • the glasses can be made by any known maker of 3D eyeware. Some well-known companies include, but are not limited to, Razor3D, Rainbow Symphony, Inc., eDimensional, and l-glasses. Alternatively, eyeglasses are not required when the projectors do not utilize the polarizing filters.
  • the system of the present invention can be used for recording or telecasting of the 3D images from the surgical robot.
  • the two video streams from surgical systems camera units can be synchronized using commercially available frame synchronizer.
  • a frame synchronizer synchronizes outputs from multiple video sources so that they are in similar phase of their frame cycle.
  • the FA-145 is a frame synchronizer and time base corrector in a single system.
  • the FA-145 includes 10-bit 4:2:2 digital component processing and a Y/C decoder.
  • the FA-145 also supports SD-SDI video or analog composite video signals in and two-way SD-SDI video output signals.
  • the analog composite output connections can also be used for signal monitoring.
  • a full frame of memory is provided to enable reliable and frame accurate synchronization, and a variety of genlock functions are provided. Individuals of skill in the art are familiar with other such synchronizers that can be used in the system of the present invention.
  • the synchronized signals thus obtained are multiplexed with a video multiplexer.
  • the multiplexers combine two genlocked video streams and output a sigle multiplexed 3D video stream.
  • Such multiplexers are commercially available like Mux1/Mux2/Mux3 from VrRex or Model 100/200/ 400/500 from 3DTV.
  • the multiplexed signal can be recorded and edited on videotape or a digital nonlinear editing system.
  • Non-linear editing (NLE) is editing using random-access video storage, so that there is no wait for the tape to shuttle to see a scene at the other end of the reel.
  • Examples of such editing software include, but are not limited to, VideoRaid RT3X, VideoRaid RT3, Premiere Pro 1.5, XSreve Raid, XPress DV v3.5, Media Studio Pro 7, and DV Storm made by companies such as Avid (Media Composers of various flavors, models, qualities, and capabilities), Accom (formerly Scitex, formerly Immix) with its "sphere” products (descended from the VideoCube and TurboCube), Quantel (Harry, Henry, Harriet, EditBox, iQ, etc.), and Medial 00 (844/X, iFinish, etc.).
  • Avid Media Composers of various flavors, models, qualities, and capabilities
  • Accom formerly Scitex, formerly Immix
  • Quantel Hardry, Henry, Harriet, EditBox, iQ, etc.
  • Medial 00 844/X, iFinish, etc.
  • the signal can also be transmitted by microwave, cables, satellite, infrared beams, telephone (ISDN and PSDN), fibreoptic channels or any other standard means used by the tele-communication companies for transmitting video signals.
  • the transmitted or recorded multiplexed signal is converted into two separate signals one each for right and left eye by commercially available decoders like SpaceBox 1/2 from 3DTV or XPO stereo or VR Video converter from VRex and displayed through the projection and screen system described above.
  • the system of the present invention can also be incorporated as part of an operating room.
  • an operating room can include a 3D display wall including the system of the present invention.
  • the wall enables a larger image to be displayed.
  • Diagrams of such operating rooms are shown in Figures 1 and 2.
  • Figure 3 is a photograph of an actual operating room.
  • the present invention provides a projection based display system for the surgical robots.
  • the surgical robots have two optical channels one for the right eye and another for the left eye.
  • the video signals from the robot are fed into a two DLP or LCD projectors.
  • the output from the projectors polarized 1 and is projected on special silver or rear projection screens.
  • the projected image is viewed through special polarized glasses to give a 3D perception.
  • the two video signals can be field multiplexed and converted into a single signal.
  • the multiplexed signal can be recorded, edited or transmitted on standard video equipments.
  • the multiplexed signal can be de-multiplexed into two separate signals for each eye to be displayed by the projection system described above.
  • the robot has a 3-D display for the surgeon and 2D CRT display for assistants and other personnel in the operating room.
  • the manufacturer of the robot does not provide a solution for 3D display for the assistants and other trainees in the operating room. This brings a disparity in the visual perspectives of the operating fields of the surgeon, his assistants and the trainees.
  • Currently available surgical robots do not have provision for recording, editing and transmitting their 3D displays. These limitations have made assisting, training and mentoring difficult.
  • Surgical robots have a telescope with two separate optical channels one for right eyes and another for the left eye. They provide two streams of video signals. The two video streams are then synchronized and displayed into the surgeon console in 3D by two cathode ray monitors and an arrangement of minors. Any surgical robot providing two separate video outputs for each eye can be used. The daVinciTM surgical system is preferred. The video streams were tapped from the synchronizers in the vision cart. S-Video cables and BNC- RCA composite cables were used for connection between the vision cart and the display system.
  • Any projector capable of projecting an electronic video signal can be used.
  • Two DLP projectors were mounted one above another on a frame.
  • the video signals from the da VinciTM vision cart are fed in the projectors through S-Video or RCA composite inputs.
  • S-Video or RCA composite inputs can also be used.
  • Each of the two projectors is designated to project image of either right or left eye.
  • two polarizing filters are mounted in front of the lenses of the two projectors.
  • CRT and DLP projectors can use linear or circular polarizers.
  • the LCD projectors can use only circular polarizers.
  • Linear polarizers are arranged in a 45° and 135° orientation.
  • Circular polarizers are arranged in a clockwise and counterclockwise orientation.
  • the polarized images are projected on a special screen. This type of screen surface maintains the integrity of the polarization of the image.
  • the preferred screens are silver matte (Dalite), Datex 1.8(Dalite) and Flimscreen 150(Stewart).
  • a rear projection system with reflectors assembly to decease the projection throw distance can also be used.
  • Special eyewear with 45° and 135° linear polarizing filters is used to visualize the projection to get a 3D perception. If the projectors have circular polarizers then eyeware used has circular polarizers arranged in a clockwise and counter-clockwise orientation. The special eyeware permits the isolation of image seen by each eye.
  • the two video signals for each eye are multiplexed into a single stream.
  • Any commercially available field multiplexer can be used.
  • the Mux-3 from Vrex Inc is used.
  • the multiplexed signal is then recorded on videotape or as DV format on a hard drive.
  • the recording thus obtained can be edited on commercially available non-linear editing software using native DV codecs.
  • a codec is a compresser/decompresser, a bit of software or hardware that takes raw video and compresses it, and can take the compressed video and decompress it back to raw video.
  • Codecs exist for all kinds of compressed video, including DV, motion- JPEG, MPEG, Indeo, Cinepak, Sorensen, wavelet, fractal, RealVideo, vXtreme, MPEG2 codec and many others.
  • the codecs Preferably, have a bit rate of 4.5MBPS and above.
  • Final Cut Pro 4 or Ulead Video Studio 4 is preferably used for the editing.
  • the multiplexed signal can be transmitted to geographically different locations using microwave, Radio frequency, satellite, fiber-optic channel or any other means of transmitting video signal in analogue or uncompressed digital form.
  • a Video Converter box from Vrex Inc is preferably used. The two-separated videos are displayed through a setup similar to that described above.
  • the present invention will have immediate effect in easing the training and mentorship for the future robotic surgeons.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Surgery (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Robotics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

A robotic system for broadcasting or telecasting a three-dimensional image from a surgical robot to a screen including at least one video input operatively connected to a surgical robot, at least one projector for receiving and projecting the 3D image from the video input, and a screen for displaying the 3D image from the projector. A method of training surgeons to use a surgical robot by performing a surgical procedure using a surgical robot, transmitting the 3D images seen by the surgeon during surgery to a screen, and viewing the 3D images, thereby training surgeons regarding surgery using the 3D imagery. A method of transmitting a three dimensional image from a surgical robot to a screen by obtaining a 3D image from a surgical robot, transmitting the 3D image to projector, and displaying the 3D image on a screen.

Description

STEREOSCOPIC PROJECTION-BASED DISPLAY FOR SURGICAL ROBOT
BACKGROUND OF THE INVENTION
1. TECHNICAL FIELD
Generally, the present invention relates to surgical robots. More specifically, the present invention relates to a video display for -use in conjunction with surgical robots.
2. BACKGROUND ART
Minimally invasive medical techniques are aimed at reducing the amount of extraneous tissue that may be damaged during diagnostic or surgical procedures, thereby reducing patient recovery time, discomfort, and deleterious side effects. Many surgeries are performed each year in the United States. A significant amount of these surgeries can potentially be performed in a minimally invasive manner. However, only a relatively small percentage of surgeries currently use minimally invasive techniques due to limitations of minimally invasive surgical instruments and techniques currently used, the difficulty experienced in performing surgeries, and the difficulty in training surgeons using such traditional instruments and techniques. Advances in minimally invasive surgical technology could dramatically increase the number of surgeries performed in a minimally invasive manner. The average length of a hospital stay for a standard surgery is significantly longer than the average length for the equivalent surgery performed in a minimally invasive surgical manner. Thus, expansion in the use of minimally invasive techniques could save millions of hospital days, and consequently millions of dollars annually, in hospital residency costs alone. Patient recovery times, patient discomfort, surgical" side effects, and time away from work can also be reduced by expanding the use of minimally invasive surgery.
There are many disadvantages relating to such traditional minimally' invasive surgical (MIS) techniques. For example, existing MIS instruments deny the surgeon the flexibility of tool placement found in open surgery. Difficulty is experienced in approaching the surgical site with straight instruments through the incisions on the abdominal wall. The length and construction of many endoscopic instruments reduces the surgeon's ability to feel forces exerted by tissues and organs on the end effector of the associated instrument. Furthermore, coordination of the movement of the end effector of the instrument as viewed in the image on the television monitor with actual end effector movement is particularly difficult, since the movement as perceived in the image normally does not correspond intuitively with the actual end effector movement. Accordingly, lack of intuitive response to surgical instrument movement input is often experienced. Such a lack of intuitiveness, dexterity and sensitivity of endoscopic tools has been found to be an impediment to the expansion of the use and training of minimally invasive surgery. Additionally, the 2D image viewed by the assisting surgeon limits the effectiveness of training.
Minimally invasive telesurgical systems for use in surgery have been and are still being developed to increase a surgeon's dexterity as well as to permit a surgeon to operate on a patient in an intuitive manner. Telesurgery is a general term for surgical systems where the surgeon uses some form of remote control, e.g., a servomechanism, or the like, to manipulate surgical instrument movements, rather than directly holding and moving the tools by hand. In such a telesurgery system, the surgeon is typically provided with an image of the surgical site on a visual display at a location remote from the patient. The surgeon can typically perform the surgical procedure at the location remote from the patient whilst viewing the end effector movement during the surgical procedure on the visual display. While viewing typically a three-dimensional image of the surgical site on the visual display, the surgeon performs the surgical procedures on the patient by manipulating master control devices at the remote location, which master control .devices control motion of the remotely controlled instruments.
Typically, such a telesurgery system can be provided with at least two master control devices (one for each of the surgeon's hands), which are normally operatively associated with two robotic arms on each of which a surgical instrument is' mounted. Operative communication between master control devices and associated robotic arm and instrument assemblies is typically achieved through a control system. The control system typically includes at least one processor which relays input commands from the master control devices to the associated robotic arm and instrument assemblies and from the arm and instrument assemblies to the associated master control devices in the case of, e.g., force feedback, or the like.
The currently available systems for telesurgery typically use special endoscopes to provide a three dimensional (3D) or stereoscopic visualization to the operating surgeon through an arrangement of television monitor and mirror system. These systems rely on traditional 2D imagery for the assistants and other personnel observing the surgery. This limitation made assisting, training and mentoring difficult due to the different visual perspective between the surgeon (in 3D) and the others (in 2D). Intuitive surgical of Sunnyvale, California provides a system under trademark of daVinci surgical system that allows a surgeon to perform minimally invasive procedure like coronary bypass grafting and robotic radical prostatectomy. The procedures are performed with instruments that are inserted through small incisions in the patient's chest or abdomen. The instruments are controlled by robotic arms. Movement of the robotic arms and actuation of instrument end effectors are controlled by the surgeon through a pair of masters and foot pedal that are coupled to an electronic controller. The master arms also control movement of an endoscopic camera. The surgeon views the internal view on a display system made of two video monitors and mirror system. The masters and the display system are typically integrated into a console that is operated by the surgeon to control the various robotic arms and medical instruments. Utilizing a robotic system to perform surgery requires a certain amount of training. It would be desirable to provide a system that would either allow a second surgeon to assist another surgeon in controlling a robotic medical system or to train other surgeons. The second surgeon could both teach and assist a surgeon learning to perform a medical procedure with a daVinci system. The primary problem with achieving this goal is that the existing equipment does not provide another surgeon with a three dimensional view. Instead the second surgeon is able to two- dimensionally view the surgery, but is not able to three-dimensionally view the surgery.
Other training devices and surgical devices disclosed in the prior art are described below. The primary problem with the disclosed devices is the lack of 3D viewing capabilities for the surgeon assisting or being trained.
For example, U.S. Patent No. 5,217,003 to Wilk discloses a surgical system that allows a surgeon to remotely operate robotically controlled medical instruments through a telecommunication link. The system allows one surgeon to operate the robotic arms at a given time. It does not disclose or contemplate a system that allows two different surgeons to operate or three-dimensionally view the surgery.
U.S. Patent No. 5,149,270 to McKeown discloses an apparatus for practicing surgical endoscopic procedures. Simulators incorporate features to simulate visual and manipulation surgical conditions for training surgeons in surgical procedures such as laparoscopy and hysteroscopy. The apparatus has a cavity in which an object simulating a human organ is mounted for performing the practice procedure. The cavity is closeable to outside view or access, thus forcing the individual to use and manipulate the instruments under conditions that mimic real life operating and diagnostic conditions. There is no disclosure of a system enabling multiple individuals to three-dimensionally view an operation from the surgeon's perspective.
U.S. Patent No. 5,273,038 to Beavin discloses a computer system receiving two dimensional slice data of a heart or other organ to be simulated in three dimensions. The three dimensional image data and chemical composition data of the heart or other organ are stored in the computer memory. Then a Voxel View or three-dimensional volume- rendering program forms images of the organ to be studied. Diagnostic data obtained from a patient with electrical measurement signals including an electro-cardiogram, electro-myogram, electro-encephalogram, and other diagnostic measured electrical signals obtained from a patient are fed into the system and are placed in computer memory. Physiological data of the patient, including the strength, weakness, and other parameters of the organ is also considered diagnostic data and is supplied into the system. The data may be fed in black and white or in color to a device that shows the organ for visualization, operation simulation, or training. The Beavin patent does not disclose a system that enables a second surgeon to view a surgical procedure three-dimensionally.
U.S. -Patent No, 5,385,474 to Brindle discloses a method for simulating anesthesiology and operating room conditions including the following six steps: displaying initial patient simulated vital sign information from a memory to signify an initial patient condition; randomly modifying the displayed patient vital, sign information according to a script matrix in a manner analogous to that in which a patient's vital signs would be effectedin the operating room by drugs or other external effects, thereby indicating a deteriorating condition; displaying user options; evaluating the timeliness and appropriateness of user input selections from the options in response to the changes in* patient vital sign information to improve its initial state or deteriorate to a critical state in accordance with the successive script blocks, in the script matrix depending upon the user's response and timeliness. While all of these conditions are beneficial for training purposes, the system does not enable the trainee to simultaneously, three- dimensionally view a surgical procedure that the trainee is not performing.
U.S. Patent No. 5,261 ,404 to Mick et al. discloses a three- dimensional mammal anatomy imaging system that provides images of the internal anatomy of a mammal. However, the system does not enable a second surgeon, or trainee, can experience the same three-dimensional view as that of the surgeon performing the surgery. U.S. Patent No. 4,331 ,422 10, 1994 to Loftin et al. discloses a training system for use in a wide variety of training tasks and environments. Artificial intelligence is used to provide computer-aided training.
U.S. Patent No. 5,130,794 to Ritchey discloses a panoramic image based virtual reality display system. However, there is still no disclosure for a system that enables a second surgeon, or trainee, can experience the same three-dimensional view as that of the surgeon performing the surgery.
It would therefore be useful to develop a surgical and/or training system that enables a second surgeon, or trainee, can experience the same three-dimensional view as that of the surgeon performing the surgery.
SUMMARY OF THE INVENTION According to the present invention, there is provided a robotic system for broadcasting or telecasting a three-dimensional image from a surgical robot to a screen including at least one video input operatively connected to a surgical robot, at least one projector for receiving and projecting the 3D image from the video input, and a screen for displaying the 3D image from the projector. A method of training surgeons to use a surgical robot by performing a surgical procedure using a surgical robot, transmitting the 3D images seen by the surgeon during surgery to a screen, and viewing the 3D images, thereby training surgeons regarding surgery using the 3D imagery is provided. A method of transmitting a three dimensional image from a surgical robot to a screen by obtaining a 3D image from a surgical robot, transmitting the 3D image to projector, and displaying the 3D image on a screen is also provided.
BRIEF DESCRIPTION OF THE DRAWINGS
Other advantages of the present invention are readily appreciated as the same becomes better understood by reference to the following detailed description, when considered in connection with the accompanying drawings wherein:
Figure 1 is a diagram of a rear projection system of the present invention; Figure 2 is a diagram of the projection system, including the screen, of the present invention ;ι and Figure 3 is a photograph of an operating room containing the system of the present invention.
DESCRIPTION OF THE INVENTION Generally, the present invention provides a device that creates a three-dimensional (3D) image that can be viewed by individuals in the operating room or telecast to a geologically .different location in 3D. Additionally, the system of the present invention can be used for training individuals. More specifically, the present invention has two components a local viewing mode and a recording/telecasting mode. The device of the present invention can be used in connection with any surgical robot or other telesurgical device.
The local viewing mode is a local live display. The display is a device that creates a 3D image viewable by individuals that are not able to view the surgeon's console of the telesurgical system but are present in the geographically same location of the surgical robot.
The recording/telecasting mode of the present invention enables individuals at geologically different location to view, in 3D, a recorded display or live display. Thus, individuals at a remote location can view a surgery as though they were present at the site of the procedure.
The term "minimally invasive surgery" is intended to include endoscopy. One of the more common forms of endoscopy is laparoscopy, which is minimally invasive inspection or surgery within the abdominal cavity. Other endoscopic techniques include, but are not limited to, arthroscopy, retroperitoneoscopy, pelviscopy, nephroscopy, cystoscopy, cistemoscopy, sinoscopy, hysteroscopy, urethroscopy, and other similar procedures. The system of the present invention can be used for local-live 3D display or for telecasting. The system includes video inputs, projectors, and a screen. The video inputs are connected to a surgical robot. The robot includes video streams that are displayed for the surgeon. The device of the present invention in connected to the video streams and channels the video stream to a projector. The projector then displays the image that is seen by the surgeon to a screen, thus enabling individuals in the room to view the exact same image as that seen by the surgeon.
The vided inputs that are utilized can be any video input devices known to those of skill in the art. For example, the inputs can include, but are not limited to, S-video cables, BNC-RCA composite cables, KVM switches, component video inputs, coaxial cables, twisted pair cables, and fiber optic cables. The projectors of the present invention are preferably projectors that are compatible with the video stream data. An example of such a projector includes, but is not limited to, a DLP or LCD projector. Examples, of such projectors include, but are not limited to, BenQ PB2220, HP mp3130, InFocus LP 70+, Mitsubishi XD50U Mini-Mits, NEC LT10, NEC LT170, Philips bCool XG1 , Olympus VP1 , Toshiba TLP-S10U and TLP-S41 U, Sharp Notevision PG-B10S, NEC VT465, Dell 3200MP, Epson PowerLite 73c, HP vp6120, Mitsubishi XD300U, NEC MT1065, NEC VT46, NEC VT460, Optoma EzPro 737, Panasonic PT-LC76U & PT-LC80U, PLUS V3- 131 , Sharp XR-1X, Toshiba TDP-P5-US, and Toshiba TLP-T501. Preferably, the system of the present invention utilizes two projectors. The projectors are preferably mounted above one another on a frame with each projector including a polarizing filter. A dual projection system is preferably used, as this enables the 3D image to be projected. Alternatively, the projectors can project the 3D image without requiring the polarization filters or use an alternative method for creating the 3D image. The screen of the present invention can be any screen capable to displaying the images. The screen is preferably capable of maintaining the integrity of the polarization of the image. Individuals of skill in the art are familiar with such screens. Examples of such screens are a Silver matte screen from Dalite or Film screen 150 from Stewart.
The robot, or surgical device, for use in conjunction with the device of the present invention can be any surgical system with capability of binocular vision can be connected to the device of the present invention. Examples of such robots include, but are not limited to, the da Vinci™ surgical robot, EndoVia robot, surgical robot hand, computer assisted surgery, and other known surgical or surgery-related robots known to be useful in performing minimally invasive surgery.
The system of the present invention improves telesurgery display systems by providing similar 3D visual perspectives to the operating surgeon on the computer console and others in the operating room. In this manner the surgical procedure can typically be performed, learned and taught with greater confidence, safety, efficacy and in some cases greater accuracy. The system of the present invention also provides means to telecast the telesurgical procedure live in 3D, to a geologically different location or to record, edit, and display the procedure later in 3D. The system improves the training of surgeons as they can watch the educational videos of the surgery performed by the telesurgical system in 3D.
The individuals that view the projected images preferably wear special eyewear in order to visualize the projection to get a 3D perception. The glasses can be either wired or wireless. Examples of 3D eyeware include, but are not limited to, Polarized 3D glasses, Pulfrich 3D glasses, and anaglyph 3D glasses. The glasses can be clip-on lenses or can be self-contained glasses. Preferably, the glasses are wireless as this makes use significantly easier. The glasses can be made by any known maker of 3D eyeware. Some well-known companies include, but are not limited to, Razor3D, Rainbow Symphony, Inc., eDimensional, and l-glasses. Alternatively, eyeglasses are not required when the projectors do not utilize the polarizing filters.
The system of the present invention can be used for recording or telecasting of the 3D images from the surgical robot. The two video streams from surgical systems camera units can be synchronized using commercially available frame synchronizer. A frame synchronizer synchronizes outputs from multiple video sources so that they are in similar phase of their frame cycle.
The FA-145 is a frame synchronizer and time base corrector in a single system. The FA-145 includes 10-bit 4:2:2 digital component processing and a Y/C decoder. The FA-145 also supports SD-SDI video or analog composite video signals in and two-way SD-SDI video output signals. The analog composite output connections can also be used for signal monitoring. A full frame of memory is provided to enable reliable and frame accurate synchronization, and a variety of genlock functions are provided. Individuals of skill in the art are familiar with other such synchronizers that can be used in the system of the present invention.
The synchronized signals thus obtained are multiplexed with a video multiplexer. The multiplexers combine two genlocked video streams and output a sigle multiplexed 3D video stream. Such multiplexers are commercially available like Mux1/Mux2/Mux3 from VrRex or Model 100/200/ 400/500 from 3DTV. The multiplexed signal can be recorded and edited on videotape or a digital nonlinear editing system. Non-linear editing (NLE) is editing using random-access video storage, so that there is no wait for the tape to shuttle to see a scene at the other end of the reel. Examples of such editing software include, but are not limited to, VideoRaid RT3X, VideoRaid RT3, Premiere Pro 1.5, XSreve Raid, XPress DV v3.5, Media Studio Pro 7, and DV Storm made by companies such as Avid (Media Composers of various flavors, models, qualities, and capabilities), Accom (formerly Scitex, formerly Immix) with its "sphere" products (descended from the VideoCube and TurboCube), Quantel (Harry, Henry, Harriet, EditBox, iQ, etc.), and Medial 00 (844/X, iFinish, etc.).
The signal can also be transmitted by microwave, cables, satellite, infrared beams, telephone (ISDN and PSDN), fibreoptic channels or any other standard means used by the tele-communication companies for transmitting video signals. The transmitted or recorded multiplexed signal is converted into two separate signals one each for right and left eye by commercially available decoders like SpaceBox 1/2 from 3DTV or XPO stereo or VR Video converter from VRex and displayed through the projection and screen system described above.
The system of the present invention can also be incorporated as part of an operating room. For example, such an operating room can include a 3D display wall including the system of the present invention. The wall enables a larger image to be displayed. Diagrams of such operating rooms are shown in Figures 1 and 2. Figure 3 is a photograph of an actual operating room. The present invention provides a projection based display system for the surgical robots. The surgical robots have two optical channels one for the right eye and another for the left eye. The video signals from the robot are fed into a two DLP or LCD projectors. The output from the projectors polarized1 and is projected on special silver or rear projection screens. The projected image is viewed through special polarized glasses to give a 3D perception. The two video signals can be field multiplexed and converted into a single signal. The multiplexed signal can be recorded, edited or transmitted on standard video equipments. The multiplexed signal can be de-multiplexed into two separate signals for each eye to be displayed by the projection system described above. The robot has a 3-D display for the surgeon and 2D CRT display for assistants and other personnel in the operating room. The manufacturer of the robot does not provide a solution for 3D display for the assistants and other trainees in the operating room. This brings a disparity in the visual perspectives of the operating fields of the surgeon, his assistants and the trainees. Currently available surgical robots do not have provision for recording, editing and transmitting their 3D displays. These limitations have made assisting, training and mentoring difficult.
Surgical robots have a telescope with two separate optical channels one for right eyes and another for the left eye. They provide two streams of video signals. The two video streams are then synchronized and displayed into the surgeon console in 3D by two cathode ray monitors and an arrangement of minors. Any surgical robot providing two separate video outputs for each eye can be used. The daVinci™ surgical system is preferred. The video streams were tapped from the synchronizers in the vision cart. S-Video cables and BNC- RCA composite cables were used for connection between the vision cart and the display system.
Any projector capable of projecting an electronic video signal can be used. Two DLP projectors were mounted one above another on a frame. The video signals from the da Vinci™ vision cart are fed in the projectors through S-Video or RCA composite inputs. However other forms of standard connectors can also be used. Each of the two projectors is designated to project image of either right or left eye. In front of the lenses of the two projectors, two polarizing filters are mounted. CRT and DLP projectors can use linear or circular polarizers. The LCD projectors can use only circular polarizers. Linear polarizers are arranged in a 45° and 135° orientation. Circular polarizers are arranged in a clockwise and counterclockwise orientation.
The polarized images are projected on a special screen. This type of screen surface maintains the integrity of the polarization of the image. The preferred screens are silver matte (Dalite), Datex 1.8(Dalite) and Flimscreen 150(Stewart). A rear projection system with reflectors assembly to decease the projection throw distance can also be used.
Special eyewear with 45° and 135° linear polarizing filters is used to visualize the projection to get a 3D perception. If the projectors have circular polarizers then eyeware used has circular polarizers arranged in a clockwise and counter-clockwise orientation. The special eyeware permits the isolation of image seen by each eye.
The two video signals for each eye are multiplexed into a single stream. Any commercially available field multiplexer can be used. Preferably, the Mux-3 from Vrex Inc is used. The multiplexed signal is then recorded on videotape or as DV format on a hard drive. The recording thus obtained can be edited on commercially available non-linear editing software using native DV codecs. A codec is a compresser/decompresser, a bit of software or hardware that takes raw video and compresses it, and can take the compressed video and decompress it back to raw video.
Codecs exist for all kinds of compressed video, including DV, motion- JPEG, MPEG, Indeo, Cinepak, Sorensen, wavelet, fractal, RealVideo, vXtreme, MPEG2 codec and many others. Preferably, the codecs have a bit rate of 4.5MBPS and above. Final Cut Pro 4 or Ulead Video Studio 4 is preferably used for the editing. The multiplexed signal can be transmitted to geographically different locations using microwave, Radio frequency, satellite, fiber-optic channel or any other means of transmitting video signal in analogue or uncompressed digital form. For display the multiplexed signal is separated into two separate video streams by commercially available decoders. A Video Converter box from Vrex Inc is preferably used. The two-separated videos are displayed through a setup similar to that described above.
The present invention will have immediate effect in easing the training and mentorship for the future robotic surgeons. The learning curves of two consecutive surgeons were analyzed. Surgeon-A was trained without and surgeon-B was trained and mentored with the stereoscopic projection based display. First 20 cases of both surgeons are analyzed (n=40). The mean operative time during mentoring for surgeon-B was 45% less than the mean operative time of the surgeon-A, during the same phase of training (p < 0001).
Throughout this application, author and year, and patents, by number, reference various publications, including United States patents. Full citations for the publications are listed below. The disclosures of these publications and patents in their entireties are hereby incorporated by reference into this application in order to more fully describe the state of the art to which this invention pertains.
The invention has been described in an illustrative manner, and it is to be understood that the terminology that has been used is intended to be in the nature of words of description rather than of limitation. Obviously, many modifications and variations of the present invention are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the described invention, the invention may be practiced otherwise than as specifically described.

Claims

CLAIMSWhat is claimed is:
1. A robotic system for broadcasting a three-dimensional image from a surgical robot to a screen, said system comprising: at least one video input operatively connected to a surgical robot; at least one projector means for receiving and projecting the 3D image from said video input; and a screen for displaying the 3D image from said projector.
2. The system according to claim 1 , wherein said video input is selected from the group consisting essentially of S-video cables, BNC-RCA composite cables, KVM switches, component video inputs, coaxial cables, twisted pair cables, and fiber optic cables.
3. The system according to claim 1 , wherein said projecter is selected from the group consisting essentially of a DLP and LCD projectors.
4. The system according to claim 1 , wherein said screen is selected from the group consisting essentially of silver matte screen and film screen.
5. The system according to claim 1 , wherein said video input includes a frame synchronizer.
6. The system according to claim 1 , wherein said video input includes a multiplexer.
7. The system according to claim 6, wherein said multiplexer is selected from the group consisting essentially of a simplex multiplexer, duplex multiplexer, full duplex multiplexer, and triplex multiplexer.
8. A system for telecasting a three-dimensional image, said system comprising: at least one video input for being operatively connected to a surgical robot; at least one projector means for receiving and projecting the 3D image from said video input; transmitting means for transmitting the image from said video input to said projector means; and a screen for displaying the 3D image emitted from said projector.
9. The system according to claim 8, wherein said video input is selected from the group consisting essentially of S-video cables, BNC-RCA composite cables, KVM switches, component video inputs, coaxial cables, twisted pair cables, and fiber optic cables.
10. The system according to claim 8, wherein said projecter is selected from the group consisting essentially of a DLP and LCD projectors.
11. The system according to claim 8, wherein said screen is selected from the group consisting essentially of silver matte screen and film screen.
12. The system according to claim 8, wherein said video input includes a frame synchronizer.
13. The system according to claim 8, wherein said video input includes a multiplexer.
14. The system according to claim 13, wherein said multiplexer is selected from the group consisting essentially of a simplex multiplexer, duplex multiplexer, full duplex multiplexer, and triplex multiplexer.
15. A training system for training surgeons comprising the system according to claim 1.
16. A method of transmitting a three dimensional image from a surgical robot to a screen, comprising the steps of: obtaining a 3D image from a surgical robot; transmitting the 3D image to projector; and displaying the 3D image on a screen.
17. The method according to claim 16, wherein said transmitting step includes transmitting the 3D image to a frame synchronizer and multiplexer prior to transmitting the 3D image to the projector.
18. A method of training surgeons to use a surgical robot comprising the steps of: performing a surgical procedure using a surgical robot; transmitting the 3D images seen by the surgeon during surgery to a screen; and viewing the 3D images, thereby training surgeons regarding surgery using the 3D imagery.
19. A robotic system for broadcasting a three-dimensional image from a surgical robot to a screen, said system comprising: at least one video input operatively connected to a surgical robot; at least one projector means for receiving and projecting the 3D image from said video input; and a screen for displaying the 3D image from said projector.
PCT/US2005/001964 2004-01-23 2005-01-21 Stereoscopic projection-based display for surgical robot WO2005074303A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US53854404P 2004-01-23 2004-01-23
US60/538,544 2004-01-23

Publications (1)

Publication Number Publication Date
WO2005074303A1 true WO2005074303A1 (en) 2005-08-11

Family

ID=34825989

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2005/001964 WO2005074303A1 (en) 2004-01-23 2005-01-21 Stereoscopic projection-based display for surgical robot

Country Status (1)

Country Link
WO (1) WO2005074303A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9498231B2 (en) 2011-06-27 2016-11-22 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US10105149B2 (en) 2013-03-15 2018-10-23 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US10219811B2 (en) 2011-06-27 2019-03-05 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US10799316B2 (en) 2013-03-15 2020-10-13 Synaptive Medical (Barbados) Inc. System and method for dynamic validation, correction of registration for surgical navigation
US11116574B2 (en) 2006-06-16 2021-09-14 Board Of Regents Of The University Of Nebraska Method and apparatus for computer aided surgery
US11911117B2 (en) 2011-06-27 2024-02-27 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5867210A (en) * 1996-02-09 1999-02-02 Rod; Samuel R. Stereoscopic on-screen surgical microscope systems
US20010028416A1 (en) * 2000-02-03 2001-10-11 Divelbiss Adam W. System and method for displaying 3D imagery using a dual projector 3D stereoscopic projection system
US20010030682A1 (en) * 1999-12-28 2001-10-18 Tserkovnyuk Walter V. 3D camera
US6788018B1 (en) * 1999-08-03 2004-09-07 Intuitive Surgical, Inc. Ceiling and floor mounted surgical robot set-up arms

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5867210A (en) * 1996-02-09 1999-02-02 Rod; Samuel R. Stereoscopic on-screen surgical microscope systems
US6788018B1 (en) * 1999-08-03 2004-09-07 Intuitive Surgical, Inc. Ceiling and floor mounted surgical robot set-up arms
US20010030682A1 (en) * 1999-12-28 2001-10-18 Tserkovnyuk Walter V. 3D camera
US20010028416A1 (en) * 2000-02-03 2001-10-11 Divelbiss Adam W. System and method for displaying 3D imagery using a dual projector 3D stereoscopic projection system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DA-LITE SCREEN COMPANY: MODEL C WEBSITE, June 2003 (2003-06-01), Retrieved from the Internet <URL:http://web.archive.org/web/20030623041156/www.da-lite.com/products/product.php?cID=9&pID=203> *
TELECOMMUNICATIONS KNOWLEDGEBASE WEBSITE, May 2001 (2001-05-01), XP002988044, Retrieved from the Internet <URL:http://web.archive.org/web/20010518215402/http://dmsweb.badm.sc.edu/grover/telecom/level1/physical.htm> *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11116574B2 (en) 2006-06-16 2021-09-14 Board Of Regents Of The University Of Nebraska Method and apparatus for computer aided surgery
US11857265B2 (en) 2006-06-16 2024-01-02 Board Of Regents Of The University Of Nebraska Method and apparatus for computer aided surgery
US9498231B2 (en) 2011-06-27 2016-11-22 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US10080617B2 (en) 2011-06-27 2018-09-25 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US10219811B2 (en) 2011-06-27 2019-03-05 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US11911117B2 (en) 2011-06-27 2024-02-27 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US10105149B2 (en) 2013-03-15 2018-10-23 Board Of Regents Of The University Of Nebraska On-board tool tracking system and methods of computer assisted surgery
US10799316B2 (en) 2013-03-15 2020-10-13 Synaptive Medical (Barbados) Inc. System and method for dynamic validation, correction of registration for surgical navigation

Similar Documents

Publication Publication Date Title
US10622111B2 (en) System and method for image registration of multiple video streams
EP0746949B1 (en) Synthesized stereoscopic imaging system and method
Satava 3-D vision technology applied to advanced minimally invasive surgery systems
US5867210A (en) Stereoscopic on-screen surgical microscope systems
AU2008270883B2 (en) Virtual interactive presence systems and methods
CA2940814C (en) Interactive display for surgery
EP3107286B1 (en) Medical robotic system providing three-dimensional telestration
Durrani et al. Three-dimensional video imaging for endoscopic surgery
JP2607828B2 (en) High Definition Television Integrated Microscope System
Breedveld et al. Observation in laparoscopic surgery: overview of impeding effects and supporting aids
WO2005074303A1 (en) Stereoscopic projection-based display for surgical robot
US12090002B2 (en) System and methods for tele-collaboration in minimally invasive surgeries
US20070274577A1 (en) &#34;System for the stereoscopic viewing of real time or static images&#34;
Kourambas et al. Advances in camera, video, and imaging technologies in laparoscopy
CN2860384Y (en) Video three-dimensional image-forming microscopic equipment for surgery
Hattori et al. Advanced autostereoscopic display for G-7 pilot project
WO1994028783A1 (en) Medical video endoscope system
Russell et al. Laparoscopic telescope with alpha port and aesop to view open surgical procedures
Keller et al. Real-time structured light depth extraction
Dodgson et al. Autostereoscopic 3D display in laparoscopic surgery
WO2021090335A1 (en) An apparatus for a real time 3d view of an operating site
Song et al. Design of the computerized 3D endoscopic imaging system for delicate endoscopic surgery
Lipkin et al. Video imaging and documentation
Jones Jr et al. Three-dimensional imaging laparoscope
Webster et al. Realtime three-dimensional remote medical skills acquisition

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase