CN118845220A

CN118845220A - Path planning method, system and equipment for surgical robot

Info

Publication number: CN118845220A
Application number: CN202310501290.7A
Authority: CN
Inventors: 张桥
Original assignee: Wuhan United Imaging Zhirong Medical Technology Co Ltd
Current assignee: Wuhan United Imaging Zhirong Medical Technology Co Ltd
Priority date: 2023-04-28
Filing date: 2023-04-28
Publication date: 2024-10-29

Abstract

The embodiment of the specification provides a path planning method of a surgical robot, which is applied to interventional operation in a medical equipment cavity, wherein the surgical robot comprises a mechanical arm, a mechanical arm tail end and a surgical instrument arranged at the mechanical arm tail end; the path planning method comprises the following steps: acquiring posture information of the surgical instrument and three-dimensional information of the medical equipment cavity; determining an internal safety point located within the medical device lumen based on the pose information of the surgical instrument and the three-dimensional information of the medical device lumen; and planning a moving path of the tail end of the mechanical arm based on the internal safety point. The embodiment of the specification also provides a path planning system of the surgical robot. The embodiment of the specification also provides surgical robot equipment, and the path planning method of the surgical robot provided by the embodiment of the specification can be realized.

Description

Path planning method, system and equipment for surgical robot

Technical Field

The present disclosure relates to the field of robots, and in particular, to a path planning method and system for a surgical robot, and a surgical robot device.

Background

Interventional procedures are a relatively common technique in modern surgery, particularly in the field of minimally invasive surgery. For example, needle procedures are guided by imaging and other sensors to penetrate the target site of a soft tissue lesion, and thus, to prevent medication, biopsy, local anesthesia, radiation, ablation therapy, etc. The interventional operation is widely applied to diagnosis and treatment of organs and tissues such as prostate, lung, liver, kidney, spinal column and the like.

The traditional interventional operation has the defects of multiple scanning verification, long operation time, high requirements on the technical level of doctors, poor repeatability and the like, and has the problems of direct exposure of the doctors to rays, limited operation space and the like. Therefore, there is a need to design a surgical robot, a path planning method and a system thereof, which can implement an interventional procedure in a medical device cavity.

Disclosure of Invention

One of the embodiments of the present specification provides a path planning method of a surgical robot. The path planning method is applied to interventional operation in a medical equipment cavity and is characterized in that the operation robot comprises a mechanical arm and a tail end instrument connected to the distal end of the mechanical arm; the path planning method comprises the following steps: acquiring pose information of the tail end instrument and three-dimensional information of the medical equipment cavity; determining an internal safety point located within the medical device lumen based on pose information of the end instrument and three-dimensional information of the medical device lumen; a path of movement of the end instrument is planned based on the internal safety point.

In some embodiments, obtaining pose information of the end instrument includes: pose information of the end instrument is determined based on the interventional procedure information and the interventional procedure site of the patient.

In some embodiments, the path planning method further comprises: determining an external safety point outside the medical equipment cavity based on pose information of the end instrument and three-dimensional information of the medical equipment cavity; the end instrument comprises an end clamping jaw and a surgical instrument clamped by the end clamping jaw; planning a path of movement of the end instrument based on the internal safety point includes: a movement path of the end jaws is planned based on the inner safety point and the outer safety point.

In some embodiments, planning a path of movement of the end instrument based on the internal safety point comprises: dividing a path of movement of the end instrument into a plurality of sub-paths based on the internal safety point; at least one sub-path is planned based on a collision-free path planning algorithm.

In some embodiments, the collision-free path planning algorithm includes a two-tree fast-expansion random tree algorithm and a path pruning algorithm.

In some embodiments, the path planning method further comprises: acquiring three-dimensional position information of the patient; the planning at least one sub-path based on the collision-free path planning algorithm comprises: the at least one sub-path is planned using the collision-free path planning algorithm based on pose information of the end instrument, three-dimensional information of the medical device lumen, and three-dimensional position information of the patient.

In some embodiments, the path planning method further comprises: planning a first retreat path of the tail clamping jaw based on a joint straight line interpolation algorithm; the planning a path of movement of the end instrument based on the internal safety point includes: an exit path of the end jaws is planned based on the internal safety point and the first escape path.

In some embodiments, the path planning method further comprises: and when the first retreat path of the tail clamping jaw is planned based on the joint straight line interpolation algorithm, executing warning operation.

One of the embodiments of the present specification provides a path planning system of a surgical robot, the system comprising: the information acquisition module is used for acquiring pose information of the surgical instrument and three-dimensional information of the medical equipment cavity; a determining module for determining an internal safety point located within the medical device cavity based on pose information of the surgical instrument and three-dimensional information of the medical device cavity; and the planning module is used for planning the moving path of the tail end of the mechanical arm based on the internal safety point.

In some embodiments, the information acquisition module acquires pose information of an end instrument based on interventional procedure information and interventional procedure site determination pose information of the end instrument of a patient.

In some embodiments, the determination module is to determine an external safety point located outside the medical device lumen based on pose information of the end instrument and three-dimensional information of the medical device lumen; the end instrument comprises an end clamping jaw and a surgical instrument clamped by the end clamping jaw; the planning module is used for planning the moving path of the tail clamping jaw based on the inner safety point and the outer safety point.

In some embodiments, the planning module is to divide the path of movement of the end instrument into a plurality of sub-paths based on the internal safety point; the planning module is used for planning at least one sub-path based on a collision-free path planning algorithm.

In some embodiments, the planning module is further configured to optimize at least one sub-path based on a path optimization algorithm.

In some embodiments, the collision-free path planning algorithm comprises a two-tree fast expansion random tree algorithm, and the path optimization algorithm comprises a path pruning algorithm.

In some embodiments, the information acquisition module is configured to acquire three-dimensional position information of the patient; the planning module is used for planning the at least one sub-path by using the collision-free path planning algorithm based on pose information of the tail end instrument, three-dimensional information of the medical equipment cavity and three-dimensional position information of the patient.

In some embodiments, the planning module is configured to plan a first retraction path of the end jaws based on a joint straight line interpolation algorithm; the planning module is further configured to plan an exit path of the end jaw based on the internal safety point and the first exit path.

In some embodiments, the system further comprises an alert module; the warning module is used for executing warning operation.

One of the embodiments of the present disclosure provides a surgical robot device, where the device includes at least one processor and at least one storage device, where the at least one storage device is configured to store instructions, and when the at least one processor executes the instructions, implement the path planning method of the surgical robot according to any one of the embodiments described above.

Drawings

The present specification will be further elucidated by way of example embodiments, which will be described in detail by means of the accompanying drawings. The embodiments are not limiting, in which like numerals represent like structures, wherein:

FIG. 1 is an application scenario diagram of a surgical robotic device system according to some embodiments of the present description;

FIG. 2 is a schematic perspective view of a surgical robotic device according to some embodiments of the present disclosure;

FIG. 3 is a schematic side view of a surgical robotic device according to some embodiments of the present disclosure;

FIG. 4 is a block diagram of a path planning system of a surgical robot shown in accordance with some embodiments of the present description;

FIG. 5 is an exemplary flow chart of a path planning method of a surgical robot shown in accordance with some embodiments of the present description;

FIG. 6 is a simplified workflow diagram of a surgical robot performing a lancing operation according to some embodiments of the present disclosure;

FIG. 7 is a theoretical schematic diagram of an internal safety point calculation method in a path planning method of a surgical robot according to some embodiments of the present disclosure;

FIG. 8A is a schematic diagram of a positional relationship between a first predetermined end instrument coordinate system and a medical device lumen coordinate system in a path planning method for a surgical robot according to some embodiments of the present disclosure;

FIG. 8B is a schematic diagram of a positional relationship between a second predetermined end instrument coordinate system and a medical device lumen coordinate system in a path planning method of a surgical robot according to some embodiments of the present disclosure;

FIG. 9 is a schematic diagram of pruning effects of the path pruning algorithm according to some embodiments of the present description;

FIG. 10 is an exemplary flow chart of a path planning method of a surgical robot according to other embodiments of the present disclosure;

FIG. 11 is a schematic view of the end jaw retraction of a surgical robot according to further embodiments of the present disclosure;

fig. 12 is a schematic view from an elevational perspective of a surgical robot with end jaws retracted according to further embodiments of the present disclosure;

Fig. 13 is a schematic view of the end jaw retraction direction of a surgical robot according to further embodiments of the present disclosure;

FIG. 14 is a schematic diagram illustrating the success of planning a first backoff path based on a joint straight line interpolation algorithm according to further embodiments of the present disclosure;

FIG. 15 is a schematic diagram illustrating a failure to plan a first backoff path based on a joint straight-line interpolation algorithm according to other embodiments of the present disclosure;

fig. 16 is a schematic diagram of a manual drag operation for a surgical robot with a failed end jaw back-off plan according to some embodiments of the present disclosure;

fig. 17 is yet another exemplary flow diagram of a path planning method of a surgical robot according to other embodiments of the present disclosure.

Detailed Description

In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is apparent that the drawings in the following description are only some examples or embodiments of the present specification, and it is possible for those of ordinary skill in the art to apply the present specification to other similar situations according to the drawings without inventive effort. Unless otherwise apparent from the context of the language or otherwise specified, like reference numerals in the figures refer to like structures or operations.

It will be appreciated that "system," "apparatus," "unit" and/or "module" as used herein is one method for distinguishing between different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

As used in this specification and the claims, the terms "a," "an," "the," and/or "the" are not specific to a singular, but may include a plurality, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that the steps and elements are explicitly identified, and they do not constitute an exclusive list, as other steps or elements may be included in a method or apparatus.

A flowchart is used in this specification to describe the operations performed by the system according to embodiments of the present specification. It should be appreciated that the preceding or following operations are not necessarily performed in order precisely. Rather, the steps may be processed in reverse order or simultaneously. Also, other operations may be added to or removed from these processes.

The interventional operation treatment is a minimally invasive treatment by using modern high-tech means, namely, under the guidance of medical imaging equipment, surgical instruments are introduced into the body of a patient to diagnose and locally treat the in-vivo pathological condition. The embodiment of the specification provides surgical robot equipment, a path planning system of a surgical robot and a path planning method of the surgical robot, which can be applied to interventional surgical treatment requiring complex operation. In some embodiments of the present disclosure, in a teleoperation scenario, the surgical robot generally calculates and determines a motion track of the mechanical arm according to the control signal after receiving the control signal of the master control end, and then the mechanical arm moves according to the determined motion track. When in real-time intervention operation, a patient lies on a movable sickbed, the movable sickbed carries the patient to move into the cavity of the medical equipment, and the tail end clamping jaw at the far end of the mechanical arm of the operation robot clamps the operation instrument to perform intervention operation on the patient. Because the space in the cavity of the medical equipment is limited, the space occupation of the tail end instrument (comprising the tail end clamping jaw and the surgical instrument) is large, so that the movement range of the mechanical arm is small, and the tail end instrument must not collide with surrounding objects in the movement process of the tail end instrument under the condition of limited space in the cavity of the medical equipment. The obstacles in the environment that present a high risk of collision are mainly surgical instruments (e.g. needles) left on the body surface of the patient, the body of the patient, the inner walls of the treatment device cavity, etc. In order to avoid collision of the end instrument during the operation, the motion path of the operation robot needs to be planned for the expected moving position and the target pose of the operation instrument before the mechanical arm moves. Therefore, the embodiments of the present disclosure provide a surgical robot apparatus, a path planning system for a surgical robot, and a path planning method for a surgical robot, so as to perform path planning on the surgical robot in advance during a real-time interventional procedure, so as to avoid collision of an end instrument during the procedure.

The embodiment of the specification provides a path planning system and a path planning method of a surgical robot, which are applied to interventional operations in a medical equipment cavity. The path planning method comprises the following steps: the method comprises the steps of acquiring pose information of an end instrument and three-dimensional information of a medical equipment cavity, determining an internal safety point positioned in the medical equipment cavity based on the pose information of the end instrument and the three-dimensional information of the medical equipment cavity, and planning a moving path of the end instrument based on the internal safety point. By the path planning system and the path planning method of the surgical robot, the mechanical arm enters the medical equipment cavity during real-time interventional operation, an internal safety point transition strategy is adopted, and a moving path of the terminal instrument is generated by a collision-free path planning algorithm, so that complex environmental barriers can be effectively avoided, and reliability and safety in the interventional operation process are ensured. In some embodiments, the path planning system and the path planning method of the surgical robot provided in the embodiments of the present disclosure may also be applied to other scenarios, for example, a scenario in which a user controls an end instrument in the vicinity of the surgical robot based on a planned movement path through a control handle.

Fig. 1 is an application scenario diagram of a surgical robotic device system according to some embodiments of the present description.

The surgical robot device system 1000 is a bionic robot system applied to interventional operations in the medical field, and can assist doctors to complete surgical actions with higher medical difficulty. In some embodiments, as shown in fig. 1, a surgical robotic device system 1000 may include a surgical robotic device 100, a network 200, a terminal 300, a processing device 400, and a storage device 500. In some embodiments, the surgical robotic device system 1000 may be applied in medical fields where complex operations need to be implemented, such as real-time interventional procedures. Real-time interventional procedures, which are interventional procedures within a medical device lumen (e.g., CT device), are performed by a user (e.g., doctor) outside the operating room by exposing the medical device to real-time radiation during the interventional procedure. The off-line intervention operation corresponding to the real-time intervention operation, namely the intervention operation outside the cavity of the medical equipment, namely the current common intervention operation mode, needs doctors to have abundant experience to judge the operation position and also needs accurate operation capability, so the current intervention operation has higher difficulty. The surgical robotic device system 1000 provided herein solves the present challenges.

The surgical robotic device 100 may perform corresponding operations according to received instructions (e.g., control signal instructions). In some embodiments, surgical robotic device 100 may include a surgical robot, a medical device cavity, and a mobile hospital bed. When the surgical robotic device 100 receives data or instructions sent by other devices or system components, the mobile patient bed may be moved with the patient to an instructed position, the medical device cavity may be opened according to the instructions, and the surgical robot may perform a procedure (e.g., an interventional procedure) based on the instructions. In some embodiments, the surgical robot includes at least a robotic arm and a tip instrument (e.g., a surgical instrument including a tip jaw, a needle, etc.), and when the surgical robot receives data or instructions sent by other devices or system components, the robotic arm on the surgical robot moves to move the tip instrument to a position indicated by the instructions and performs a surgical action (e.g., inserts the needle into the patient). In some embodiments, the surgical robot may also be provided with sensors to detect kinematic parameters (e.g., position, angle, speed, etc.) as the links move and feed the kinematic parameters back to the processing device 400 or the terminal 300. In some embodiments, the surgical robot may also be provided with a camera to acquire an image of the surgical robot and the environment in which it is located and send the image to the processing device 400 or the terminal 300. For more details about the surgical robot, reference may be made to the relevant descriptions of fig. 2 and 3, which are not repeated here.

Network 200 may include any suitable network capable of facilitating the exchange of information and/or data by surgical robotic device 100. In some embodiments, at least one component of the surgical robotic device system 1000 (e.g., the surgical robotic device 100, the terminal 300, the processing device 400, the storage device 500) may exchange information and/or data with at least one other component of the surgical robotic device system 1000 over the network 200. For example, the processing device 400 may obtain a control signal input by a user from the terminal 300 through the network 200. In some embodiments, network 200 may include at least one network access point. For example, the network 200 may include wired and/or wireless network access points (e.g., base stations and/or internet switching points) through which at least one component of the surgical robotic device system 1000 may connect to the network 200 to exchange data and/or information.

The terminal 300 may be in communication and/or connection with the surgical robotic device 100, the processing device 400, and/or the storage device 500. In some embodiments, terminal 300 may include a mobile device 310, a tablet computer 320, a laptop computer 330, or the like, or any combination thereof. For example, mobile device 310 may include a mobile control handle, a Personal Digital Assistant (PDA), a smart phone, or the like, or any combination thereof. In some embodiments, the terminal 300 may include a display that may be used to display information or images related to the surgical procedure, such as current surgical data of the surgical robotic device 100 or images of the environment in which the surgical robot is located, etc.

In some embodiments, terminal 300 may include an input device. The input device may be selected from keyboard input, touch screen (e.g., with haptic or tactile feedback) input, voice input, eye tracking input, gesture tracking input, brain monitoring system input, image input, video input, or any other similar input mechanism. Input information received via the input device may be transferred via, for example, a bus to processing device 400 for further processing. Other types of input devices may include cursor control devices, such as a mouse, a trackball, or cursor direction keys. In some embodiments, the user may input the control signal through an input device. In some embodiments, terminal 300 may include an output device. The output device may include a display, speakers, printer, etc., or any combination thereof. The output device may be used to output parameters related to the surgical robotic device 100, etc., determined by the processing device 400. In some embodiments, the terminal 300 may be part of the processing device 400.

The processing device 400 may process data and/or information obtained from the surgical robotic device 100, the at least one terminal 300, the storage device 500, or other components of the path planning system 2000 of the surgical robot. For example, the processing device 400 may obtain a control signal from the terminal 300 for further calculating a motion profile of the surgical robot at a next moment. For another example, the processing device 400 may obtain system configuration parameters of the surgical robot device 100 from the storage device 500, such as related description parameters of the surgical robot and the medical device cavity (e.g., modeling parameters), related description parameters of the robotic arm and the end instrument of the surgical robot (e.g., surgical instrument model, etc.), related description parameters of the patient (e.g., patient condition information), etc., for further performing the surgical robot path trajectory calculation. In some embodiments, the processing device 400 may be a single server or a group of servers. The server farm may be centralized or distributed. In some embodiments, the processing device 400 may be local or remote. For example, the processing device 400 may access information and/or data from the surgical robotic device 100, the storage device 500, and/or the terminal 300 via the network 200. As another example, the processing device 400 may be directly connected to the surgical robotic device 100, the terminal 300, and/or the storage device 500 to access information and/or data. As another example, the treatment device 400 may be mounted on the surgical robotic device 100. In some embodiments, processing device 400 may be implemented on a cloud platform. For example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, and the like, or any combination thereof.

Storage device 500 may store data, instructions, and/or any other information. For example, the storage device 500 may store system configuration parameters and the like of the surgical robotic device 100. In some embodiments, the storage device 500 may store data obtained from the surgical robotic device 100, the terminal 300, and/or the processing device 400. In some embodiments, storage device 500 may store data and/or instructions that processing device 400 uses to perform or use to implement the exemplary methods described in this specification. In some embodiments, storage device 500 may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), and the like, or any combination thereof. In some embodiments, storage device 500 may be implemented on a cloud platform.

In some embodiments, the storage device 500 may be connected to the network 200 to communicate with at least one other component (e.g., the processing device 40, the terminal 300) in the surgical robotic device system 1000. At least one component in the surgical robotic device system 1000 may access data stored in the storage device 500 through the network 200. In some embodiments, the storage device 500 may be part of the processing device 400. In some embodiments, the processing device 400 and the storage device 500 may be integrated in the surgical robotic device 100.

It should be noted that the foregoing description is provided for the purpose of illustration only and is not intended to limit the scope of the present application. Many variations and modifications will be apparent to those of ordinary skill in the art, given the benefit of this disclosure. The features, structures, methods, and other features of the described exemplary embodiments of the application may be combined in various ways to obtain additional and/or alternative exemplary embodiments. For example, the storage device 500 may be a data storage device 500 that includes a cloud computing platform, such as a public cloud, a private cloud, a community, a hybrid cloud, and the like. However, such changes and modifications do not depart from the scope of the present specification.

FIG. 2 is a schematic perspective view of a surgical robotic device according to some embodiments of the present disclosure; fig. 3 is a schematic side view of a surgical robotic device according to some embodiments of the present disclosure.

In some embodiments, as shown in fig. 2, 3, surgical robotic device 100 may include a surgical robot 110, a medical device cavity 120, and a mobile hospital bed 130. In some embodiments, the surgical robotic device 100 is configured to perform real-time interventional procedures with the patient lying on the mobile hospital bed 130, the mobile hospital bed 130 carrying the patient for movement into the medical device cavity 120, and the distal end jaws of the robotic arm of the surgical robot 110 hold the surgical instrument for performing interventional procedures on the patient. During a real-time interventional operation (e.g., a puncture operation), after a surgical instrument (e.g., a puncture needle) is installed outside the medical device cavity 120 by using a distal end clamping jaw of the mechanical arm 111 of the surgical robot 110, the medical device cavity 120 needs to be accessed for performing the interventional operation, and after the interventional operation (e.g., the puncture needle is inserted into the surgical site of the patient in a preset posture) is completed, the mechanical arm 111 may exit the medical device cavity 120, and then the interventional operation workflow of the next surgical instrument is performed or the operation is completed.

The surgical robot may be a mechanical electronic device that mimics the functions of a human arm, wrist, and hand. In some embodiments, as shown in fig. 2, surgical robot 110 may include a robotic arm 111 and an end instrument 112 coupled to a distal end of robotic arm 111. In some embodiments, the mechanical arm 111 may include at least two links that are movably connected in sequence, and when the mechanical arm 111 receives data or instructions sent by other devices or system components, a joint on the mechanical arm 111 connected to the links moves, so that the mechanical arm 111 moves to a position indicated by the instructions, and further, the end instrument 112 moves to the position indicated by the instructions. In some embodiments, the end instrument 112 may include a link at the distal-most end of the robotic arm 111 and a surgical instrument, and effect penetration of the surgical instrument into the patient. In some embodiments, the end instrument 112 may include end jaws 1121, the end jaws 1121 being configured to grip the surgical instrument 1122, and upon completion of the lancing operation, the end jaws 1121 are released and the robotic arm 111 is retracted from the medical device cavity 112 with the end jaws 1121. In some embodiments, where a portion of the interventional procedure requires penetration of multiple surgical instruments 1122, the robotic arm 111 withdraws from the medical device lumen 112, re-clamps the next surgical instrument 1122 outside of the medical device lumen 120, and then proceeds to the penetration workflow of the next surgical instrument 1122.

In some embodiments, the tip coordinate system of the tip instrument 112 may be established at the origin of the tip instrument 112 (e.g., point O in fig. 13), and the position and pose of the tip instrument 112 may be represented by the tip coordinate system and modeled parameters of the tip instrument 112. In some embodiments, the origin of the end instrument 112 may be a designated point on the end instrument 112. For more details regarding the end coordinate system of the mechanical arm, reference may be made to the description of step 4400, which is not repeated herein.

The medical device lumen 120 may be used to scan an image of a patient for viewing by a physician. In some embodiments, the medical device cavity 120 may be a cavity of a magnetic resonance imaging device (Magnetic Resonance Imaging, abbreviated as MRI), an electronic computer tomography (Computer Tomography, abbreviated as CT), or other image scanning devices. In some embodiments, medical device lumen 120 may also be a lumen formed by a combination of multiple medical imaging devices.

A mobile patient table 130 for carrying a patient, and the mobile patient table 130 is movable within the bore of the medical device cavity 120 to enable imaging examinations of the patient on the mobile patient table 130.

Fig. 4 is a block diagram of a path planning system of a surgical robot shown in accordance with some embodiments of the present description.

As shown in fig. 4, the path planning system 2000 includes an information acquisition module 2100, a determination module 2200, and a planning module 2300. In some embodiments, information acquisition module 2100, determination module 2200, and planning module 2300 may be implemented by processing device 400.

The information acquisition module 2100 may be used to acquire information/data required for an interventional procedure. In some embodiments, the information acquisition module 2100 may be used to acquire pose information of the end instrument 112 and three-dimensional information of the medical device lumen 120. In some embodiments, the pose information of the end instrument 112 may be preset by a physician in the treatment device 400 according to the condition of the patient. In some embodiments, the three-dimensional information of the medical device lumen 120 may be preset in the processing device 400 according to the three-dimensional structure of the medical device and the model of the medical device. The pose information of the end instrument and the three-dimensional information of the medical device lumen 120 may be referred to in the detailed description of step 3100, and will not be described here.

The determination module 2200 may be used to determine the location of the security point. In some embodiments, the determination module 2200 may determine the internal safety point 121 located within the medical device lumen 120 based on pose information of the end instrument 112 and three-dimensional information of the medical device lumen 120. In some embodiments, the position of the end instrument 112 within the medical device cavity 120 is shown in fig. 3 below, the height of the mobile hospital bed 130 is fixed, the movement range of the mechanical arm 111 is small due to the large space occupation of the mechanical arm end 112, and it is necessary to ensure that the mechanical arm end 112 does not collide with surrounding objects during the movement of the mechanical arm end 112 under the condition of limited space in the medical device cavity 120, and the barriers in the environment with high collision risk are mainly surgical instruments left on the patient body, the inner wall of the medical device cavity 120, and the like. Thus, prior to the initiation of an interventional procedure, an internal safety point 121 within the medical device lumen 120 may be determined based on pose information of the end instrument 112 and three-dimensional information of the medical device lumen 120. The specific determination method of the internal safety point 121 in the medical device lumen 120 may be referred to as the detailed description of step 3200, and will not be described herein.

The planning module 2300 may be used to plan a path of movement of the end instrument 112. In some embodiments, the planning module 2300 may plan the path of movement of the end instrument 112 based on the internal safety point 121. In some embodiments, planning module 2300 may also plan the path of movement of end instrument 112 into multiple sub-paths based on internal safety point 121. For a specific method of planning the movement path of the end instrument 112 based on the internal safety point 121, reference may be made to the detailed description of step 3300, which is not repeated here.

In some embodiments, the path planning system 2000 further includes an alert module 2400. Alert module 2400 may be used to perform alert operations. In some embodiments, the alert module 2400 may perform an alert operation when the planning of the first backoff path of the end instrument 112 based on the joint straight line interpolation algorithm fails. The triggering of the warning operation may be described in detail in step 4500, which is not described herein.

It should be noted that the above description of the path planning system and its modules is for convenience of description only, and is not intended to limit the present description to the scope of the illustrated embodiments. It will be appreciated by those skilled in the art that, given the principles of the system, various modules may be combined arbitrarily or a subsystem may be constructed in connection with other modules without departing from such principles. For example, the information acquisition module 2100, the determination module 2200, and the planning module 2300 disclosed in fig. 4 may be different modules in one system, or may be one module to implement the functions of two or more modules described above. For another example, each module may share one memory module, or each module may have a respective memory module. Such variations are within the scope of the present description.

FIG. 5 is an exemplary flow chart of a path planning method of a surgical robot shown in accordance with some embodiments of the present description; FIG. 6 is a simplified workflow diagram of a surgical robot performing a lancing operation according to some embodiments of the present disclosure; FIG. 7 is a theoretical schematic diagram of an internal safety point calculation method in a path planning method of a surgical robot according to some embodiments of the present disclosure; FIG. 8A is a schematic diagram of a positional relationship between a first predetermined end instrument coordinate system and a medical device lumen coordinate system in a path planning method for a surgical robot according to some embodiments of the present disclosure; FIG. 8B is a schematic diagram of a positional relationship between a second predetermined end instrument coordinate system and a medical device lumen coordinate system in a path planning method of a surgical robot according to some embodiments of the present disclosure; fig. 9 is a schematic diagram of pruning effects of the path pruning algorithm according to some embodiments of the present description. The path planning method for the surgical robot will be described in detail with reference to fig. 5 to 9.

As shown in fig. 5, some embodiments of the present disclosure provide a path planning method for a surgical robot, and a flow 3000 of the path planning method may include the following steps. The process 3000 may be performed by a processing device (e.g., processing device 400). For example, the process 3000 may be implemented as a set of instructions (e.g., an application) stored in a memory external to, for example, the storage device 500, a surgical robot (e.g., the surgical robot 110), and accessible by the path planning system. The processing device may execute the set of instructions and, when executing the instructions, may configure it to perform the process 3000. The operational schematic of the flow 3000 presented below is illustrative. In some embodiments, the process may be accomplished with one or more additional operations not described above and/or one or more operations not discussed. In addition, the order in which the operations of flow 3000 are illustrated in fig. 5 and described below is not intended to be limiting.

Step 3100 obtains pose information of the end instrument 112 and three-dimensional information of the medical device lumen 120. In some embodiments, step 3100 may be performed by processing device 400 or information acquisition module 2100.

The pose information of the end instrument 112 may refer to information of the position, angle, shape, etc. of the end instrument 112 in the surgical space. In some embodiments, the pose information of the end instrument 112 may be target pose information when the end instrument 112 enters the patient's body.

In some embodiments, pose information of the end instrument 112 may be determined based on interventional procedure information and the interventional procedure site of the patient.

The interventional operation information of the patient may include disease condition information of the patient, three-dimensional information of the patient, an approximate location requiring an operation, and the like. In some embodiments, the patient's condition information may include information such as the patient's name, sex, height, weight, medical history, and location of the lesion to be treated in the current interventional procedure. In some embodiments, the interventional information of the patient may be pre-stored in the processing device 400 or the information acquisition module 2100, and a user (e.g., doctor) may input the patient's name through the terminal 300 to read the interventional information of the patient. In some embodiments, lesion location information of the patient may be determined by real-time observation by the medical device.

The interventional procedure site may be an interventional site on the patient's body that the physician scans based on the patient's condition and medical equipment and determines from clinical experience, from which the surgical instrument 1122 enters the patient's body. In some embodiments, the interventional surgical site may also include a depth of penetration of the surgical instrument 1122 into the patient's body to achieve precise access to the location of the lesion in need of the procedure.

In some embodiments, pose information of the end instrument 112 may be entered by a physician through an input device on the terminal 300. In some embodiments, the pose information of the end instrument 112 is primarily determined by first determining the pose information of the surgical instrument 1122, and the pose information of the end jaws 1121 used to clamp the surgical instrument 1122 is determined based on the pose information of the surgical instrument 1122, i.e., the pose information of the end instrument 112 is determined. For example, the terminal 300 may be provided with a three-dimensional software input interface on which a physician may draw target pose information for the surgical instrument 1122 as it enters the patient to reach a lesion. Because the medical device cavity 120 has limited space, the surgical device 1122 is inconvenient to rotate in the medical device cavity 120, so that the moving mechanical arm 111 keeps the posture unchanged in the process of moving the surgical device 1122 to the intervention operation position (i.e., the puncture point 124 described below), and the target posture information when the surgical device 1122 moves to the intervention operation position can be obtained through the target posture information when the surgical device 1122 enters the patient body and reaches the focus and the position information of the intervention operation position, which are input by a doctor. In some embodiments, the surgical instrument 1122 may be a needle, and the physician may input pose information of the needle on the input interface, for example, may input the position of the end-to-end points of the needle, where the relative position of the end-to-end points represents the relative position of the needle insertion point (e.g., Q ₁ in fig. 8A) of the needle and the target point (e.g., P ₁ in fig. 8A) in the patient, and the straight line connecting the end-to-end points represents the target pose information (including the target position and the target pose) of the needle.

In some embodiments, to ensure accuracy of the pose of the surgical instrument 1122 as it is introduced into the patient's body, the surgical instrument 1122 is first adjusted to the target pose after the surgical instrument 1122 is mounted on the end jaws 1121, and the surgical instrument 1122 is moved to the intervention surgical site (i.e., the penetration site 124 described below) according to the planned movement path by the mobile robot 111.

The three-dimensional information of the medical device lumen 120 may refer to three-dimensional information of the outline of the medical device lumen 120. In some embodiments, the medical device lumen 120 may be three-dimensionally modeled by photographing or three-dimensional scanning, etc., to store three-dimensional information of the medical device lumen 120 in the processing device or information acquisition module 2100. In some embodiments, the three-dimensional information of the medical device lumen 120 may also be obtained by three-dimensional modeling by way of offline three-dimensional CAD modeling, and the three-dimensional information of the medical device lumen 120 may be pre-stored in the processing device 400 or the information acquisition module 2100. For example, various kinds and types of medical devices are pre-stored in the processing device 400 or the information acquisition module 2100, the user can select a corresponding medical device through the terminal 300, and the processing device 400 can acquire a signal input by the user from the terminal 300 through the network 200, so as to call out the three-dimensional information of the medical device cavity 120 suitable for the interventional operation. When planning the path of the surgical robot, the three-dimensional information of the medical device lumen 120 may be used as obstacle avoidance information for the end instrument 112.

In some embodiments, processing device 400 may include multiple processors, each processing one set of information simultaneously and individually. For example, the acquisition of pose information for the end instrument 112 and the acquisition of three-dimensional information for the medical device lumen 120 may be processed by different processors. By simultaneously acquiring and processing a plurality of groups of information, the information acquisition speed and the real-time performance of information acquisition can be improved.

At step 3200, an internal safety point 121 within the medical device lumen 120 is determined based on the pose information of the end instrument 112 and the three-dimensional information of the medical device lumen 120. In some embodiments, step 3200 may be performed by processing device 400 or determination module 2200.

In some embodiments, referring to fig. 6, an internal safety point 121 (abbreviated as IS point) IS located inside the medical device lumen 120, and the internal safety point 121 may be calculated based on the relative positions of the medical device lumen 120 and the robotic arm 111. The relative positions of the medical device lumen 120 and the robotic arm 111 are determined based on pose information of the end instrument 112 and three-dimensional information of the medical device lumen 120. In some embodiments, the internal safety point 121 may be calculated from the link kinematics of the robotic arm 111. It should be noted that, the internal safety point 121 is not a point in a narrow sense, but refers to a relatively safe five-dimensional spatial pose of the mechanical arm 111 inside the medical device cavity 120, described by five joint angles of the mechanical arm 111, so that the internal safety point 121 is a "point" of the five-dimensional configuration space of the mechanical arm 111. When the mechanical arm 111 moves to the internal safety point 121, it is ensured that the subsequent movement of the mechanical arm 111 (including the movement link of the mechanical arm 111 and the end instrument 112) is not limited, i.e. the three-dimensional information of the medical device cavity 120 does not affect the movement of the link, and the end instrument 112 does not collide with the other phenomena. In some embodiments, the position of the internal safety point 121 may be changed during the interventional procedure to accommodate different pose information of the end instrument 112, it being understood that the position of the internal safety point 121 changes based on the change in pose information of the end instrument 112.

In some embodiments, the internal safety point 121 may also be determined based on target pose information of the end instrument 112. For example, with the target pose of the end instrument 112 as the starting pose of the end instrument 112, the end instrument 112 remains pose unchanged only changing position information during movement of the robotic arm 111 to move the surgical instrument 1122 to an interventional procedure position (i.e., the puncture point 124 described below). Accordingly, the pose of the end instrument 112 at the internal safety point 121 is the same as the initial pose, and the position information of the internal safety point 121 is determined based on the pose information of the end instrument 112.

In some embodiments, referring to fig. 7, an internal safety point 121 (IS point) may be calculated based on the coordinate system H of the medical device lumen 120, the robot-based coordinate system I, the coordinate system J of the mobile hospital bed 130 at registration, and the preset end instrument coordinate system K. Wherein, there is a scanning layer in the center of the medical device cavity 120, and when the patient scans the CT image, the body part to be scanned passes through the scanning layer, and the center plane of the scanning layer is the collimation center line G of the medical device cavity 120. The coordinate system H of the medical device lumen 120 is located on the collimation centerline G of the medical device lumen 120. Registration is an operation of determining the relative positions and attitudes of the surgical robot 110 and the medical device cavity 120 (including moving the patient bed 130) before each operation, and after registration is completed, the robot base coordinate system I and the coordinate system J of the moving patient bed 130 at registration are obtained, and the positional attitude relationship (which may be defined as a matrix R, and is indicated by a black bold arrow R in fig. 7) between the robot base coordinate system I and the coordinate system J of the moving patient bed 130 at registration is determined. In some embodiments, the coordinate system L of the mobile patient bed 130 is fixed on the mobile patient bed 130 in registration. As shown in fig. 7, the coordinate system L of the moving couch 130 at registration may be located on the couch tail on the moving couch 130. Therefore, the coordinate system H of the medical device chamber 120 and the coordinate system L of the moving patient bed 130 at registration are both known coordinate systems and fixed coordinate systems, and thus the positional posture relationship (which may be defined as a matrix S, indicated by a black bold arrow S in fig. 7) between the robot base coordinate system K and the coordinate system L of the moving patient bed 130 at registration can be determined.

In some embodiments, to simplify the calculation method of the internal safety point 121 (IS point), a preset end instrument coordinate system K corresponding to the internal safety point 121 may be preset in the medical device cavity 120, and then the positional posture relationship between the coordinate system H of the medical device cavity 120 and the preset end instrument coordinate system K (may be defined as a matrix T, and IS represented by a black bold arrow T in fig. 7). Therefore, the positional posture relationship (which may be defined as a matrix W, and indicated by a black bold arrow W in fig. 7) between the preset end instrument coordinate system K and the robot base coordinate system I can be obtained from the matrix T and the known matrices R and S, and then the joint angle information of the mechanical arm 111, which is the internal safety point 121, can be obtained through a kinematic inverse solution.

In some embodiments, the line between the origin of coordinates of the preset end instrument coordinate system K and the origin of coordinates of the coordinate system H of the medical device lumen 120 is inclined in the same direction as the line between the point of penetration of the surgical instrument 112 and the target point within the patient. Referring to fig. 8A and 8B, a preset end instrument coordinate system K corresponding to the two internal safety points 121 may be preset in the medical device cavity 120 based on the target pose information of the surgical instrument 112, which are a first preset end instrument coordinate system K ₁ and a first preset end instrument coordinate system K ₂, respectively. Wherein the X-coordinate of the first preset end instrument coordinate system K ₁ is smaller than the X-coordinate of the coordinate system H of the medical device lumen 120, and the X-coordinate of the first preset end instrument coordinate system K ₂ is larger than the X-coordinate of the coordinate system H of the medical device lumen 120. The positional pose relationship (which may be defined as matrix T ₁, represented by black bold arrow T ₁ in fig. 8A) between the coordinate system H of the medical device lumen 120 and the first preset end instrument coordinate system K ₁, and thus the matrix T ₁, is determined. The positional pose relationship (which may be defined as matrix T ₂, represented by black bold arrow T ₂ in fig. 8B) between the coordinate system H of the medical device lumen 120 and the second preset end instrument coordinate system K ₂, and thus the matrix T ₂, is determined.

As shown in fig. 8A, the needle insertion point of the surgical instrument 112 is Q ₁, the target point in the patient is P ₁, and the connection line between Q ₁ and P ₁ may represent the target pose information of the surgical instrument 112. When the X coordinate of the needle insertion point Q ₁ is smaller than that of the target point P ₁, a matrix T ₁ and a known matrix R and a known matrix S are selected to obtain a position posture relation (a matrix W) between a preset end instrument coordinate system K and a robot base coordinate system I.

As shown in fig. 8B, the needle insertion point of the surgical instrument 112 is Q ₂, the target point in the patient is P ₂, and the connection line between Q ₂ and P ₂ may represent the target pose information of the surgical instrument 112. When the X coordinate of the needle insertion point Q ₂ is smaller than that of the target point P ₂, a matrix T ₂ and a known matrix R and a known matrix S are selected to obtain a position posture relation (a matrix W) between a preset end instrument coordinate system K and a robot base coordinate system I.

At 3300, a path of movement of end instrument 112 may be planned based on internal safety point 121. In some embodiments, step 3300 may be performed by processing device 400 or determination module 2300.

In some embodiments, the path of movement of the end instrument 112 includes an entry path and/or an exit path. The needle insertion path may include a path of movement of the end instrument 112 from the initial point 123 to the puncture point 124. The exit path may include a path of movement of the end instrument 112 from the puncture point 124 to the initiation point 123.

In some embodiments, referring to fig. 6, the path of movement of the end instrument 112 planned in step 3300 may be a needle insertion path in which the end instrument 112 moves from an initial point 123 to an internal safety point 121 (i.e., IS point) and then to a puncture point 124. The puncture point 124 may be the location of the surgical instrument 1122 when it reaches a preset surgical location. In some embodiments, the needle insertion path may also be divided into a plurality of sub-paths. The method for planning the multiple sub-paths may be specifically described in detail below, and is not described herein.

According to the path planning method of the embodiment, the whole planned path can be divided into a plurality of sub-paths by adopting the transition point strategy of the internal safety point 121, so that the planned path is ensured to be relatively fixed. The internal safety point 121 is used for carrying out overall path transition on the overall path of the real-time puncture operation workflow, so that the path is ensured to be relatively fixed, and the path planning can be faster and more accurate. In some embodiments, the entire movement path may be divided into: upon access, the end instrument 112 is moved from outside the medical device lumen 120 to an internal safety point 121, from the internal safety point 121 to a surgical site (e.g., a puncture point 124 in fig. 6); upon withdrawal, the end instrument 112 is moved from the surgical site to the internal safety point 121 and from the internal safety point 121 out of the medical device lumen 120. For further description of the internal security dots 121, see further description below with respect to fig. 6, which is not repeated here.

In some embodiments, the path planning method of the surgical robot may further determine an external safety point 122 (abbreviated as OS point) located outside the medical device lumen 120 based on pose information of the end instrument 112 and three-dimensional information of the medical device lumen 120.

In some embodiments, the external safety point 122 may be determined based on three-dimensional information of the medical device lumen 120. An external safety point 122 is located outside of the medical device lumen 120. In some embodiments, the external safety point 122 may be preset as a mounting point for the surgical instrument 1122. In some embodiments, the end instrument 112 includes an end jaw 1121 and a surgical instrument 1122 held by the end jaw 1121. The mounting of the surgical instrument 1122 refers to the operation of holding the surgical instrument 1122 by the distal end clamping jaw 1121 of the robot arm 111. In some embodiments, the external safety point 122 may be fixed throughout each interventional procedure.

In some embodiments, the outer space of the medical device lumen 120 is larger and the outer safety point 122 is more selectable. Determination of the external safety point 122 also requires consideration of pose information of the end instrument 112, and the surgical instrument 1122 needs to be mounted to the end jaw 1121 at the external safety point 122, and thus an external safety point 122 for facilitating the mounting of the surgical instrument 1122 needs to be determined based on the pose information of the end instrument 112.

In some embodiments, the determination of the external safety point 122 may also be based on the internal safety point 121 such that the path of the external safety point 122 to the internal safety point 121 is as little roadblock as possible. In some embodiments, the height of the outer safety point 122 in the vertical direction is the same as that of the inner safety point 121, and the path of movement of the end instrument 112 from the outer safety point 122 to the inner safety point 121 without a roadblock can be a straight line on a horizontal plane, so that the movement process of the end instrument 112 on the path can be significantly simplified, and the path planning calculation process can be simplified.

In some embodiments, the end instrument 112 corresponding to the external safety point 122 is about 1500mm horizontally from the center of the aperture of the medical device lumen 120, with the external safety point 122 being farther from the patient. The distance between the lowest point of the end instrument 112 corresponding to the external safety point 122 and the moving patient bed 130 is over 500mm, which is well above the patient's lying thickness. Thus, the external safety point 122 provided by this embodiment is a location that is sufficiently safe for the end instrument 112 to effectively avoid collisions with the patient when the surgical instrument 1122 is mounted to the end jaw 1121.

In some embodiments, the end instrument 112 travel path may be planned based on the inner safety point 121 and the outer safety point 122. Referring to fig. 6, prior to the interventional procedure, the end instrument 112 may be located at an initial point 123 (simply referred to as the Home point); when the interventional operation IS performed, the mechanical arm 111 moves to drive the end instrument 112 to move from the Home point 123 to the OS point 122, after the OS point 122 installs the surgical instrument 1122 on the end clamping jaw 1121, the end instrument 112 enters the IS point 121 in the medical equipment cavity 120, then continues to move to the puncture point 124, and then performs the interventional operation based on the posture of the surgical instrument 1122 input by the doctor; after the intervention is completed, the surgical instrument 1122 may remain on the patient's body, the distal end of the arm 111 is released from the end jaw 1121, the arm 111 drives the end jaw 1121 back to the OS point outside the medical device cavity 120, and then the next surgical instrument 1122 is installed for the next intervention. In some embodiments, after the distal instrument 112 moves to the puncture point 124, the doctor can further fine tune the posture of the surgical instrument 1122 by remotely controlling the mechanical arm 111 with the master hand at the master end, further ensuring the posture accuracy of the surgical instrument 1122, and then performing the interventional operation.

In some embodiments, the path planning method of the surgical robot may further divide the movement path of the end instrument 112 planned based on the internal safety point 121 into a plurality of sub-paths. The path of movement of the end instrument 112 includes an insertion path and an exit path. In some embodiments, both the needle entry path and the exit path of the end instrument 112 may be split into multiple sub-paths. For example, referring to fig. 6, the needle insertion path of the end instrument 112 may include a first needle insertion path, a second needle insertion path, and a third needle insertion path. The first needle insertion path may be a path of movement of the end instrument 112 from an initial point 123 to an external safety point 122; the second needle insertion path may be a path of movement of the end instrument 112 from the outer safety point 122 to the inner safety point 121; the third needle insertion path may be the path of movement of the end instrument 112 from the internal safety point 121 to the puncture point 124. For another example, the exit path of the robotic arm tip 112 may include a first exit path, a second exit path, and a third exit path. The first exit path may be the path of movement of the end jaw 1121 from the puncture site 124 to the internal safety site 121; the second exit path may be the path of movement of the end jaw 1121 from the inner safety point 121 to the outer safety point 122; the third exit path may be the path of movement of the end jaw 1121 from the external safety point 122 to the initial point 123.

In some embodiments, the path planning method of the surgical robot may further plan at least one sub-path based on a collision-free path planning algorithm.

In some embodiments, the at least one sub-path planned based on the collision-free path planning algorithm may be the second needle-insertion path and/or the third needle-insertion path. In some embodiments, the sub-paths planned based on the collision-free path planning algorithm may be multiple. For example, the sub-paths planned based on the collision-free path planning algorithm may include a second needle-insertion path and a third needle-insertion path; for another example, the sub-paths planned based on the collision-free path planning algorithm may include a second needle-insertion path, a third needle-insertion path, a first exit path, and a second exit path. In some embodiments, each sub-path in the path of movement of the end instrument 112 may be planned based on a collision-free path planning algorithm.

In some embodiments, the collision-free path planning algorithm may include a two-tree fast-expanding random tree algorithm.

The dual-tree fast-expansion random tree algorithm, also called RRTConnect algorithm, is to simultaneously grow two fast-expansion random trees from the starting point to the target point to search the state space. The RRTConnect algorithm, based on a specified starting point and a target point, will plan a collision-free path. The fast double-tree expanding random tree algorithm is one space searching sampling based path planning algorithm, and the path between the starting point and the target point has several broken line segments, each representing one path step length, with one path comprising several path step lengths.

In some embodiments, the path planning method of the surgical robot may further include a path optimization algorithm. In some embodiments, the path optimization algorithm may include a path pruning algorithm. The path pruning algorithm is an algorithm optimization, namely, through a certain judgment, unnecessary traversal processes are avoided, and in an image, certain branches in the search tree are pruned, so that the algorithm is called pruning.

In some embodiments, the path planning method of the surgical robot provided in the present specification may employ a dual-tree fast-expansion random tree algorithm to plan a movement path of the end instrument 112, or plan at least one sub-path in the movement path of the end instrument 112.

In some embodiments, the path planning method of the surgical robot may first calculate the basic path by adopting RRTConnect algorithm, and some redundant parts of the basic path may be pruned by the path pruning algorithm, so that the path after pruning not only ensures no collision, but also obviously shortens the length, and is smoother in the movement process of the mechanical arm 111.

The path pruning algorithm comprises a unidirectional path pruning algorithm and a bidirectional path pruning algorithm. In some embodiments, the path planning method of the surgical robot can use a combination of a dual-tree fast-expansion random tree algorithm and a unidirectional path pruning algorithm. In some embodiments, the path planning method of the surgical robot may employ a combination of a dual-tree fast-expanding random tree algorithm and a bi-directional path pruning algorithm. The bidirectional path pruning algorithm is improved on the basis of the unidirectional pruning algorithm, bidirectional heuristic pruning is carried out from a starting point and a target point, and the RRTconnect algorithm is also a bidirectional search algorithm. Thus, for the RRTconnect search algorithm, the bi-directional pruning algorithm is faster in path generation than the uni-directional pruning algorithm.

In some embodiments, the path planning method of the surgical robot employs a two-tree fast-expanding random tree algorithm and a bi-directional path pruning algorithm combined algorithm. Referring to fig. 9, a circle, triangle, polygon, etc. in the figure represent an obstacle based on a selected start point a and target point F (e.g., start point a IS the OS point in fig. 6, target point F IS the IS point in fig. 6). Firstly, a base path M is obtained through a RRTConnect algorithm by base path planning (shown as a solid black line in the figure), and then redundant paths generated by the base path M are trimmed through a bidirectional path pruning algorithm to obtain a pruned path N (shown as a broken black line in the figure). The pruning principle of the bidirectional pruning is as follows: the starting point A and the target point F perform bidirectional trimming at the same time, the starting point A is trimmed to the point B, the target point F is trimmed to the point E, the point B is trimmed to the point C, the point E is trimmed to the point D, finally the point D is trimmed to the point C, 5 times of trimming is performed, each trimming strategy is a quick heuristic trimming method, namely, connection is performed from the starting point of trimming to turning points on the basic path M until collision is encountered, and the trimming is completed until the previous point is traced back. According to the path planning method of the surgical robot, the algorithm of combining the double-tree rapid expansion random tree algorithm and the bidirectional path pruning algorithm is adopted, so that a pruning path can be accurately and efficiently obtained. Because the bidirectional path pruning algorithm is similar to the double-tree rapid expansion random tree algorithm search bidirectional pruning strategy, the double algorithm combines the planned movement path of the tail end 112 of the mechanical arm, so that the calculation is accurate and efficient, the total length of the path is effectively shortened, and the movement process of the mechanical arm 111 is smoother.

In some embodiments, each sub-path of the needle insertion path (including the first needle insertion path, the second needle insertion path, and the third needle insertion path) may employ an algorithm that combines a dual-tree fast-expansion random tree algorithm with a bi-directional path pruning algorithm. In some embodiments, each sub-path of the exit path (including the first exit path, the second exit path, and the third exit path) may employ a two-tree fast-expansion random tree algorithm in combination with a bi-directional path pruning algorithm.

In the present description, the path planning algorithm of the mechanical arm is actually planned in a three-dimensional space, and for convenience of explanation of the principle, the path planning of a two-dimensional plane is adopted for display.

In some embodiments, when the surgical robotic device system 1000 is operating, the processing device 400 may acquire control signals and computationally determine starting and target points in the path planning algorithm described above based on the control signals.

In some embodiments, the control signals are control data or control instructions sent by an external device (e.g., terminal 130) to surgical robotic device system 1000. In some embodiments, the control signal may be used to instruct a joint on the robotic arm 112 to rotate to cause the robotic arm 112 to move the robotic arm tip 112 to a desired position. In some embodiments, the control signal may include data such as location information or time information. In some embodiments, the processing device 400 may obtain the control signal from the terminal 300 through the network 200. In some embodiments, processing device 400 may calculate a planned path that determines a starting point to a target point based on the control signal.

In some embodiments, the method for planning the movement path of the robot arm end 112 may also be calculated by using other planning methods, such as one or more algorithms including a fast-expansion random tree algorithm, a unidirectional pruning algorithm, and the like.

In some embodiments, when planning any of the above sub-paths using a collision-free path planning algorithm, the planning may be based on pose information of the end instrument 112, three-dimensional information of the medical device lumen 120, and three-dimensional position information of the patient.

The three-dimensional information of the patient includes three-dimensional spatial position information of the patient while lying on the mobile patient table 130, which remains substantially stationary during the entire interventional procedure. In some embodiments, the patient may be modeled in three dimensions by photographing or three-dimensional scanning, etc., to store the three-dimensional information of the patient in a storage device.

In some embodiments, a three-dimensional database of information for the patient may be pre-stored in the storage device 500. For example, the three-dimensional information database of the patient may be based on the mapping relationship between the information such as the height, weight, and three-dimensional size of the human body and the three-dimensional information. The three-dimensional information of the patient can be input by the doctor through the input device on the terminal 300, and the processing device 400 can extract the mapped three-dimensional information from the storage device 500 as the three-dimensional information of the patient in the current operation upon receiving the control signal. For example, the terminal 300 may be provided with an input interface, and a doctor may input information such as height, weight, and three-dimensional size of the patient on the interface, and the processing device 400 may obtain, from the terminal 300 through the network 200, a control signal input by a user, so as to call out three-dimensional information of the patient adapted to the interventional operation. The specific collision-free path planning algorithm in this embodiment is described above, and will not be described here again.

FIG. 10 is an exemplary flow chart of a path planning method of a surgical robot according to other embodiments of the present disclosure; FIG. 11 is a schematic view of the end jaw retraction of a surgical robot according to further embodiments of the present disclosure; fig. 12 is a schematic view from an elevational perspective of a surgical robot with end jaws retracted according to further embodiments of the present disclosure; fig. 13 is a schematic view of the end jaw retraction direction of a surgical robot according to further embodiments of the present disclosure; FIG. 14 is a schematic diagram illustrating the success of planning a first backoff path based on a joint straight line interpolation algorithm according to further embodiments of the present disclosure; FIG. 15 is a schematic diagram illustrating a failure to plan a first backoff path based on a joint straight-line interpolation algorithm according to other embodiments of the present disclosure; fig. 16 is a schematic diagram of a manual drag operation for a surgical robot with a failed end jaw back-off plan according to some embodiments of the present disclosure; fig. 17 is yet another exemplary flow diagram of a path planning method of a surgical robot according to other embodiments of the present disclosure. Another embodiment of the path planning method for the surgical robot will be described in detail with reference to fig. 10 to 17.

As shown in fig. 10, further embodiments of the present disclosure provide a path planning method for a surgical robot, and a flow 4000 of the path planning method may include the following steps. The process 4000 may be performed by a processing device (e.g., the processing device 400). For example, the flow 4000 may be implemented as a set of instructions (e.g., an application program) stored in a memory external to, for example, the storage device 500, a surgical robot (e.g., the surgical robot 110), and accessible by the path planning system. The processing device may execute the set of instructions and, when executing the instructions, may configure it to perform flow 4000. The operational schematic of flow 4000 presented below is illustrative. In some embodiments, the process may be accomplished with one or more additional operations not described above and/or one or more operations not discussed. In addition, the order in which the operations of flow 4000 are illustrated in FIG. 10 and described below is not intended to be limiting.

At step 4100, pose information of the end instrument 112 and three-dimensional information of the medical device lumen 120 are acquired. In some embodiments, step 4100 may be performed by processing device 400 or information acquisition module 2100.

At step 4200, an internal safety point 121 within medical device lumen 120 is determined based on pose information of end instrument 112 and three-dimensional information of medical device lumen 120. In some embodiments, step 4200 may be performed by processing device 400 or determination module 2200.

Step 4300, a path of movement of end instrument 112 may be planned based on internal safety point 121. In some embodiments, step 4300 may be performed by processing device 400 or determination module 2300.

Steps 4100-4300 are the same as steps 3100-3300, and are described in detail above and are not repeated here.

Step 4400, planning a first retraction path of the end jaw 1121 based on a joint straight interpolation algorithm. In some embodiments, step 4400 may be performed by processing device 400 or determination module 2300.

To prevent uncontrolled interference and collision between the end jaws 1121 and the surgical instrument 1122 left on the patient's body after the end of the insertion, the initial path of the retraction path of the end jaws 1121 may allow the end jaws 1121 to move in a direction opposite the holder opening. As shown in fig. 11, a distal end of the robotic arm 111 is connected to a terminal jaw 1121, the terminal jaw 1121 being adapted for holding a surgical instrument 1122. In some embodiments, the end clamp 1121 may include two clamps that are symmetrical left and right, which when closed may be used to clamp the surgical instrument 1122, and when separated may be used to unclamp the surgical instrument 1122. After the interventional puncture is completed (i.e., the surgical instrument 1122 is inserted into the patient at a predetermined angle), the two grippers of the end jaw 1121 are opened, and the end jaw 1121 is withdrawn in the opposite direction to the opening of the grippers, so that collision between the end jaw 1121 and the surgical instrument 1122 left on the patient can be avoided. As shown in fig. 12, when the end jaw 1121 withdraws from the first retreat path, the gripper posture of the end jaw 1121 remains unchanged, and the position of the end jaw 1121 remains horizontally moved.

In some embodiments, referring to fig. 6, a back-off point 125 may be determined on the path between puncture point 124 to the IS point. The path between the puncture point 124 to the back-off point 125 is the first back-off path.

When the joint straight line interpolation algorithm plans the first retraction path of the end jaw 1121, the position of the retraction point 125 needs to be determined first. The calculation principle of the first backoff path is as follows: as shown in fig. 13, the retraction direction of the end jaw 1121 is the direction opposite to the projection direction of the X-axis of the coordinate system of the end instrument 112 in the horizontal plane, and the retraction distance of the end jaw 1121 is set to dis. Let the transformation matrix T of the coordinate system of the end jaw 1121 with respect to the robot base coordinate system before the end jaw 1121 is retracted be:

The transformation matrix T _BaseToClutch represents a homogeneous transformation matrix of the coordinate system of the end jaw 1121 with respect to the robot base coordinate system before the end jaw 1121 performs the first retreat path. Wherein a _x、a_y、a_z represents the projected components of the x-axis unit vector of the end jaw 1121 coordinate system onto the x, y, z axes of the robot base coordinate system; similarly, b _x、b_y、b_z represents the projected components of the y-axis unit vector of the end jaw 1121 coordinate system onto the x, y, z axes of the robot base coordinate system; c _x、c_y、c_z represents the projected components of the z-axis unit vector of the end jaw 1121 coordinate system onto the x, y, z axes of the robot base coordinate system; and p _x、p_y、p_z represents the x, y, z values of the origin of the coordinate system of the end jaw 1121 in the robot base coordinate system.

The horizontal plane, that is, the XOY plane of the robot-based coordinate system, the retreat distance of the end jaw 1121 in the X direction and the Y direction is:

The transformation matrix T of the end jaw 1121 coordinate system with respect to the robot base coordinate system after the end jaw 1121 performs the first retreat path is:

The transformation matrix T _BaseToClutch' represents a homogeneous transformation matrix of the coordinate system of the end jaw 1121 with respect to the robot base coordinate system after the end jaw 1121 performs the first retreat path. Wherein ,a_x、a_y、a_z、b_x、b_y、b_z、c_x、c_y、c_z、p_x、p_y、p_z is the same as above.

The object joint angle data of the end clamping jaw 1121 can be obtained by inverting the T _BaseToClutch' after the end clamping jaw 112 is retracted, the calculated object joint angle data is the position of the retracted point 125, and then the puncture point 124 is used as the starting point and the retracted point 125 is used as the target point to perform path planning on the first retracted path.

In some embodiments, as shown in fig. 14, the first escape path may be planned using a joint straight line interpolation algorithm (i.e., moveJ algorithm) that may connect a straight line path from a start point (i.e., the puncture point 124) to a target point (i.e., the escape point 125), if there is no obstacle between the start point and the target point, the first escape path planning is successful, and if there is an obstacle between the start point and the target point, the first escape path planning is failed.

In step 4500, when the first retracted path planning of the end jaw 1121 fails, a warning operation is performed. In some embodiments, step 4500 may be performed by the processing device 400 or the alert module 2400.

Since MoveJ algorithm has low planning success rate under the condition of complex environment, warning operation can be executed for the condition of failure of the first back-off path planning. In some embodiments, the alert operation may include issuing an alert signal, e.g., the alert signal may be issued through an output device of the terminal 300. The warning signal can comprise the forms of text information on the display, voice warning information broadcasted by the loudspeaker, flashing signal lamps on the display and the like.

In some embodiments, after receiving the alert signal, the user (e.g., doctor) may use a manual needle withdrawal method to solve the problem of the first back-off path planning failure. For example, when the first retraction path planning fails, the user may manually drag the mechanical arm 111 through the guidance of the interface prompt, and the separation of the end jaw 1121 and the surgical instrument 1122 is completed, so that the needle retraction is realized, that is, the movement of the first retraction path is manually completed by the user.

In some embodiments, as shown in fig. 16, the user may manually drag the end jaw 1121 from the puncture point 124 to any collision-free back-off point 125, and then the user may start the path planning system again by inputting a control signal through the terminal 300, and the end jaw 1121 may autonomously plan a moving path to the OS point with the back-off point 125 as a starting point. After the end jaw 1121 is moved to the point OS, a subsequent workflow is engaged, ensuring that the procedure is performed.

In some embodiments, the workflow of flow 4000 may also be as shown in FIG. 17. The end instrument 112 automatically plans a path from Home point 123 to OS point 122, a surgical instrument 1122 IS installed at OS point 122, the end instrument 112 automatically plans a path to IS point 121 within the treatment device cavity 120, and then the end instrument 112 automatically plans a path to puncture point 124, the surgical instrument 1122 performs an interventional procedure, and a first retraction path of the end jaw 1121 IS planned using a joint straight interpolation algorithm. When the first retraction path is successfully planned, the end clamping jaw 1121 automatically moves to a retraction point 125, the end clamping jaw 1121 continues to automatically plan a path to move to an OS point 122, then a next surgical instrument 1122 is installed, the next surgical instrument 1122 enters the treatment device cavity 120 again for interventional operation, and the cycle is performed until all preset surgical instruments 1122 are punctured into the body of the patient; when the first retraction path planning fails, the end jaw 1121 is moved to the retraction point 125 by manual dragging by the user, then the end jaw 1121 again initiates the automatic planning path movement to the OS point 122, then the next surgical instrument 1122 is installed to again perform the next round of interventional procedure, or the operation is completed, and the end jaw 1121 is retracted to the Home point.

It should be noted that the description of the process 4000 is merely for illustration and description, and is not intended to limit the application scope of the present disclosure. Various modifications and changes to the flow 4000 may be made by those skilled in the art under the guidance of this specification. However, such modifications and variations are still within the scope of the present description.

The present specification also provides a surgical robotic device comprising at least one processor and at least one storage device. The at least one memory device is configured to store instructions that, when executed by the at least one processor, implement the path planning method of the surgical robot described in any of the embodiments of the present specification.

Possible benefits of embodiments of the present description include, but are not limited to: (1) The path planning method of the surgical robot provided by the specification adopts a strategy of a safety transition point, wherein the safety transition point is used for carrying out transition of the whole path by using an external safety point outside a treatment equipment cavity and an internal safety point inside the treatment equipment cavity on the path of the whole real-time puncture surgical workflow, so that the path is relatively fixed, and the path planning can be faster and more accurate; (2) The whole moving path of the end instrument is divided into a plurality of sub-paths, part or all of the sub-paths can be subjected to path planning by adopting a collision-free path planning algorithm, and the sectional path planning can lead the path planning to be more accurate; (3) The sub-paths except the first backoff path can adopt a double-tree rapid expansion random tree algorithm (RRTConnect algorithm) and add a path pruning algorithm, the path pruning algorithm adopts a method for bidirectional heuristic pruning from a starting point and a target point to prune the path, and can prune irregular paths marked by RRTConnect algorithm rules, so that the planned path is ensured not to collide with an environmental object and is smooth enough; (4) A joint straight line interpolation algorithm (MoveJ algorithm) is adopted for a first retreat path when the tail end clamping jaw retreats, and the path planned by the path planning algorithm is definite and unique and is applicable to path planning of the first retreat path (because the retreat path planning requires that the retreating direction of the mechanical arm is the opening opposite direction of the tail end clamping jaw); (5) And under the condition that the first backoff path planning fails, a manual backoff strategy can be adopted, and when the first backoff path planning fails, a user can guide the manual dragging of the mechanical arm through interface prompt, so that the separation of the tail end clamping jaw and the surgical instrument is completed, and backoff is realized.

It should be noted that, the advantages that may be generated by different embodiments may be different, and in different embodiments, the advantages that may be generated may be any one or a combination of several of the above, or any other possible advantages that may be obtained.

While the basic concepts have been described above, it will be apparent to those skilled in the art that the foregoing detailed disclosure is by way of example only and is not intended to be limiting. Although not explicitly described herein, various modifications, improvements, and adaptations to the present disclosure may occur to one skilled in the art. Such modifications, improvements, and modifications are intended to be suggested within this specification, and therefore, such modifications, improvements, and modifications are intended to be included within the spirit and scope of the exemplary embodiments of the present invention.

Meanwhile, the specification uses specific words to describe the embodiments of the specification. Reference to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic is associated with at least one embodiment of the present description. Thus, it should be emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions in this specification are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the present description may be combined as suitable.

Likewise, it should be noted that in order to simplify the presentation disclosed in this specification and thereby aid in understanding one or more inventive embodiments, various features are sometimes grouped together in a single embodiment, figure, or description thereof. This method of disclosure does not imply that the subject matter of the present description requires more features than are set forth in the claims. Indeed, less than all of the features of a single embodiment disclosed above.

In some embodiments, numerical parameters set forth in the specification and claims are approximations that may vary depending upon the desired properties sought to be obtained by the individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and employ a method for preserving the general number of digits. Although the numerical ranges and parameters set forth herein are approximations that may be employed in some embodiments to confirm the breadth of the range, in particular embodiments, the setting of such numerical values is as precise as possible.

Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments of this specification. Other variations are possible within the scope of this description. Thus, by way of example, and not limitation, alternative configurations of embodiments of the present specification may be considered as consistent with the teachings of the present specification. Accordingly, the embodiments of the present specification are not limited to only the embodiments explicitly described and depicted in the present specification.

Claims

1. A path planning method of a surgical robot, applied to an interventional procedure in a medical equipment cavity (120), characterized in that the surgical robot (110) comprises a mechanical arm (111) and an end instrument (112) connected to the distal end of the mechanical arm (111); the path planning method comprises the following steps:

acquiring pose information of the end instrument (112) and three-dimensional information of the medical device lumen (120);

Determining an internal safety point (121) located within the medical device lumen (120) based on pose information of the end instrument (112) and three-dimensional information of the medical device lumen (120);

-planning a movement path of the end instrument (112) based on the internal safety point (121).

2. The path planning method of a surgical robot according to claim 1, characterized in that acquiring pose information of the end instrument (112) comprises: pose information of the end instrument (112) is determined based on the interventional procedure information and the interventional procedure site of the patient.

3. The path planning method of a surgical robot of claim 1, wherein the path planning method further comprises: determining an external safety point (122) located outside the medical device lumen (120) based on pose information of the end instrument (112) and three-dimensional information of the medical device lumen (120);

The end instrument (112) comprises an end jaw (1121) and a surgical instrument (1122) held by the end jaw (1121);

planning a path of movement of the end instrument (112) based on the internal safety point (121) comprises: -planning a movement path of the end jaws (1121) based on the inner safety point (121) and the outer safety point (122).

4. The path planning method of a surgical robot according to claim 1, characterized in that planning the movement path of the end instrument (112) based on the internal safety point (121) comprises:

Dividing the path of movement of the end instrument (112) into a plurality of sub-paths based on the internal safety point (121);

At least one sub-path is planned based on a collision-free path planning algorithm.

5. The path planning method of a surgical robot of claim 4, further comprising a path optimization algorithm; the collision-free path planning algorithm comprises a double-tree rapid expansion random tree algorithm; the path optimization algorithm comprises a path pruning algorithm.

6. The path planning method of a surgical robot of claim 4, further comprising: acquiring three-dimensional position information of the patient;

The planning at least one sub-path based on the collision-free path planning algorithm comprises: the at least one sub-path is planned using the collision-free path planning algorithm based on pose information of the end instrument (112) and three-dimensional information of the medical device lumen (120) and three-dimensional position information of the patient.

7. A path planning method of a surgical robot according to claim 3, characterized in that the path planning method further comprises: planning a first retreat path of the terminal clamping jaw (1121) based on a joint straight line interpolation algorithm;

The planning of the movement path of the end instrument (112) based on the internal safety point (121) comprises:

-planning an exit path of the end jaw (1121) based on the internal safety point (121) and the first exit path.

8. The path planning method of a surgical robot of claim 7, further comprising: when planning a first retreat path of the end jaw (1121) based on the joint straight line interpolation algorithm fails, a warning operation is performed.

9. A path planning system for a surgical robot, comprising: the system comprises:

the information acquisition module is used for acquiring pose information of the tail end instrument (112) and three-dimensional information of the medical equipment cavity (120);

A determination module for determining an internal safety point (121) located within the medical device lumen (120) based on pose information of the end instrument (112) and three-dimensional information of the medical device lumen (120);

-a planning module for planning a movement path of the end instrument (112) based on the internal safety point (121).

10. Surgical robot device, characterized in that it comprises at least one processor and at least one storage device for storing instructions which, when executed by the at least one processor, implement a path planning method of a surgical robot according to any of claims 1-8.