KR101913586B1 - Dental image registration method, apparatus, and recording medium thereof - Google Patents

Abstract

The present invention relates to a tooth image matching method, an apparatus for the same, and a recording medium for recording the same. According to the present invention, a plurality of view images each corresponding to different directions for tooth image data are generated and provided, thereby increasing user convenience by minimizing a position movement and rotation operation of an object by a user when matching a tooth image and improving the efficiency of implant planning and surgical guide design.

Description

TECHNICAL FIELD [0001] The present invention relates to a tooth image registration method, a device for the same, and a recording medium on which the image registration method,

The present invention relates to a tooth image registration, an apparatus therefor, and a recording medium on which the image is registered. More particularly, the present invention relates to a tooth image registration method for registering tooth images based on image features, an apparatus therefor, and a recording medium will be.

When a scene or an object is photographed at a different time or viewpoint from a computer vision, images according to different coordinate systems are obtained. Image matching refers to a process for displaying such different images in one coordinate system. Through the image matching, it is possible to confirm the correspondence relationship between images obtained through different measurement methods.

In the digital design of a surgical guide, the image is usually subjected to image registration between CT data and scan data before entering the implant planning stage. The accuracy of the image matching is very important for the subsequent procedure because the matched image is the foundation of the implant planning work which determines the safe and optimal implant placement position by grasping the bone tissue and neural tube position.

In order to match a tooth image, a user needs to input a plurality of points, which is a reference of image matching. According to the conventional method, the user must find and input an appropriate point while changing the direction or angle of the tooth image.

As described above, when moving a video object to a desired position, a virtual operation means such as an arc-ball or a trackball is utilized. However, according to such an operation means, a large amount of time is required because a plurality of drag operations or click operations are required until the user moves in a desired direction, and a user who is uncomfortable in handling the operation means takes an unintended direction There is a problem that a lot of trial and error must be performed. In addition, it is very inefficient because the user must repeat the manipulation operation for finding the appropriate point by rotating the tooth image object each time the matching is performed.

Due to the problems of the related art as described above, it takes a lot of time and cumbersome to match the teeth image, which is a factor that delays the overall digital design process and deteriorates the efficiency.

The present invention has been proposed in order to solve the problems of the related art that it takes a time unnecessary for the image matching and causes inefficiency by inputting the reference point of the image matching while directly changing the direction or angle of the object in the tooth image matching A method of matching a teeth image, an apparatus for the same, and a recording medium on which the method is recorded.

According to an aspect of the present invention, there is provided a tooth image matching method for matching two tooth images, the method comprising the steps of: Generating a plurality of view images respectively corresponding to the plurality of view images; Generating a plurality of view images corresponding to the different directions of the second tooth image data based on the image coordinate system information of the second tooth image data; Displaying the view image of the first tooth image data and the second tooth image data on a divided screen; Receiving a feature point serving as a reference for image matching through the user input unit in the plurality of view images; And performing matching of the first tooth image data and the second tooth image data based on the input minutiae points.

Here, the different directions may include at least one of a left direction, a right direction, and a front direction.

According to another aspect of the present invention, there is provided a tooth image matching apparatus for matching two teeth images, the apparatus comprising: a first tooth image matching unit for matching the first tooth image and the second tooth image data based on the image coordinate system information of the first tooth image data and the second tooth image data; A view image generating unit generating a plurality of view images corresponding to different directions with respect to the data and the second tooth image data; A display unit for displaying the plurality of view images for the first tooth image data and the second tooth image data on a divided screen; A user input unit for inputting a feature point serving as a reference of image matching in the plurality of view images; And an image matching unit for matching the first tooth image data and the second tooth image data based on the input minutiae points.

At this time, the view image generating unit may generate the view image so that the scales of the first tooth image data and the second tooth image data correspond to each other.

The tooth image matching apparatus may further include a feature extraction unit extracting a candidate point suitable for the feature point or a candidate region suitable for positioning the feature point based on the first tooth image data and the second tooth image data, , The view image generating unit may determine the direction of the view image based on the position of the candidate point or the candidate region.

Here, the display unit may display the candidate points or candidate areas extracted through the image feature extracting unit on the view image, thereby helping the user to determine the location of the feature points.

The tooth image matching apparatus may further include a direction reference storage unit for storing reference information for determining a direction of the view image, and the view image generating unit may generate a view image based on the reference information stored in the direction reference storage unit, It is possible to generate an image.

In addition, the display unit may display the view images in the same direction among the view images of the first tooth image data and the second tooth image data at positions corresponding to each other, and the display unit displays the first tooth image data and the second tooth image data, And the result of matching of the second tooth image data can be displayed.

Meanwhile, one of the CT image, the scan image, and the magnetic resonance image may be applied to the first tooth image data and the second tooth image data, respectively.

As described above, according to the present invention, it is possible to increase the convenience of the user by minimizing the object manipulation by the user in the tooth image registration, and to improve the efficiency of the implant planning and surge guide design have.

1 is a block diagram showing a configuration of a tooth image matching apparatus according to an embodiment of the present invention;
2 is a flowchart of a tooth image matching method according to an embodiment of the present invention; And
3 is an example of a view image generated by the tooth image matching apparatus according to an embodiment of the present invention and a screen displaying a result of image matching.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings and the inventor is not limited to the concept of terminology for describing his or her invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

The tooth image matching apparatus according to the present invention performs matching between two tooth images, i.e., first tooth image data and second tooth image data. Here, the first and second tooth image data are images having different coordinate systems due to the reasons such as being acquired through the heterogeneous sensor or obtained at different points of time, and each of them is a computerized tomography (CT) image, 2-D images, 3-D images, and 3-D images taken through various types of equipment including a scan image through a model scanner or an intra-oral scanner, a magnetic resonance image (MRI) Or more multidimensional images.

1 is a block diagram of a tooth image matching apparatus 100 according to an embodiment of the present invention. 1, a tooth image matching apparatus 100 according to an embodiment of the present invention includes a user input unit 10, a display unit 20, a view image generation unit 30, a direction reference storage unit 40, A feature extraction unit 50, and an image matching unit 60.

The user input unit 10 is implemented by a variety of input devices such as a mouse, a keyboard, and a button in order to receive various inputs required by the user in the tooth image matching process.

The display unit 20 displays a menu screen, a graphical user interface (GUI), a result screen, and the like for receiving various information. The display unit 20 may be implemented by various display devices, The input unit 10 and the display unit 20 may be integrated into one device.

The view image generating unit 30 acquires the image coordinate system information of the first tooth image data and the image coordinate system information of the second tooth image data and generates a plurality of image coordinate data for each of the first tooth image data and the second tooth image data, And generates a view image. Here, the view image is an image of an object viewed from a predetermined direction or angle. For example, the view image may include a left side image, a right side image, a front image, and the like, Direction, a bottom direction, and the like, and may include images according to various directions and angles. Each view image generated corresponds to a different direction.

As described above, the view image generating unit 30 processes the first tooth image data and the second tooth image data based on the image coordinate system information of each tooth image data to generate a plurality of view images at different view points in different directions . At this time, the view image of the first tooth image data and the view image of the second tooth image data are generated in the same direction. For example, when it is assumed that three view images are generated from the left side image, the right side image, and the front side image with respect to the first tooth image data, the view image for the second tooth image data is also displayed on the left side, right side, A corresponding view image is generated.

On the other hand, the view image generating unit 30 adjusts the scales of the first tooth image data and the second tooth image data based on the image coordinate system information of the first tooth image data and the second tooth image data, The view image can be generated so that the scales correspond to each other.

In order to generate the plurality of view images for the first tooth image data and the second tooth image data, the view image generating unit 30 needs to determine the specific view direction and the number of view images, (40) and the image feature extraction unit (50).

First, the direction reference storage unit 40 stores reference information about a view direction for the view image generation unit 30 to generate a view image. The direction reference storage unit 40 may specify and store the view direction so that a uniform view image is generated with respect to all the teeth image data on the left side, the right side, the front side, etc. However, Or may store specific criteria regarding the view direction according to categories such as the patient's age, sex, oral characteristics, and the like. For example, a criterion for the view direction may be provided such that the age of the patient is so small that some of the teeth have not yet been removed, or that if a part of the teeth of the patient is lost, the area where the teeth are present is included.

The view image generating unit 30 may determine a specific direction based on the reference information stored in the direction reference storage unit 40 and generate a view image according to the determined direction. At this time, the number of view images generated according to the determined direction will also be determined.

The view image generating unit 30 may acquire various information in cooperation with the electronic chart in which the patient's personal information or the medical care information is input in order to determine a specific view direction satisfying the criteria stored in the direction reference storage unit 40 Or the user input unit 10, or may process the tooth image data to acquire the direct information.

The image feature extraction unit 50 analyzes the first tooth image data and the second tooth image data and extracts a candidate region suitable for locating a candidate point or a feature point suitable as a reference point for image matching.

For example, the image feature extracting unit 50 extracts, from the image data, an edge, which is a part where the brightness value of the pixel suddenly changes in one direction, or a corner where the brightness value changes abruptly in all directions Corner), it is possible to extract an appropriate candidate point or candidate region based on the image matching standard. The image feature extraction unit 50 may extract candidate points or candidate regions through various image processing algorithms based on structure, shape analysis, and gray scale in the image.

The view image generating unit 30 may determine the specific direction of the view image to be generated and the number of view images considering the positions and the number of the candidate points or candidate regions extracted through the image feature extracting unit 50 . For example, assuming that a point near the second molar is transposed to the candidate point through the image feature extraction unit 50 and the points near the second molar on both sides are extracted, the view image generation unit 30 is a view in which the upper candidate point appears well, Three images of the right side direction can be generated as a view image.

As described above, the view image generation unit 30 determines the view direction in which the candidate points or candidate regions extracted according to the extracted specific positions and the number of candidate points or candidate regions are best represented, and the appropriate number of view images to be generated And the scale of the view image to be generated together with it. For example, when adjacent candidate points or candidate regions are close to each other, a view image in which a specific portion is enlarged can be generated so that the user can easily identify the corresponding portion. If the distance between the candidate point and the candidate region is relatively long, The view image can be generated by adjusting the scale so as to view the part.

The tooth image matching apparatus 100 according to an embodiment of the present invention may take only one of the orientation reference storage unit 40 and the image feature extraction unit 50 as described above. , 50, or by selecting information according to one configuration, the number of view directions or view images can be determined. Alternatively, the information according to one configuration can be used preferentially, and in the case where the configuration can not be used, another configuration can be activated. For example, the tooth image matching apparatus 100 generates a view image based on the reference information stored in the direction reference storage unit 40, and when the reference information can not be applied, the image feature extraction unit 50 Can be controlled to be activated.

The first and second tooth image data generated through the view image generation unit 30 and the view image of the second tooth image data are displayed on a screen divided through the display unit 20. [

The user can input minutiae serving as criteria for matching the first tooth image data and the second tooth image data in each view image displayed on the display unit 20 through the user input unit 10. [

The user inputs a portion of the view image of the first tooth image data and the view image of the second tooth image data, which are considered to be the same position, as feature points. For example, the user can input the minutiae in the tooth area that appears relatively clearly in the first tooth image data and the second tooth image data. On the other hand, although the positions of the minutiae are not particularly limited, it is preferable to input the minutiae so that the minutiae are uniformly distributed in the entire image without concentrating on any one part of the image. The user may input at least one or more minutiae points for all the displayed view images, but the minutiae points may be input only in some view images.

At this time, the display unit 20 displays the view images along the same direction among the respective view images of the first tooth image data and the second tooth image data at positions corresponding to each other, so that the user can easily grasp the same part in the two image data .

The image matching unit 60 performs matching between the first tooth image data and the second tooth image data on the basis of the feature point input to the view image through the user input unit 10. At this time, the matching can be performed by various matching algorithms based on the characteristics of the image.

The display unit 20 displays the matching result of the tooth image data so that the user can check the matching result. When the matching result is displayed, the display unit 20 provides a mark that quantitatively grasps the accuracy of the image matching result by, for example, matching out of the matching image or displaying a relatively inaccurate portion with different colors, The accuracy of the matching can be grasped objectively.

In the meantime, the tooth image matching apparatus 100 stores the matching image according to the result of matching the first tooth image data and the second tooth image data in a memory (not shown), and then, in a simulation for establishing an implant procedure plan And can be utilized by loading.

Then, information about candidate points or candidate regions extracted through the image feature extraction unit 50 is stored and displayed together with the view image, thereby guiding the user to input more appropriate points when the user inputs the feature points And may also assist the user in reselecting the feature point by providing stored information even when the registration result is required to be modified after the image registration is completed.

2 is a flowchart illustrating a matching process according to a tooth image matching method according to an embodiment of the present invention. Hereinafter, an organic operation process of the configuration of the tooth image matching apparatus 100 of FIG. 1 will be described with reference to FIG.

When the first tooth image data and the second tooth image data to be subjected to the image matching through the image equipment are acquired, the view image generating unit 30 generates an image in which the region where the feature point, The appropriate view direction and the number of view images for the first tooth image data and the second tooth image data are determined (S10, S20). In this case, if the orientation of the patient is different from that of the image coordinate axis of the image data, a step of correcting the slope of the image data and the like may be preceded before determining the number of view images and the number of view images.

On the other hand, the specific view direction and the number of view images are determined based on the reference information stored in the direction reference storage unit 40 or the positions of the candidate points or candidate regions extracted through the image feature extraction unit 50, And the like. For reference, the view direction and the number for generating a view image are applied to the first tooth image data and the second tooth image data in the same manner.

For example, when the view direction is directly specified in the direction reference storage unit 40, the view image generating unit 30 can generate a view image according to the view direction, When the view direction is defined, patient information such as age, sex, and race, and patient's medical information such as missing part or diagnosis name of the tooth are utilized or a given tooth image data is directly processed to obtain the first tooth image data and the second tooth image A specific view direction can be determined after first determining whether data satisfies a predetermined condition.

As another example, when the positions of the candidate points or candidate regions extracted through the image feature extraction unit 50 are relatively densely arranged on one side, the view image generation unit 30 generates a plurality of The view image can be generated by concentrating the candidate points or the candidate regions in one side direction in which the candidate points or candidate regions are concentrated as compared with the other directions. In addition, the rotation angle difference or scale between each view image can be determined differently according to the distance between each candidate point and the candidate region. for example. The closer the distance between the candidate points is, the smaller the rotational angle difference between each view image can be determined, and the scale of the view image can be determined such that the corresponding portion is enlarged so that the vicinity of the adjacent candidate point can be clearly distinguished.

When the direction, number, and scale of the view image are determined as described above, the view image generation unit 30 generates the view image based on the image coordinate system information, the direction, number, and scale determined for the first tooth image data and the second tooth image data A corresponding plurality of view images are generated and displayed (S30, S40).

When the user inputs a feature point to the view image through the user input unit 10, the image matching unit 60 aligns the two tooth image data based on the inputted feature point (S50, S60). As described above, matching can be performed by various matching algorithms that perform matching based on image characteristics.

The image matching result is displayed on the display unit 20 so that the user can confirm the image matching result (S70). At this time, a quantitative marking that can objectively grasp the matching accuracy can be added to the result display window by changing the color according to the matching degree . This can help the user to determine whether the matching result needs to be modified.

FIG. 3 shows an example of a view image and a result of image matching generated by the tooth image matching apparatus 100 according to an embodiment of the present invention. The first tooth image data includes an oral scan image, a second tooth image data Shows an example of a CT image.

Referring to FIG. 3, the area A is an area where the view images 301, 302, and 303 are displayed for the first tooth image data, and the area B is the view images 304, 305, and 306 for the second tooth image data. And the view images 301 to 306 along the left direction, the front direction, and the right direction in order from the left side are displayed on the divided screen.

The view images in the same direction among the respective view images of the first tooth image data and the second tooth image data are displayed at positions corresponding to each other. For example, a left-side view image 301 for the first tooth image data and a left-side view image 304 for the second tooth image data are displayed at positions corresponding to each other. Thus, the user can easily grasp the minutiae corresponding to the same position with respect to the two tooth image data. In addition, the user can more intuitively grasp the feature points by generating the view images so that the scales of the two teeth image data correspond to each other.

For reference, the points p1 to p6 input to the six view images 301 to 306 show examples of the feature points input through the user input unit 10.

Adjusting means for manually adjusting the direction of the displayed view image in case that the vicinity of the feature point to be inputted by the user is not clearly displayed in the generated view image or in the case where it is difficult to find an appropriate feature point in the view image Time) can be added.

The C region is a functional user interface (UI) 308 for adjusting the VOI (Volumn of Interest) in the matching image with the resultant image 307 showing the matching result. The resultant image 307, The matching degrees are displayed in different colors.

The user can evaluate the matching result based on the provided result image, modify the position of the minutiae point or add a new minutiae point if necessary to correct the previous image matching result.

In the above-described embodiment, the CT image and the oral scan image are matched with each other. However, the CT image-CT image, the scan image-scan image, the MRI image-CT image, It is needless to say that various combinations of the 3D-3D image and the 3D-3D image can be matched by the tooth image matching method according to the present invention.

Meanwhile, the tooth image matching method according to the embodiment of the present invention may be realized by a program that can be executed by a computer, and may be implemented by various recording media such as a magnetic storage medium, an optical reading medium, and a digital storage medium.

As described above, according to the tooth image matching method of the present invention, an apparatus therefor, and a recording medium on which the tooth image is recorded, the user does not need to manually input the point as the reference for image matching while manually changing the direction or angle of the object. It is possible to increase the convenience of the user significantly compared with the technique, and accordingly, the efficiency of the implant planning and the design of the surge guide can be enhanced.

Implementations of the various techniques described herein may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or combinations thereof. Implementations may be implemented in a computer program product, such as an information carrier, e.g., a machine readable storage device, such as a computer readable storage medium, for example, for processing by a data processing apparatus, Apparatus (computer readable medium) or as a computer program tangibly embodied in a propagation signal. A computer program, such as the computer program (s) described above, may be written in any form of programming language, including compiled or interpreted languages, and may be stored as a stand-alone program or in a module, component, subroutine, As other units suitable for use in the present invention. A computer program may be deployed to be processed on one computer or multiple computers at one site or distributed across multiple sites and interconnected by a communications network.

Processors suitable for processing a computer program include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The elements of a computer may include at least one processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer may include one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks, or may receive data from them, transmit data to them, . ≪ / RTI > Information carriers suitable for embodying computer program instructions and data include, for example, semiconductor memory devices, for example, magnetic media such as hard disks, floppy disks and magnetic tape, compact disk read only memory A magneto-optical medium such as a floppy disk, a read only memory (ROM), a random access memory (RAM) Access memory (RAM), flash memory, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), and the like. The processor and memory may be supplemented or included by special purpose logic circuitry.

In addition, the computer-readable medium can be any available media that can be accessed by a computer, and can include computer storage media.

While the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the description of features that may be specific to a particular embodiment of a particular invention Should be understood. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various device components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and devices will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

It should be noted that the embodiments of the present invention disclosed in the present specification and drawings are only illustrative of specific examples for the purpose of understanding and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

10: user input unit 20: display unit
30: view image generation unit 40: direction reference storage unit
50: image feature extraction unit 60: image matching unit

Claims (11)

A tooth image matching method for matching two tooth images,
Generating a plurality of view images corresponding to different directions with respect to the first tooth image data based on the image coordinate system information of the first tooth image data;
Generating a plurality of view images corresponding to the different directions of the second tooth image data based on the image coordinate system information of the second tooth image data;
Displaying the view image of the first tooth image data and the second tooth image data on a divided screen;
Receiving a feature point serving as a reference for image matching through the user input unit in the plurality of view images; And
And performing matching of the first tooth image data and the second tooth image data based on the input minutiae,
In the step of displaying the view image on the divided screen, the plurality of view images are displayed at different positions on the screen,
The generating of the view image may include extracting a candidate point suitable for the feature point or a candidate region for the feature point based on the first tooth image data and the second tooth image data, And the direction, number and scale of the view image to be generated based on the position of the candidate region, the distance between the candidate points or the candidate region, the patient information and the tooth loss region are determined.
The method according to claim 1,
Wherein the different directions include at least one of a left direction, a right direction, and a front direction.
A computer-readable recording medium having recorded thereon a program for executing the method for matching teeth images according to claim 1 or 2.
A tooth image matching apparatus for matching two tooth images,
A feature extractor for extracting a candidate point suitable as a reference point for image matching based on the first tooth image data and the second tooth image data or a candidate area suitable for positioning the feature point;
Generating a plurality of view images corresponding to different directions for the first tooth image data and the second tooth image data based on the image coordinate system information of the first tooth image data and the second tooth image data, An image generation unit;
A display unit for displaying the plurality of view images for the first tooth image data and the second tooth image data on a divided screen;
A user input unit for inputting a feature point serving as a reference of image matching in the plurality of view images; And
And an image matching unit for matching the first tooth image data and the second tooth image data based on the input minutiae,
The view image generation unit generates the view image based on the position of the candidate point or the candidate region extracted through the image feature extraction unit, the distance between the candidate points or the candidate region, the direction of the view image to be generated based on the patient information and the tooth loss region, The number, and the scale,
Wherein the display unit displays the plurality of view images at different positions on the screen, respectively.
5. The method of claim 4,
Wherein the view image generating unit generates the view image so that the scales of the first tooth image data and the second tooth image data correspond to each other.
delete 5. The method of claim 4,
Wherein the display unit displays the candidate point or candidate region extracted through the image feature extraction unit on the view image.
5. The method of claim 4,
Further comprising a direction reference storage for storing reference information for determining a direction of the view image,
Wherein the view image generating unit generates the view image based on the reference information stored in the direction reference storage unit.
5. The method of claim 4,
Wherein the display unit displays the view images along the same direction among the view images of the first tooth image data and the second tooth image data at positions corresponding to each other.
5. The method of claim 4,
Wherein the display unit displays the matching result of the first tooth image data and the second tooth image data.
5. The method of claim 4,
Wherein the first tooth image data and the second tooth image data are one of a CT image, a scan image, and a magnetic resonance image, respectively.

Images