CN110766666B

CN110766666B - Skeleton fault outline large sample library generation method based on region segmentation and GAN model

Info

Publication number: CN110766666B
Application number: CN201910953409.8A
Authority: CN
Inventors: 何坤金; 陈义仁; 朱新成; 金梦; 陈正鸣
Original assignee: Changzhou Campus of Hohai University
Current assignee: Changzhou Campus of Hohai University
Priority date: 2019-10-09
Filing date: 2019-10-09
Publication date: 2022-11-04
Anticipated expiration: 2039-10-09
Also published as: CN110766666A

Abstract

The invention discloses a method for generating a large sample library of a skeleton fault outline based on region segmentation and a GAN model. The method comprises the following steps: constructing a template with regional characteristics and hierarchical attributes; step two: constructing an initial contour sample library based on template area mapping; step three: and expanding the outline sample library based on the morphological feature information and the GAN model. The invention provides a scientific and reasonable generation method of a skeleton fault outline sample library aiming at the problem of 3D model repair of damaged skeleton in the clinical fracture case of orthopedic surgery, and the large sample library constructed based on the method can quickly and accurately carry out intelligent repair on the damaged skeleton through a computer technology and a machine learning technology, thereby providing a computer-assisted section for the operation treatment of bone surgeons, and having important practical application significance and application prospect in clinical orthopedics.

Description

Skeleton fault outline large sample library generation method based on region segmentation and GAN model

Technical Field

The invention belongs to the technical field of digital orthopedics and computer aided design, and particularly relates to a method for generating a large skeleton fault outline sample library based on region segmentation and a GAN model.

Background

The main purpose of modern digital orthopedic research is to combine the advanced digital technology and computer technology with orthopedics clinic closely, and provide a computer assistant section for orthopedics doctors to diagnose orthopedics diseases, treat fractures by surgery, etc. Mature commercialized software can conveniently reconstruct a three-dimensional model of a skeleton from medical tomographic image data of a patient interest part obtained by an imaging department, and is convenient for a doctor to formulate a preoperative planning scheme, personalized implant design and the like. However, for a fracture case, the reconstructed three-dimensional model is a few damaged bone fragments, and how to repair the damaged bone fragments efficiently and restore the original appearance of the individual is still a challenge problem to be solved in the technology. The existing damaged bone repairing methods mainly comprise a geometric splicing method, a mirror image repairing method, a template registration deformation method and the like, and the existing repairing technology still has a plurality of defects due to the complexity and randomness of the fracture.

In recent years, with the re-rise and vigorous development of artificial intelligence and machine learning technologies, researchers in various industries are favored, and even many countries are rising to the future technical strategy planning decision level. Therefore, artificial intelligence has gradually penetrated into research in the medical health industry, such as intelligent diagnosis, personalized health intelligent guidance, intelligent medical treatment, and the like. At present, the research of artificial intelligence and machine learning technology in the field of digital orthopedics is mainly focused on the aspect of medical image segmentation, and the research of repairing damaged bones is not involved.

The method is based on the intelligent repair of the clinically obtained damaged bone from the view point of the fault outline by using a machine learning means, and for this reason, the key point is that a large amount of sample data is required as a support. However, medical data relates to factors such as patient privacy, ethics, policy and regulations, and the amount of data that can be collected is quite limited.

The main contributions of the invention are: firstly, a rule for constructing fault contour sample data based on the existing 3D skeleton sample model is provided; secondly, a method for generating a large amount of sample data based on the existing fault contour sample data is provided, and large sample data support is provided for the intelligent repair of the damaged bone in the later period.

Disclosure of Invention

The invention provides a scientific and reasonable generation method of a skeleton fault outline sample library aiming at the problem of 3D model repair of damaged skeleton in the clinical fracture case of orthopedic surgery, and the large sample library constructed based on the method can quickly and accurately carry out intelligent repair on the damaged skeleton through a computer technology and a machine learning technology, thereby providing a computer-assisted section for the operation treatment of bone surgeons, and having important practical application significance and application prospect in clinical orthopedics.

In order to achieve the above purpose, the technical solution adopted by the present invention is to provide a method for generating a large sample library of a bone fracture profile based on region segmentation and a GAN model, which mainly comprises the following steps:

the method comprises the following steps: and constructing a template with regional characteristics and hierarchical attributes.

Step 1a: and (5) building a template. And calculating a geometric mean model of the acquired 3D bone model sample as a template.

Step 1b: and defining the characteristic information. And defining related characteristic information on the template according to the anatomical morphological characteristics, topological structure characteristics and medical semantics of the skeleton.

The characteristic information refers to symbolic information capable of remarkably reflecting bone morphology change, topological structure change or medically significant regions, mainly comprises some characteristic points (including the highest (left) point and the lowest (right) point of a bone) or characteristic surfaces, is a mark mainly used for region segmentation, and aims to reflect that the topological structure or the geometric shape of a fault contour between different regions has remarkable change. The definition and selection of the feature information should be considered based on the normal direction of the fault plane.

For some characteristic feature points, a plane parallel to the fault plane is made through the feature point, which is also called a feature plane. The feature information of the region segmentation described below is a feature plane of the marker region segmentation, and includes a feature plane constructed by feature points and a defined feature plane.

The fault contour generated by cutting the bone by the feature face is called a feature contour, and the feature contour can be classified into a previous region or a next region adjacent to the feature contour.

Step 1c: and (5) dividing the region. The template is region-segmented based on the feature information defined in step 1 b.

The bone part between two adjacent characteristic surfaces belongs to a region, and the distance between the two characteristic surfaces is called the height or the length of the region, and is called the height in the following without loss of generality.

Step 1d: and arranging the number of layers of fault cutting. And (3) setting a corresponding number of fault cutting layers for each segmented region in the step 1c according to the anatomical morphological change characteristics of the bone in the region.

The number of cutting layers of the region refers to the number of fault contour samples generated in the region in the subsequent stage. Because the geometrical form of the profile of the fault in some areas is changed greatly, a larger layer value can be set so as to fully embody the diversified characteristic information of the profile; and in some areas, the geometrical shape change of the fault outline is smaller and basically similar, and a smaller layer value is set so as to reduce redundant characteristic information.

The template with the regional characteristics and the hierarchical attributes, which is constructed in the step, enables the fault profile constructed according to the template to have the following characteristics: on one hand, topological structures in the regions are consistent and geometric forms are similar, topological structures among the regions are different, and the difference of the geometric forms is large; on the other hand, the generation of the fault contour is performed in a mode of isomorphism in the region (uniform cutting and equal spacing) and isomorphism between the regions (different cutting spacing in different regions), and the morphological characteristics of different regions are fully embodied.

Step two: and constructing an initial contour sample library based on the template region mapping.

Step 2a: and (5) template registration deformation. Directly searching the significant characteristic information in the sample; and for part of characteristic information which is difficult to search, respectively carrying out registration deformation on all bone 3D model samples and the template obtained in the step one.

The significant characteristic information refers to information which can be directly searched in the model through an algorithm, such as the highest point, the lowest point and the like; because the bone morphological structure is complex, it is difficult to directly search for partial feature information (such as interfaces between certain regions), in this case, a template region feature mapping method is adopted, and the template can perform region segmentation in a manual interaction manner.

And step 2b: and (4) mapping the region characteristics. And mapping the region characteristic information in the template and the layer number information of each region to each 3D model sample so as to perform region segmentation on the 3D model sample.

The regional feature information mapping maps only feature information that cannot be directly searched in the sample in step 2 a.

And step 2c: and (5) constructing a fault cutting surface. And (c) constructing a group of mutually parallel fault cutting planes with equal intervals for each region for each 3D model sample in the step 2b according to the region information and the layer number information of each region, wherein the number of the cutting surfaces is equal to the layer number set by the corresponding region.

And step 2d: and generating an initial fault outline sample library. And 3, performing fault cutting on all the 3D model samples by using the fault cutting plane constructed in the step 2c, and generating a fault profile sample library with region information and hierarchical attributes.

3D model of marker acquisition sample library collection is S = { S ₁ ,S ₂ ,…,S _n F = { F) set of feature planes ₁ ,F ₂ ,…,F _m+1 The set of areas divided by bones is omega = { omega = ₁ ,Ω ₂ ,…,Ω _m H = { H), the height set of each region is H = { H = ₁ ,H ₂ ,…,H _m And the set of cutting layers arranged in each area is N = { N = { (N) } ₁ ,N ₂ ,…,N _m }. Wherein n is the number of skeleton three-dimensional model samples, m is the number of divided regions, and region omega _i Corresponding height of H _i Number of cutting layers of N _i (i =1,2, \8230;, m), and the set Ω satisfies the condition:

the step of constructing an initial fault profile sample set C is as follows:

step 2d \ u 01: initialization, setting

Setting the number m of the areas and a cutting layer number set N;

step 2d \u02: for 3D model sample S _i (i =1,2, \8230;, n), constructing a feature surface set F, and dividing S by F _i Generating a region set omega, calculating a height set H, and using the heightSymbol F _j (j =2,3, \8230;, m) cutting S _i Generating feature profiles

Then step 2d _03and step 2d _04are performed;

step 2d \u03: for region omega _k (k =1,2, \8230;, m), a set of spacing H is constructed _k /N _k And parallel to the plane of the fault plane, calculating them and omega _k Generating a subset of the area profile

Step 2d \u04: device for placing

The profile sample in the profile sample library constructed by the above steps has three attributes of a region, a level and a sample number, wherein the region attribute refers to a region number to which the profile sample belongs, the level attribute refers to a layer number of the profile sample in the region, and the sample number attribute refers to a sample number of the profile sample in the layer to which the profile sample belongs. Thus, a profile sample can be identified by a triple "(region number, layer number, sample number).

Step three: and (4) expanding a contour sample library based on morphological feature information and the GAN model.

Step 3a: and (4) training the GAN model and expanding a sample library. For the initial contour sample library constructed in step 2d, a GAN (generic adaptive Networks) model of a corresponding region is trained by using sample sets in different regions, and a large number of new samples with the same type and different local forms are generated by using the trained GAN model to expand the region fault contour sample library.

The GAN model is a generative confrontation network model.

First, step 3a _01and step 3a _02are executed for the tomographic contour sample data of each region; then, step 3a _03is executed.

Step 3a _01: discretizing the outline data;

uniformly resampling n points for each contour sample, carrying out topology uniformization and normalization processing, and forming the n points into a 2 n-dimensional vector (x) ₁ ,y ₁ ,x ₂ ,y ₂ ,…,x _n ,y _n ) Wherein (x) _i ,y _i ) Is the coordinates of the ith point (i =1,2,3, \8230;, n);

step 3a _02: training a GAN model;

first, random noise data having a mode length of 1 is input to a generative model

Generating a new sample x, training a discrimination model together with the original sample set, and outputting 1 if x is the original sample; if the sample is generated for generating the model, 0 is output. The generation model and the discrimination model mainly learn the distribution characteristics (continuity characteristics of contour points) and the geometric characteristics (curvature change characteristics and tangent vector change characteristics of points) of the contour points in the original sample data. Then, alternately training the generating model and the judging model, and stopping training when the output probability of the judging model is about 0.5 no matter the original sample or the sample x generated by the generating model;

step 3a \u03: and expanding the contour sample set. Firstly, generating a large number of regional new sample contour sets by using the model trained in the step 3a \u02; then, calculating the similarity average value of the geometric form of each newly generated sample in each region and the morphological characteristics of the original contour set of each layer in the region to which the sample belongs, classifying the similarity average value into the level with the highest similarity average value, and randomly distributing if a plurality of levels meet the condition; finally, the generated new contour sample set is merged into the original contour sample library to form an expanded fault contour large sample library;

and step 3b: and (4) redundant sample elimination. Removing redundant samples with completely identical morphological characteristics from the profile sample subsets of each region in the fault profile sample library expanded in the step 3 a; if the sample size of a certain area or a certain layer is not large enough, the steps are repeated until the satisfaction is reached.

The redundant samples refer to a plurality of samples which have the same shape and can be overlapped by rigid transformation, and only one sample is reserved for the samples.

The invention has the beneficial effects that: firstly, the anatomical morphological characteristics, medical semantics and topological structure characteristics of bones are comprehensively considered to carry out region segmentation and fault segmentation on a 3D model sample, and an initial fault outline sample library with region characteristics and hierarchical attributes is generated; secondly, training a GAN model to expand an initial contour sample library and eliminating redundant samples with completely identical geometric forms so as to ensure the completeness of the samples as much as possible.

Drawings

FIG. 1 is a flow chart of the operation of the method for generating a large sample library of bone fault contours in the present invention.

FIG. 2 is a schematic diagram of the construction of a template with region features according to the present invention.

FIG. 3 is a schematic diagram of a template region division and initial fault contour generation method according to the present invention.

Fig. 4 is a schematic diagram illustrating the characteristic information of the femoral region in the present invention.

FIG. 5 is a schematic diagram of a template mapping-based 3D model sample region segmentation method according to the present invention.

FIG. 6 is a schematic diagram of discretization of a fault profile in the present invention.

FIG. 7 is a schematic diagram of a fault profile sample library expansion method based on a GAN model in the invention.

FIG. 8 is a schematic diagram of a method for determining a hierarchical attribute of a new sample in a certain area according to the present invention.

Detailed Description

Example (b):

the invention is further explained by taking the femur as an example and combining the concrete implementation steps with reference to the attached drawings. As shown in fig. 1, a method for generating a large sample library of a bone fracture contour based on region segmentation and a GAN model includes the following steps:

the method comprises the following steps: a template with regional characteristics and hierarchical attributes is constructed.

Step 1a: and (5) constructing a template. For the collected 3D bone model samples, a geometric mean model thereof is calculated as a template, as shown in fig. 2.

As shown in fig. 3 (a), the feature information for performing the region segmentation on the femur is mainly divided into two types, i.e., feature points and feature planes, where the feature points include: the femoral head highest point, the greater trochanter highest point, the femoral head lowest point, the medial condyle lowest point, the lateral condyle lowest point and the intercondylar notch initial point; the characteristic face mainly includes: femoral neck starting surface, large and small trochanter interface, near-dry interface (interface of proximal femur and femoral shaft), dry-distal interface (interface of femoral shaft and distal femur), etc.; in addition, a plane passing through the feature point and parallel to the fault plane is also referred to as a feature plane.

As shown in fig. 3 (a), the feature plane constructed in step 1b is used to divide the femur template into 10 regions, wherein the feature information corresponding to each region is shown in fig. 4.

Step 1d: and arranging the number of layers of fault cutting. And (3) setting a corresponding fault cutting layer number for each region in the step 1c according to the anatomical morphological change characteristics of the bone in the region.

Step 2a: and (5) registering and deforming the template. Direct search is carried out on the significant characteristic information (such as the highest point of the femoral head); for part of the feature information which is difficult to search, all the bone 3D model samples are respectively registered and deformed with the template obtained in the first step with reference to (a) to (b) in fig. 5.

And step 2b: and (5) mapping the regional characteristics. The region feature information in the template and the layer number information of each region are mapped to each 3D model sample, thereby performing region segmentation thereon, as shown in (c) of fig. 5.

And step 2c: and (5) constructing a fault cutting surface. Referring to fig. 3 (b), for each 3D model sample in step 2b, according to the region information and the layer number information of each region, a set of tomographic cutting planes with equal spacing and parallel to each other is constructed for each region, and the number of cutting planes is equal to the layer number set for the corresponding region.

And step 2d: and generating an initial fault outline sample library. Referring to (c) of fig. 3, all the 3D model samples are tomographically cut using the tomogram constructed in step 2c, and a tomogram profile sample library with region information and hierarchical attributes is generated.

Step three: and expanding the outline sample library based on the morphological feature information and the GAN model.

Step 3a: and (4) training the GAN model and expanding a sample library. And (3) respectively training a GAN model of an affiliated area by using the sample sets in different areas for the initial contour sample library constructed in the step (2 d), and generating a large number of new samples with the same type and different local forms by using the trained GAN model to expand the area sample set.

First, step 3a _01and step 3a _02are executed for the tomographic contour sample data of each region; then, step 3a _03is performed.

Step 3a \u01: the contour data is discretized.

Referring to fig. 6, n points are uniformly resampled for each contour sample and subjected to topology uniformization and normalization processing, and the n points are formed into a 2 n-dimensional vector (x) ₁ ,y ₁ ,x ₂ ,y ₂ ,…,x _n ,y _n ) Wherein (x) _i ,y _i ) Is the coordinates of the ith point (i =1,2,3, \8230;, n).

Step 3a _02: and (5) training a GAN model.

Referring to FIG. 7, first, random noise data having a mode length of 1 is input to the generative model

Generating a new sample x, training a discrimination model together with the original sample set, and outputting 1 if x is the original sample; if it is a generative model productThe raw sample outputs 0. The generation model and the discrimination model mainly learn distribution features (continuity features of contour points) and geometric features (curvature change features and tangent vector change features of points) of the contour points in the original sample data. Then, the generative model and the discriminant model are alternately trained so that the training is stopped when the output probability of the discriminant model is in the vicinity of 0.5 for both the original sample and the sample x generated by the generative model.

Step 3a _03: and expanding the outline sample set. Firstly, generating a large number of regional new sample contour sets by using the model trained in the step 3a \u02; then, referring to fig. 8, calculating a similarity average value of the geometric form of each newly generated sample in each region and the morphological characteristics of each layer of original contour set in the region to which the sample belongs, classifying the similarity average value into a level with the highest similarity average value, and randomly distributing if a plurality of levels meet the condition; and finally, merging the generated new contour sample set into the original contour sample library to form an expanded fault contour large sample library.

And step 3b: and (4) redundant sample elimination. Removing redundant samples with completely same morphological characteristics from the profile sample subsets of each area in the fault profile sample library expanded in the step 3 a; if the sample size of a certain area or a certain layer is not large enough, the steps are repeated until the satisfaction is reached.

All that is not further described is prior art. The femur is used as an example only, and the basic ideas and methods of the present invention are equally applicable to the remaining bones, although the anatomy and topology of the other bones in the human body are different. A fault outline sample library aiming at a certain bone can be generated by slightly modifying the characteristics of the bone. In addition, the fault in the present invention may be any fault, such as an axial plane, a sagittal plane, a coronal plane, and the remaining fault planes.

The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle and the basic idea of the present invention.

Claims

1. A method for generating a large sample library of a skeleton fault outline based on region segmentation and a GAN model is characterized by comprising the following steps:

the method comprises the following steps: constructing a template with regional characteristics and hierarchical attributes;

step two: constructing an initial contour sample library based on template area mapping;

the second step comprises the following steps:

step 2a: template registration deformation;

directly searching the significant characteristic information in the sample; for part of feature information which is difficult to search, respectively carrying out registration deformation on all skeleton 3D model samples and the template with the regional features and the hierarchical attributes constructed in the step one;

and step 2b: mapping the regional characteristics;

mapping the region characteristic information and the layer number information of each region in the template constructed in the step one to each 3D model sample so as to perform region segmentation on the 3D model sample;

and step 2c: constructing a fault cutting surface;

constructing a group of fault cutting planes which are equal in spacing and parallel to each other for each region of each 3D model sample in the step 2b according to the region information and the layer number information of each region, wherein the number of the cutting surfaces is equal to the layer number set by the corresponding region;

step 2d: generating an initial fault contour sample library, performing fault cutting on all 3D model samples by using the fault cutting surface constructed in the step 2c, and generating a fault contour sample library with region information and hierarchy attributes;

step three: expanding a contour sample library based on morphological characteristic information and a GAN model;

the third step comprises the following steps:

step 3a: training a GAN model and expanding a sample library;

respectively training a GAN model of an affiliated area by using sample sets in different areas for the initial contour sample library in the step 2d, and generating a large number of new samples with the same type and different local forms by using the trained GAN model to expand the fault contour sample library in the area;

and step 3b: removing redundant samples;

and (3) removing redundant samples with completely identical geometric forms from the profile sample subsets of each region in the fault profile sample library expanded in the step (3 a).

2. The method for generating the large sample library of the bone fault contours based on the region segmentation and the GAN model as claimed in claim 1, wherein the first step comprises the following steps:

step 1a: constructing a template;

calculating a geometric mean model of the collected 3D skeleton model sample as a template;

step 1b: defining characteristic information;

defining related characteristic information on the template obtained in the step 1a according to the anatomical morphological characteristics, topological structure characteristics and medical semantics of the skeleton;

step 1c: dividing the region;

performing region segmentation on the template based on the feature information defined in the step 1 b;

step 1d: setting the number of layers for fault cutting;

and (3) setting a corresponding fault cutting layer number for each segmented region in the step 1c according to the anatomical morphological change characteristics of the bone in the region.