JP4703193B2 - Image processing device - Google Patents

Description

  The present invention relates to image processing for processing and displaying a three-dimensional medical image captured in time series by a medical image diagnostic apparatus such as an ultrasonic diagnostic apparatus, an X-ray CT apparatus, or an MRI apparatus. Relates to the device.

  In order to observe a medical image of a patient imaged by a medical image diagnostic apparatus such as an ultrasonic diagnostic apparatus or an X-ray CT apparatus, an image processing apparatus capable of image display and image processing is used. This image processing apparatus can observe a three-dimensional image in addition to displaying a two-dimensional image. In this observation, in order to compare the images, for example, the images can be easily compared by observing the images at the same observation angle and magnification rate.

  In addition, some image processing apparatuses that display a three-dimensional image can display a plurality of three-dimensional images. When displaying a plurality of three-dimensional images, based on shooting information and display parameters at the time of shooting. One that displays in synchronization is known (for example, Patent Document 1).

JP 2002-125937 A

  However, in the conventional image processing apparatus, time-series three-dimensional image data captured every predetermined time is not displayed in synchronization. For this reason, it is difficult to display a plurality of three-dimensional image data in synchronism in real time while scanning is performed by the medical image diagnostic apparatus, and images are captured by different medical image diagnostic apparatuses. It has been difficult to display the time-series three-dimensional image data superimposed.

  Further, in the conventional image processing apparatus, an area to be observed is extracted from the 3D image data during scanning, and the 3D images included in the extracted observation area are not synchronized and compared. . Therefore, when dealing with time-series three-dimensional image data, particularly when comparing a plurality of time-series three-dimensional image data, it is necessary to extract a region of the three-dimensional image data in each time phase. Since the amount of data is enormous, the operator has to spend a great deal of labor and time. This makes it difficult to compare time-series 3D image data before and after surgery, comparison with typical cases, and the like.

  As described above, in the image processing apparatus according to the prior art, time-series 3D image data cannot be compared with each other during scanning (real time), and only processing for the temporarily stored 3D image data is possible. Therefore, its application was limited.

  In addition, since a three-dimensional image included in a desired observation area has not been extracted during scanning (real time), it has been difficult to apply to intraoperative navigation systems such as robotic surgery.

  Further, when an ultrasonic diagnostic apparatus is used as a medical image diagnostic apparatus, for example, an apparatus that displays time-sequential tomographic image data synchronously using an electrocardiogram signal (ECG signal) by a stress echo method is known. Yes. However, in the conventional ultrasonic diagnostic apparatus, since the position of the tomographic image data depends on the position of the ultrasonic probe, when comparing a plurality of tomographic image data, the coordinate system of each tomographic image data is different. It was difficult to compare the tomographic image data of each other and compare them, and this alignment relied on the experience and intuition of doctors and engineers. Further, when displaying in synchronization using the ECG signal, a desired observation area is extracted from the tomographic image data, and the tomographic images included in the desired observation area are compared in real time, or lacking. Obtaining image data by interpolation processing has not been performed.

  The present invention solves the above-described problem, and the three-dimensional image data collected in time series by scanning based on the range of the observation area extracted from the time-series three-dimensional image data collected in advance. It is an object of the present invention to provide an image processing apparatus capable of extracting a desired observation region in real time from time to time. As a result, the display device displays three-dimensional image data collected in time series using the medical image diagnostic apparatus and time-series three-dimensional image data collected and stored in advance by the medical image diagnostic apparatus. It is an object of the present invention to provide an image processing apparatus capable of facilitating comparison of three-dimensional images included in a desired observation area every time when displayed and compared.

  It is another object of the present invention to provide an image processing apparatus capable of temporally synchronizing a plurality of three-dimensional image data by using biological information such as an electrocardiogram signal. As a result, an object of the present invention is to provide an image processing apparatus capable of easily comparing a plurality of three-dimensional image data.

  In addition, a plurality of three-dimensional images for comparing the display angle, magnification, and other display operations in the operation method of the displayed image by matching the orientation (coordinate system) of the three-dimensional image included in the extracted observation region By sharing display parameters such as image data and display parameters such as opacity (image opacity) used for image processing, it is easy to compare time-series 3D image data, It is an object of the present invention to provide an image processing apparatus that can easily confirm a therapeutic effect and can provide new medical information for assisting diagnosis and diagnosis.

  Further, since it becomes possible to extract a desired observation area in real time from the time-series three-dimensional image data during the scan, the time-series three-dimensional image data during the scan, the time series collected in the past Since it is possible to compare typical three-dimensional image data or an actual image of a subject, it can be applied to an intraoperative navigation system such as robotic surgery.

According to the first aspect of the present invention, the first time-series three-dimensional image data collected every predetermined time together with the biological information of the subject changing with time, and the biological information together with the biological information in advance every predetermined time. In response to the collected second time-series three-dimensional image data, based on the biological information, the first time-series three-dimensional image data and the second time-series three-dimensional data Time synchronization means for synchronizing image data, and 3D image data included in a second observation region that differs for each time phase from the second time-series 3D image data collected in advance, for each time phase The first observation region for the first time-series three-dimensional image data is obtained for each time phase based on the second observation region that is different for each time phase, and the first time It differs from time-series 3D image data for each time phase. Sequential three-dimensional image data included in the first observation region for each time phase, a region extracting means for extracting said first time-series three-dimensional image data and the second time-series 3D An image processing apparatus having display means for synchronously displaying three-dimensional image data extracted for each time phase from image data .

According to the first aspect of the present invention, desired time-series three-dimensional image data obtained by scanning is desired based on an observation region extracted from pre-collected time-series three-dimensional image data. It becomes possible to extract the three-dimensional image data included in the observation area in real time. By applying this invention, it becomes possible to easily compare three-dimensional images included in the observation region at each time. Furthermore, based on biological information that changes with time, it is possible to synchronize time-series 3D image data collected in advance with time-series 3D image data obtained by scanning. . By applying this, it becomes possible to compare three-dimensional images collected at the same time, and the comparison of images becomes easy. According to the second aspect of the present invention, the range of the observation area to be extracted following the change of the observation target is changed and applied to the time-series three-dimensional image data obtained by scanning. Thus, it becomes possible to extract a 3D image included in a range suitable for the time-series 3D image data. According to the invention described in claim 5 , even if the 3D image data is missing at the predetermined time, the 3D image data collected before and after the predetermined time is used for the predetermined time. By generating the three-dimensional image data by interpolation, it is possible to compare the three-dimensional images at the predetermined time. According to the invention described in claims 6 and 7 , by matching the coordinate system for each time, the time-series three-dimensional image data collected in advance and the time-series obtained by scanning It is possible to match the direction and position of the three-dimensional image data on the display device with time. By applying this invention, it becomes easy to compare three-dimensional images. In addition, according to the invention described in claims 8 and 9, it is easy to compare the three-dimensional images by applying the operation information or the contents of the image processing to the three-dimensional images at each time.

  As described above, three-dimensional image data included in a desired observation area is extracted from time-series three-dimensional image data in a so-called real-time during scanning, and time-series three-dimensional image data are obtained based on biological information. Are further synchronized, and further, the time-series three-dimensional image for comparing the display operation such as the orientation of the three-dimensional image included in the extracted observation region and the angle and magnification of the three-dimensional image displayed on the display device. By sharing the data with each other, it is possible to easily compare time-series three-dimensional image data, and to easily check changes in the affected area over time, treatment effects, and the like. This makes it possible to provide new information effective for diagnosis to a doctor or the like.

  In addition, by applying the present invention, time-series comparison of time-series three-dimensional image data collected in the past is facilitated. In addition, since time-series comparison of time-series three-dimensional image data collected by different types of medical image diagnostic apparatuses is possible, for example, changes over time in affected areas before and after surgery, treatment effects, etc. Confirmation can be facilitated. In addition, it is possible to easily compare the time-series three-dimensional image data of the subject with a typical case that changes with time. In this manner, new information effective for diagnosis can be provided to a doctor or the like.

  Furthermore, by applying the present invention, it becomes possible to extract an image included in the region to be observed and display it on the display device during the operation, so that the operator observes the temporal change of the organ and the like in real time during the operation. It becomes possible. Further, since it is possible to compare with a pre-operative time-series three-dimensional image during the operation, it is also possible to compare with an image or a real image collected in the past while performing treatment or the like.

[First Embodiment]
First, an image processing apparatus according to a first embodiment of the present invention will be described with reference to FIGS.

(Constitution)
The configuration of the image processing apparatus according to the first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a schematic configuration of a medical image system including an image processing apparatus according to the first embodiment of the present invention. The medical image system includes a medical image diagnostic apparatus 1, a position information collection unit 6, a biological information collection unit 7, an image processing device 20, and a storage device 40.

  The medical image diagnostic apparatus 1 is configured by, for example, an X-ray CT apparatus, an ultrasonic diagnostic apparatus, or the like. FIG. 1 shows an example of an X-ray CT apparatus. The irradiation unit 2 includes an X-ray source, a high voltage generator, an X-ray source control device, and the like, and emits an X-ray beam toward an imaging target that is a subject. The X-ray source generates so-called cone beam X-rays that spread in two directions, for example, a channel direction and a slice direction (the body axis direction of the subject). The detector 3 is composed of an X-ray detector that detects an X-ray beam irradiated from the irradiation unit 2 and transmitted through the subject. In the detector 3, for example, a plurality of X-ray detection elements are arrayed in two directions orthogonal to each other, thereby forming a two-dimensional X-ray detector.

  The data acquisition unit 4 (DAS) has data acquisition elements arranged in an array like each X-ray detection element of the detector 3, and scans the X-ray beam detected by the detector 3 with a scan control unit ( Data is collected in correspondence with the data collection control signal output by (not shown). This collected data becomes X-ray projection data.

  The image reconstruction unit 5 reconstructs image data by performing a back projection process on the X-ray projection data output from the data collection unit 4. This back projection method is the same as a known method. For example, reconstruction by a three-dimensional image reconstruction algorithm represented by a method called the Feldkamp method is performed, and X-ray absorption in a wide target region (volume) in the slice direction is performed. Coefficient three-dimensional distribution data (hereinafter referred to as volume data) is reconstructed. The reconstructed volume data is generated by a data processing device (not shown) such as a tomographic image of an arbitrary cross section, a projection image from an arbitrary direction, a three-dimensional surface image of a specific organ by a rendering process, based on an instruction from an operator. It is converted into so-called pseudo three-dimensional image data and sent to the image processing device 20.

  The position information collection unit 6 includes, for example, a three-dimensional position sensor that operates based on a magnetic or optical principle, collects three-dimensional position information of the subject (imaging target) P, and obtains the three-dimensional position information for each time. To the position control unit 33. That is, the position information collection unit 6 collects time-series three-dimensional position information and outputs it to the position control unit 33. As will be described later, anatomical feature points extracted from the three-dimensional image data are also set as the three-dimensional position information.

  The biological information collection unit 7 acquires biological information such as an electrocardiographic signal (ECG signal), myoelectric signal (EMG signal), or joint motion information of the subject during scanning with the medical image diagnostic apparatus 1. is there. For example, when an electrocardiograph is used, the biological information collection unit 7 detects an electrocardiogram signal (ECG signal) of the subject and converts the detected electrocardiogram signal (ECG signal) into a digital signal. An electrocardiogram signal (ECG signal) obtained at the same time as the time-series 3D image data is output to the time control unit 34 as supplementary information of a plurality of 3D image data collected every time, and further the biological information. It is stored in the storage unit 43. The myoelectric signal (EMG signal) is a signal obtained by measuring the action potential of the muscle from the skin surface, and the motion information of the joint is the joint when the subject's joint is intentionally moved. It shows the angle to be made, and when it is moved, it changes with time and becomes a different angle.

  The image processing apparatus 20 includes an operation input unit 21, an image display unit 22, and a control unit 30. The image display unit 22 displays a time-series three-dimensional image collected by the medical image diagnostic apparatus 1. The operation input unit 21 performs operations such as image processing, measurement, and region extraction on the time-series three-dimensional image displayed on the image display unit 22, and a signal corresponding to the operation is sent to the system control unit 31. Is output.

  The control unit 30 includes a system control unit 31, an image control unit 32, a position control unit 33, a time control unit 34, an area control unit 35, and an operation control unit 36. The system control unit 31 is connected to each unit of the image processing apparatus 20 and controls the operation of the entire image processing apparatus 20.

  The image control unit 32 controls time-series three-dimensional image data output from the medical image diagnostic apparatus 1. Specifically, the time-series three-dimensional image data output from the image reconstruction unit 5 is read during scanning by the medical image diagnostic apparatus 1 and is simultaneously stored in the image storage unit 41 of the storage device 40. The time-series three-dimensional image data collected in advance is read and output to the position control unit 33.

  The position control unit 33 includes a coordinate system of time-series 3D image data collected by the medical image diagnostic apparatus 1 and a coordinate system of time-series 3D image data stored in the image storage unit 41. Are the same. The position control unit 33 extracts time-series three-dimensional position information that is slightly different from time to time, such as markers and anatomical feature points displayed in the image, and each time based on the three-dimensional position information. Local coordinates are defined in the collected images, and the local coordinates of the three-dimensional images to be compared are matched. For example, when an auxiliary tool or the like is previously attached to the subject P at the time of imaging and imaging is performed in that state, a marker is displayed in the image. Therefore, the position control unit 33 uses the marker as three-dimensional position information and the local control unit 33 in each image Define a coordinate system. The anatomical feature point means a feature point on the skeleton as shown in FIG.

  Specifically, the position control unit 33 extracts 3D position information from the time-series 3D image being scanned, and further, 3D position information in the 3D image stored in the image storage unit 41. Is read. And the position control part 33 defines the local coordinate system of the three-dimensional image in each time from these time-sequential three-dimensional position information, and makes a local coordinate system correspond.

  Further, the position control unit 33 reads time-series three-dimensional position information of the subject (imaging target) P collected by the position information collecting unit 6 including the three-dimensional position sensor, and uses the three-dimensional position information as a reference. Alternatively, a local coordinate system may be defined in the images collected at each time to match the local coordinates of the images.

  The time control unit 34 reads the biological information collected by the biological information collection unit 7 and the biological information stored in the biological information storage unit 43. This biological information is information that changes with time, and corresponds to the above-described electrocardiogram signal (ECG signal), myoelectric signal (EMG signal), and the like. Then, the time control unit 34, based on the read biological information, time-series 3D image data obtained by scanning, and time-series 3D image data stored in the image storage unit 41, Synchronize. In other words, in order to display the 3D image data collected at the same time (time phase) on the image display unit 22 at the same time, it is obtained by scanning based on biological information as supplementary information of the 3D image data. Of the three-dimensional image data and the time-series three-dimensional image data collected in advance, the three-dimensional image data collected at the same time (time phase) are synchronized, and the image display unit 22 is synchronized. The displayed three-dimensional image data is displayed simultaneously.

  Furthermore, when the number of images per unit time (for example, per second) differs between the three-dimensional image data obtained by scanning and the three-dimensional image data collected in advance, an image with a small number of images per unit time Interpolation processing is performed on the data to synchronize the three-dimensional image data obtained by scanning with the three-dimensional image data collected in advance.

  The area control unit 35 extracts 3D image data included in a desired observation area from the 3D image data. For example, the region control unit 35 outputs a three-dimensional image within the range of a desired observation region among the three-dimensional images, and displays the three-dimensional image included in the observation region extracted by the image display unit 22. It is. In this embodiment, the area control unit 35 extracts 3D image data included in a desired observation area from previously collected time-series 3D image data stored in the area information storage unit 44. At this time, information indicating the observation area (extraction area information) is read, and the 3D image data included in the area corresponding to the extraction area information is extracted from the time-series 3D image data obtained by scanning. Do. This extracted area information is also time-series and varies from time to time.

  The observation area is designated by the operation input unit 21. For example, when an operator operates a mouse or the like provided in the operation input unit 21 to specify a region (range) in which an image displayed on the image display unit 22 is to be observed, the region control unit 35 An image included in the designated observation area (range) is extracted and displayed on the image display unit 22. Information (extraction region information) indicating the designated observation region is stored in the region information storage unit 44. In this manner, 3D image data included in a desired observation area is extracted from time-series 3D image data collected in advance.

  The operation control unit 36 reads operation information and image processing parameters stored in the operation information storage unit 45, and uses the operation information and image processing parameters as time-series three-dimensional image data obtained by scanning. To reflect. The operation information is information indicating an operation such as a rotation operation, a movement, or an enlargement performed on the image displayed on the image display unit 22. Further, the image processing parameters include an opacity that is a coefficient for determining the opacity of the image, a parameter (WL / WW) that is a coefficient for determining contrast, brightness, and the like. Operations such as rotational operations and image processing performed on the time-series 3D image data collected in advance are also performed on the time-series 3D image data being scanned, and the image display unit 22 Unify the rotation angle, size, opacity, contrast, etc. of the displayed image. In other words, on the screen of the image display unit 22, the observation angle, the enlargement ratio, the position, the opacity, the contrast, and the like of the three-dimensional image are unified to facilitate observation. The operation information and image processing parameters are also time-series, and differ from time to time.

  In addition, the operation control unit 36 not only reflects the operation information input from the operation input unit 21 on the time-series three-dimensional image data obtained by scanning, but also collects time-series 3 collected in advance. This is also reflected in the dimensional image data. That is, when operation information is input from the operation input unit 21, the operation control unit 36 displays time-series three-dimensional image data obtained by scanning displayed in the image display unit 22 and previously collected. The operation information is reflected in the time-series three-dimensional image data. For example, when information indicating a rotation operation is input from the operation input unit 21, the operation control unit 36 rotates time-series 3D image data obtained by scanning in the image display unit 22 and collects in advance. The time-series three-dimensional image data thus rotated is rotated in the same manner. Similarly, when the time-series 3D image data collected in advance is rotated, the operation control unit 36 also rotates the time-series 3D image data obtained by scanning. In this way, the observation angle, the enlargement ratio, etc. of the image displayed on the image display unit 22 are unified.

  Furthermore, when certain image processing is performed on the time-series three-dimensional image data obtained by scanning, the operation control unit 36 performs time-series three-dimensional image data collected in advance. To apply. For example, in order to unify the opacity, contrast, brightness, etc. of the image, the same parameters as the image processing parameters (opacity, etc.) applied to the time-series 3D image data obtained by scanning are set. The image processing is performed on the time-series three-dimensional image data collected in advance. In this way, the opacity and contrast of the image displayed on the image display unit 22 are unified.

  The image display unit 22 displays time-series three-dimensional image data processed by the image control unit 32, the position control unit 33, the time control unit 34, the region control unit 35, or the operation control unit 36.

  Further, the system control unit 31 may individually release the processing of the image control unit 32, the position control unit 33, the time control unit 34, the region control unit 35, and the operation control unit 36 in accordance with an instruction from the operation input unit 21. it can. That is, under the control of the system control unit 31, only certain processing by the position control unit 33 may be performed on certain image data, and other processing by the time control unit 34, the region control unit 35, etc. may not be performed. . Also, with the control of the system control unit 31, certain image data is processed by the position control unit 33, the time control unit 34, and the region control unit 35, and is not processed by the operation control unit 36. good. According to the image processing apparatus 20 according to this embodiment, it is possible to display an image on the image display unit 22 by not sharing all the processes as described above but sharing only the information of the designated process.

  The storage device 40 includes an image storage unit 41, a position information storage unit 42, a biological information storage unit 43, a region information storage unit 44, and an operation information storage unit 45.

  The image storage unit 41 stores time-series three-dimensional image data collected in advance by the medical image diagnostic apparatus 1 every predetermined time. For example, time-series three-dimensional image data collected by an X-ray CT apparatus, an ultrasonic diagnostic apparatus, an MRI apparatus, or the like, which is slightly different for each time, is stored. The medical image diagnostic apparatus 1 according to this embodiment is assumed to be an X-ray CT apparatus, but the image storage unit 41 includes other apparatuses such as ultrasonic waves in addition to image data collected by the X-ray CT apparatus. Image data collected by a diagnostic apparatus, an MRI apparatus, or the like may be stored. That is, not only image data collected by the same diagnostic apparatus but also image data collected by another diagnostic apparatus may be stored. For example, in order to compare images collected by different diagnostic apparatuses, image data collected by different diagnostic apparatuses is stored.

  The position information storage unit 42 stores time-series three-dimensional position information slightly different from time to time, which is represented by anatomical feature points or markers by an auxiliary tool previously attached to the subject at the time of imaging. ing. Alternatively, the three-dimensional position information of the subject P collected by the position information collecting unit 6 is stored.

  The biological information storage unit 43 stores biological information that changes with time, such as an electrocardiogram signal (ECG), a myoelectric signal (EMG), or joint motion information. This biological information is information collected from the subject P at the same time during the scan by the medical image diagnostic apparatus 1, and forms supplementary information of a plurality of three-dimensional image data collected every time.

  The area information storage unit 44 stores the range of the observation area when the 3D image data included in the desired observation area is extracted from the time-series 3D image data collected by the medical image diagnostic apparatus 1 in advance. The information (area extraction information) that is different for each time is stored.

  In the operation information storage unit 45, an operation that is performed on the screen of the image display unit 22 with respect to time-series three-dimensional image data collected in advance by the medical image diagnostic apparatus 1, and that varies with time. Information is stored.

  In the example shown in FIG. 1, an image processing apparatus 20, which is a characteristic part of this embodiment, is provided outside the medical image diagnostic apparatus 1 as an X-ray CT apparatus, and is sent from the medical image diagnostic apparatus 1. Although image data is processed, the present invention is not limited to this example. For example, the image processing apparatus 20 may be provided in the medical image diagnostic apparatus 1 (in the X-ray CT apparatus) and image processing may be performed inside the medical image diagnostic apparatus 1. In other words, the medical image diagnostic apparatus 1 (X-ray CT apparatus) itself may have the function of the image processing apparatus 20 and the medical image diagnostic apparatus 1 may perform image processing.

  Accordingly, the medical image diagnostic apparatus 1 and the image processing apparatus 20 are connected via a network or the like, and the image data collected by the medical image diagnostic apparatus 1 is transmitted to the image processing apparatus 20, and the image processing apparatus 20 Image processing may be performed, or the image processing apparatus 20 may be provided in the medical image diagnostic apparatus 1, and the medical image diagnostic apparatus 1 itself may perform image processing. Further, the storage device 40 may be provided in the medical image diagnostic apparatus 1 together with the image processing apparatus 20.

  In the example described above, an X-ray CT apparatus is used as the medical image diagnostic apparatus 1, but an ultrasonic diagnostic apparatus, an MRI apparatus, or the like may also be used. FIG. 2 shows an example in which an ultrasonic diagnostic apparatus is used as a medical image diagnostic apparatus. In the medical image system shown in the block diagram of FIG. 2, only the configuration of the medical image diagnostic apparatus is different. A medical image diagnostic apparatus 10 shown in FIG. 2 includes an ultrasonic diagnostic apparatus, and includes an ultrasonic probe 11, a transmission / reception unit 12, a signal processing unit 13, and a DSC (digital scan converter) 14.

  The ultrasonic probe 11 is a two-dimensional ultrasonic probe in which ultrasonic transducers that transmit and receive ultrasonic waves are arranged in a matrix (lattice). Then, ultrasonic waves are transmitted and received three-dimensionally by scanning, and three-dimensional data having a shape that spreads radially from the surface of the probe is received as an echo signal. Further, when a one-dimensional ultrasonic probe is used as the ultrasonic probe 2, three-dimensional data is collected by mechanically scanning the ultrasonic probe 2.

  The transmission / reception unit 12 includes a transmission unit and a reception unit. The transmission / reception unit 12 supplies an electrical signal to the ultrasonic probe 11 to generate an ultrasonic wave and receives an echo signal received by the ultrasonic probe 11.

  The transmission unit in the transmission / reception unit 12 includes a clock generation circuit, a transmission delay circuit, and a pulsar circuit (not shown). The clock generation circuit is a circuit that generates a clock signal that determines the transmission timing and transmission frequency of the ultrasonic signal. The transmission delay circuit is a circuit that performs transmission focus with a delay when transmitting ultrasonic waves. The pulsar circuit incorporates pulsars corresponding to the number of individual paths (channels) corresponding to each ultrasonic transducer, generates a drive pulse at a delayed transmission timing, and each ultrasonic transducer of the ultrasonic probe 12. To supply.

  The reception unit in the transmission / reception unit 12 includes a preamplifier circuit, an A / D conversion circuit, and a reception delay / addition circuit (not shown). The preamplifier circuit amplifies the echo signal output from each ultrasonic transducer of the ultrasonic probe 11 for each reception channel. The A / D converter circuit A / D converts the amplified echo signal. The reception delay / adder circuit gives a delay time necessary for determining the reception directivity to the echo signal after A / D conversion, and adds the delay time. By the addition, the reflection component from the direction according to the reception directivity is emphasized. A signal output from the receiving unit is referred to as RF data.

  The signal processing unit 13 includes a B mode processing unit, a Doppler processing unit, and a color mode processing unit. The RF data output from the transmission / reception unit 12 is subjected to predetermined processing in any of the processing units.

  The B-mode processing unit visualizes echo amplitude information and generates B-mode ultrasound raster data from the echo signal. Specifically, bandpass filter processing is performed on the RF data, and then the envelope of the output signal is detected, and compression processing by logarithmic transformation is performed on the detected data. In addition, processing such as edge enhancement may be performed. Data generated by the B-mode processing unit is referred to as B-mode ultrasonic raster data.

  The Doppler processing unit is composed of a phase detection circuit, an FFT operation circuit, and the like, extracts Doppler shift frequency components from RF data, and further performs FFT processing to generate data having blood flow information.

  The color mode processing unit visualizes the moving blood flow information and generates color ultrasonic raster data. Blood flow information includes information such as speed, dispersion, and power, and blood flow information is obtained as binarized information. Specifically, the color mode processing unit includes a phase detection circuit, an MTI filter, an autocorrelator, and a flow velocity / dispersion calculator. This color mode processing unit performs high-pass filter processing (MTI filter processing) to separate tissue signals and blood flow signals, and performs blood flow information such as blood flow velocity, dispersion, and power by autocorrelation processing. Ask for multiple points. In addition, non-linear processing for reducing and reducing tissue signals may be performed.

  A DSC 14 (Digital Scan Converter: digital scan converter) converts ultrasonic raster data into voxel data represented by orthogonal coordinates in order to obtain an image represented by an orthogonal coordinate system. The DSC 14 converts the signal-processed data represented by the scanning line signal sequence output from the signal processing unit 13 described above into coordinate system data based on the spatial information (scan conversion process). That is, the scanning method is converted by reading out in synchronization with the standard television scanning so that the signal sequence synchronized with the ultrasonic scanning can be displayed on the image display unit 22 of the television scanning method.

  The voxel data output from the DSC 14 is subjected to volume rendering processing by a volume rendering processing circuit (not shown) to generate three-dimensional image data. The three-dimensional image data is stored in the image storage unit 41 via the image control unit 32 and displayed by the image display unit 22.

  Even when an ultrasonic diagnostic apparatus is used as the medical image diagnostic apparatus 10, an image processing apparatus 20 is provided in the medical image diagnostic apparatus 10 (ultrasonic diagnostic apparatus), as in the above-described example of the X-ray CT apparatus. Image processing may be performed in the medical image diagnostic apparatus 10 (ultrasound diagnostic apparatus).

(Operation)
Next, the operation of the image processing apparatus according to the first embodiment of the present invention will be described with reference to FIGS. First, control for matching the coordinate system between time-series 3D image data obtained by scanning and time-series 3D image data collected in advance and stored in the image storage unit 41 This will be described with reference to FIGS. FIG. 3 is a flowchart showing, in order, coordinate transformation processing by the image processing apparatus according to the first embodiment of the present invention. FIG. 4 is a view for explaining coordinate conversion processing by the image processing apparatus according to the first embodiment of the present invention.

  First, time-series three-dimensional image data that is slightly different in time is collected by the medical image diagnostic apparatus 1 (10) including an X-ray CT apparatus or an ultrasonic diagnostic apparatus, and the image shown in FIG. 1 or FIG. The image data is stored in the image storage unit 41 via the control unit 32. The time-series three-dimensional image data is output from the image control unit 32 to the image display unit 22 via the system control unit 31, and the image display unit 22 displays the three-dimensional image data that is slightly different for each time. The Thereby, the operator can observe the three-dimensional image data as a moving image in a so-called real time. The time-series 3D image data stored in the image storage unit 41 is used as time-series 3D image data collected in advance (time-series 3D image data collected in the past). In particular, it may be referred to as “time-series three-dimensional image data A”.

  The medical image diagnostic apparatus 1 (10) collects new time-series three-dimensional image data that is slightly different from time to time. Here, the time-series three-dimensional image data currently obtained by scanning may be referred to as “time-series three-dimensional image data B” for convenience. The new time-series three-dimensional image data B is sequentially output from the medical image diagnostic apparatus 1 (10) to the image control unit 32 of the image processing apparatus 20.

  Then, when new time-series three-dimensional image data B is collected by scanning, the image control unit 32 reads time-series three-dimensional image data A collected in advance from the image storage unit 41. Then, the image control unit 32 outputs the time-series three-dimensional image data A collected in advance and the time-series three-dimensional image data B obtained by scanning to the position control unit 33.

  The position control unit 33 reads time-series three-dimensional position information slightly different from time to time in the time-series three-dimensional image data A collected in advance, which is stored in the position information storage unit 42 (step S01). The position control unit 33 reads time-series three-dimensional position information represented by anatomical feature points (feature points on the skeleton), for example, as shown in FIG. The three-dimensional position information represented by the anatomical feature points is slightly different from time to time.

  Further, the position control unit 33 extracts time-series three-dimensional position information slightly different from time to time in the time-series three-dimensional image data B obtained by scanning (step S02). For example, the position control unit 33 extracts anatomical feature points (feature points on the skeleton) from the time-series three-dimensional image data B obtained by the scan sent from the image control unit 32, and the anatomy is extracted. The characteristic feature point is set as three-dimensional position information. The three-dimensional position information represented by the anatomical feature points is also slightly different from time to time.

  Then, the position control unit 33 analyzes the anatomical feature points of the time-series 3D image data A collected in advance and the anatomical feature points of the time-series 3D image data B obtained by scanning. And determine whether or not they match (step S03). If the position control unit 33 determines that the anatomical feature points do not match (step S03, No), for example, a new anatomical feature point is obtained from the three-dimensional image data B obtained by scanning. Are matched with the anatomical feature points of the three-dimensional image data A collected in advance (step S04).

  When the position control unit 33 determines that the anatomical feature points match (step S03, Yes), the position control unit 33 defines a local coordinate system in each three-dimensional image data (step S05). For example, as shown in FIG. 4, when three anatomical feature points are extracted by the position control unit 33, a coordinate system represented by the three points is defined as a local coordinate system. FIG. 4A shows pre-collected 3D image data A and its anatomical feature points, and FIG. 4B shows 3D image data B obtained by scanning and its anatomy. The characteristic points are shown. And the position control part 33 defines a local coordinate system according to three anatomical feature points, as shown to Fig.4 (a), (b). When aligning a plurality of three-dimensional images, the local coordinates defined differ depending on the diagnostic part to be matched.

  Then, the position control unit 33 performs coordinate conversion on the local coordinate system of the three-dimensional image data B obtained by scanning, and matches the local coordinate system of the three-dimensional image data A collected in advance (step S06). For example, as shown in FIG. 4C, the position control unit 33 performs coordinate conversion, rotation, and further movement of the local coordinate system of the time-series three-dimensional image data B obtained by scanning. , Make the local coordinate system of both match. As a result, the three-dimensional image data B represented by the local coordinate system is also rotated, and the directions and positions of the three-dimensional image data A and the three-dimensional image data B coincide.

  Since the direction and position of both image data can be matched in this way, the time-series 3D image data B obtained by scanning and the time-series 3D image data A collected in advance are obtained. When the image is displayed on the image display unit 22, the direction and the position of both images coincide with each other, so that the comparison between both images is facilitated.

  Further, without using anatomical feature points as the three-dimensional position information, the three-dimensional position information of the subject P collected by the position information collecting unit 6 composed of a three-dimensional position sensor, or previously attached to the subject at the time of imaging. Alternatively, three-dimensional position information represented by a marker or the like imaged by the auxiliary tool may be used. In this case, the position controller 33 stores the three-dimensional position of the subject P collected by the three-dimensional position sensor in the time-series three-dimensional image data A collected in advance and stored in the position information storage unit 42. Information, 3D position information represented by markers, etc. are read.

  Further, the position control unit 33 receives the three-dimensional position information of the subject P collected by the three-dimensional position sensor in the time-series three-dimensional image data B obtained by scanning from the position information collecting unit 6, Reads three-dimensional position information represented by a marker or the like.

  Then, the position control unit 33 matches the positions and orientations of the markers and the like of the 3D image data A with the markers and the like of the 3D image data B. In this way, by adjusting the direction and position with reference to the three-dimensional position information such as the marker, the directions and positions of the three-dimensional image data A and the three-dimensional image data B coincide with each other. , Making the comparison easier.

  After the directions and positions of the three-dimensional image data are matched as described above, the three-dimensional image data is output to the time control unit 34. The time control unit 34 is preliminarily collected with time-series three-dimensional image data B obtained by scanning based on biological information such as an electrocardiogram waveform (ECG signal) collected by the biological information collection unit 7. The time-series three-dimensional image data A is synchronized. The control by the time control unit 34 will be described with reference to FIGS. FIG. 5 is a flowchart showing, in order, processing for displaying two time-series three-dimensional image data in synchronization by the image processing apparatus according to the first embodiment of the present invention.

  First, the time control unit 34 receives the three-dimensional image data A and the three-dimensional image data B output from the position control unit 33, and further stores the collected three-dimensional data stored in the biological information storage unit 43. Biometric information that is supplementary information of the image data A is read (step S10). Next, the time control unit 34 reads biological information, which is incidental information of the 3D image data B obtained by scanning, from the biological information collection unit 7 (step S11).

  When an electrocardiograph is used as the biological information collecting unit 7, since an electrocardiographic signal (ECG signal) of the subject (imaging target) P is collected, the electrocardiographic signal (ECG signal) becomes biological information. . Moreover, when an electromyograph is used, since an EMG signal is collected, the EMG signal becomes biometric information. In addition, biological information may be joint motion information indicating the movement of the subject's joint when the subject is intentionally moved.

  The time control unit 34 determines whether or not the biological information as the supplementary information of the three-dimensional image data A matches the biological information as the supplementary information of the three-dimensional image data B (step S12). When the information is different (No at Step S12), the time control unit 34 matches the biological information (Step S13). For example, when a myoelectric signal (EMG signal) is read as biological information of the three-dimensional image data A and an electrocardiogram signal (ECG signal) is read as biological information of the three-dimensional image data B, the biological information is different. Yes. In this case, the time control unit 34 reads an electrocardiogram signal (ECG signal) from the biological information storage unit 43 as the biological information of the three-dimensional image data A, for example, and matches the biological information.

  In this embodiment, a case where an electrocardiograph is used for the biological information collection unit 7 to collect an electrocardiogram signal (ECG signal) of the subject P will be described. When the biometric information matches (Yes in step S12), the time control unit 34 matches the time scales of the biometric information and matches the time (time phase) of the biometric information (step S14). Here, when an electrocardiogram signal (ECG signal) is used as biological information, a process for matching time scales will be described with reference to FIGS. FIG. 6 is a diagram showing an electrocardiogram signal (ECG signal). FIG. 7 is a diagram for explaining processing for synchronizing and displaying two time-series three-dimensional images by the image processing apparatus according to the first embodiment of the present invention.

  ECG1 shown in FIG. 6 is an electrocardiogram signal (ECG signal) collected at the same time when collecting time-series three-dimensional image data A collected in advance. ECG2 is an electrocardiogram signal (ECG signal) that is collected at the same time that time-series three-dimensional image data B obtained by scanning is collected. In this example, ECG1 has a shorter heartbeat interval than ECG2. As shown in FIG. 6, the electrocardiogram signal (ECG signal) has peaks named “P”, “Q”, “R”, “S”, “T”, “U”.

  The time control unit 34 uses ECG1 and ECG2 to match the time scales of the three-dimensional image data A and the three-dimensional image data B. For example, the time control unit 34 extracts only the R wave (peak R) of the ECG signal and matches the R wave (peak R), thereby matching the time scales. For example, as shown in FIG. 7, the ECG1 time axis is extended to ECG1 'so that the ECG1 R wave (peak R) matches the ECG2 R wave (peak R). Further, not only the R wave (peak R) but also other peaks may be used to match the time scales. For example, the time control unit 34 may set ECG1 'by extending the time axis of ECG1 so that each peak of ECG1 matches each peak of ECG2. As described above, the time scales of the three-dimensional image data A and the three-dimensional image data B are matched, the three-dimensional image data at the same time (time phase) are synchronized, and output to the image display unit 22 for image display. By simultaneously displaying on the unit 22, it is possible to compare three-dimensional images collected at the same time (time phase).

  Then, the time control unit 34 calculates the number of images per unit time (for example, per second) of the three-dimensional image data A and the three-dimensional image data B, respectively, and determines the image data with the smaller number of images per unit time. Interpolation processing is performed on the three-dimensional image data at a predetermined time (time phase). For example, when the number of images per unit time is larger in the three-dimensional image data B than in the three-dimensional image data A (Yes in step S15), an interpolation process is performed on the three-dimensional image data A, and the unit The number of images per time is matched with the three-dimensional image data B (step S17). On the other hand, when the number of images per unit time is larger in the three-dimensional image data A than in the three-dimensional image data B (No in step S15), interpolation processing is performed on the three-dimensional image data B, and the unit The number of images per time is matched with the three-dimensional image data A (step S16).

  Here, a case where the number of images per unit time of the three-dimensional image data A and the three-dimensional image data B is the same and the interpolation process is not performed will be described.

  As shown in FIG. 7, among the time-series three-dimensional image data A collected in advance, the three-dimensional image data collected at the peak P is set as the image data g11, and the three-dimensional image collected at the peak R. The image data is image data g12, and the three-dimensional image data collected at the peak T is image data g13. Subsequently, the 3D image data collected at the peak R is set as image data g14, the 3D image data collected at the peak T is set as image data g15, and the 3D image collected at the peak P is collected. The data is image data g16.

  Also, among the time-series 3D image data B obtained by scanning, the 3D image data collected at the peak P is set as the image data g21, and the 3D image data collected at the peak R is used as the image data g21. Image data g22, and the three-dimensional image data collected at peak T is image data g23. Subsequently, the 3D image data collected at the peak R is set as image data g24, the 3D image data collected at the peak T is set as image data g25, and the 3D image collected at the peak P is collected. The data is image data g26.

  Then, the 3D image data A and the 3D image data B having the same time (time phase) of ECG1 ′ and ECG2 are simultaneously displayed on the image display unit 22 so that the 3D image at the same time (time phase) is displayed. Data can be displayed and compared. By synchronously displaying in this way, it becomes easy to compare the time-series 3D image data A collected in advance with the time-series 3D image data B obtained by scanning.

  For example, by simultaneously displaying the image data g12 and the image data g22 collected at the peak R, that is, the image data g12 collected in advance in accordance with the display timing of the image data g22 obtained by scanning. By displaying, it is possible to compare images at the peak R.

  Next, the case where the number of images per unit time is larger in the three-dimensional image data B than in the three-dimensional image data A (step S15, Yes) will be described. In this case, since the three-dimensional image data A corresponding to the three-dimensional image data B collected at a certain time (time phase) is missing and does not exist, an interpolation process is performed on the three-dimensional image data A, The missing 3D image data at the time (time phase) is generated and matched with the 3D image data B at the time (time phase).

  The case of interpolating time-series three-dimensional image data A collected in advance will be described with reference to FIGS. 8 and 9 are diagrams for explaining image interpolation processing by the image processing apparatus according to the first embodiment of the present invention.

  As shown in FIG. 8, there is no image data g13 corresponding to the image data g23 in a state where the time scale is extended by the time control unit 34. That is, the three-dimensional image data A at time (time phase) t3 (peak T) is missing and does not exist. Therefore, when the three-dimensional image data A and the three-dimensional image data B are compared, it becomes impossible to compare the image data at time (time phase) t3 (peak T). Therefore, the time control unit 34 uses the image data g12 and g14 collected at times (time phases) t2 and t4 before and after the time (time phase) t3, and an image lacking the time (time phase) t3. Data g13 is obtained by interpolation.

  FIG. 9A shows the relationship between the voxel position and the voxel value in the volume data at times (time phases) t2 and t4. The horizontal axis indicates the voxel position of the volume data, and the vertical axis indicates the voxel value at the voxel position. For example, the voxel value at time (time phase) t2 at an arbitrary voxel position n is v2, and the voxel value at time (time phase) t4 is v4.

  FIG. 9B shows a voxel value at time (time phase) t2 and a voxel value at time (time phase) t4 at an arbitrary voxel position n. Since the voxel value v3 at the time (time phase) t3 is estimated to be between the voxel values v2 and v4, the user-specified function “v = f (t)” connecting the voxel values v2 and v4 The time control unit 34 calculates the voxel value v3 at time (time phase) t3. This function “v = f (t)” can be specified as a linear or non-linear function. Then, the time control unit 34 calculates voxel values at all voxel positions, and generates missing image data g13 at time (time phase) t3.

  Thus, by interpolating the image data and simultaneously displaying the 3D image data A and the 3D image data B on the image display unit 22, the 3D image data at the same time (time phase) is displayed. It becomes possible to compare.

  Next, the case where the number of images per unit time is larger in the three-dimensional image data A than in the three-dimensional image data B (step S15, No) will be described. In this case, since the 3D image data B corresponding to the 3D image data A collected at a certain time (time phase) does not exist, interpolation processing is performed on the 3D image data B, Image data at the time (time phase) is generated and matched with the three-dimensional image data A.

  A case of interpolating time-series three-dimensional image data B obtained by scanning will be described with reference to FIGS. 10 and 11 are diagrams for explaining image interpolation processing by the image processing apparatus according to the first embodiment of the present invention.

  For example, as shown in FIG. 10, it is assumed that the image data g23 corresponding to the image data g13 is missing and does not exist. That is, the three-dimensional image data B at the time (time phase) t3 is missing and does not exist. In this case as well, the three-dimensional image data B at time t3 may be generated by interpolation using the above-described interpolation method. However, since the three-dimensional image data B is currently collected by scanning, 3 at time t3 If the 3D image data at time t4 has not been collected before the 3D image data is displayed, the 3D image data at time t3 cannot be interpolated. That is, since there is no data after the image data g24 at the current time represented by time t0 in FIG. 10, at the stage of time t0, as described above, the three-dimensional image data at time t2 and time t4 is used. It is not possible to generate missing image data g23 at time (time phase) t3 by performing interpolation processing.

  Therefore, when interpolating the time-series three-dimensional image data B currently obtained by scanning, by shifting the display update timing of the image display unit 22, the interpolation processing is completed and image data is generated, Display the complemented image data.

  The image display update timing will be described with reference to FIG. The horizontal axis is time t. Data is collected by the data collection unit 4 every predetermined time by scanning of the medical image diagnostic apparatus 1, and volume data V1 is reconstructed one after another by the image reconstruction unit 5. For example, when interpolating three-dimensional image data between the volume data V1 and V2, when the volume data V1 and V2 are reconstructed, the time control unit 34 uses the volume data V1 and V2. The volume data V1 and the volume data V2 during the collection are interpolated and generated. When the volume data V3 is reconstructed and the three-dimensional image data between the volume data V2 and V3 is interpolated and obtained, the time control unit 34 uses the volume data V2 and V3, and the volume data V2 and V3. Interpolated between the volume data.

  The image display unit 22 displays the three-dimensional image data corresponding to the interpolated volume data after displaying the three-dimensional image data corresponding to the volume data V1 and before displaying the three-dimensional image data corresponding to the volume data V2. Display data. The image display unit 22 displays the three-dimensional image data corresponding to the volume data V2 after a predetermined time after the interpolated three-dimensional image data is displayed. The image display unit 22 maintains a constant display update interval, and a three-dimensional image corresponding to the volume data V1, a three-dimensional image generated by interpolation of V1 and V2, a three-dimensional image corresponding to the volume data V2, and V2. The three-dimensional image is displayed in the order of the three-dimensional image generated by the interpolation of V3, the three-dimensional image corresponding to the volume data V3, and so on.

  That is, after the three-dimensional image V1 is displayed and before the next image is displayed, the three-dimensional image V2 is reconstructed, interpolated using the three-dimensional images V1 and V2, and then interpolated three-dimensional. An image is displayed, and then a three-dimensional image V2 is displayed.

  Referring to the image data shown in FIG. 10, the image data g22 is reconstructed, and while the image data g22 is displayed, scanning and reconstruction are performed to reconstruct the image data g24. Interpolated image data g23 is generated using the image data g24, the image data g23 is displayed following the display of the image data g22, and then the image data g24 is displayed. In this way, by shifting the display update timing, the interpolated image can be displayed and can be compared with the image data g13. The image display unit 22 includes a memory, a counter, and the like necessary for performing display after a predetermined time.

  The time control unit 34 synchronizes the time-series 3D image data A collected in advance and the 3D image data B obtained by scanning, and outputs them to the region control unit 35. Next, the control by the area control unit 35 will be described with reference to FIGS. FIG. 12 is a flowchart showing, in order, processing for extracting a desired region from a time-series three-dimensional image by the image processing apparatus according to the first embodiment of the present invention. FIGS. 13 to 15 are diagrams for explaining processing for extracting a desired region from a time-series three-dimensional image by the image processing apparatus according to the first embodiment of the present invention.

  First, the region control unit 35 reads from the region information storage unit 44 information (region extraction information) indicating a region extraction range applied to the time-series three-dimensional image data A collected in advance (step S20). Further, the three-dimensional image data B obtained by scanning is read (step S21). Then, the area control unit 35 calculates area extraction information for the 3D image data B from the read area extraction information (step S22), and extracts the range of the calculated area extraction information from the 3D image data B (step S22). S23). Note that the read area extraction information is slightly different from time to time. In addition, in the 3D image data A and the 3D image data B, there may be a deviation in the observation region to be extracted. Therefore, the region control unit 35 finely adjusts the range of the observation region to correct the deviation, Region extraction information suitable for the three-dimensional image data B is calculated (step S22). Then, based on the finely adjusted region extraction information, the region control unit 35 extracts the three-dimensional image data included in the fine-adjusted range from the three-dimensional image data B (Step S23).

  For example, when the time-series 3D image data A collected in advance represents an image of the upper body (chest) of the subject, in the past, a heart portion is extracted from the upper body image and displayed. Shall be. Information indicating the extracted range (region extraction information) is stored in the extraction information storage unit 44 for each time, and the region control unit 35 reads the region extraction information at each time. Then, the region control unit 35 applies the read region extraction information to the time-series three-dimensional image data B obtained by scanning, and extracts the three-dimensional image data in a range reflecting the region extraction information. To do.

  For example, as illustrated in FIG. 13A, the region control unit 35 reads region extraction information at each time (time phase) stored in the region information storage unit 44 in advance. Then, the region control unit 35 applies the region extraction information to the three-dimensional image data A collected at time (time phase) t1 shown in FIG. Thus, the three-dimensional image data in the range of the region extraction information is extracted. Similarly, the region control unit 35 applies the region extraction information to the three-dimensional image data B collected at time (time phase) t1 shown in FIG. As shown in c), three-dimensional image data in the range of the region extraction information is extracted. Similarly, at the times (time phases) t2 and t3, the three-dimensional image data is extracted according to the region extraction information applied at each time (time phase).

  In the case where region extraction in a different range for each time is performed on time-series three-dimensional image data, region extraction information is also stored in time series. For example, the region extraction information at time (time phase) t1 may be different from the region extraction information at time (time phase) t2. Therefore, the area control unit 35 extracts the image of the desired observation area by applying the area extraction information that varies with time to the three-dimensional image data A and B. This makes it possible to compare three-dimensional images included in the observation area at each time (time phase).

  Further, the region control unit 35 generates a mask (extraction region information) indicating the extraction region from the time-series three-dimensional image data extracted from the time-series three-dimensional image data A collected in advance. The mask may be applied to time-series three-dimensional image data B obtained by scanning to extract three-dimensional image data included in a desired observation area. As shown in FIG. 14, the area control unit 35 uses each of the three-dimensional image data g31, g32, g33, g34... Extracted at each time of the time-series three-dimensional image data A collected in advance. Masks (extraction region information) m1, m2, m3, m4,... At time (time phase) are generated, and the mask (extraction region information) m1 is generated in the three-dimensional image data B corresponding to each time (time phase). , M2, m3, m4,... Are extracted to extract the three-dimensional image data g41, g42, g43, g44,. This makes it possible to compare three-dimensional images included in the observation area at each time (time phase).

  When an observation region having the same range as the region extraction range applied to the time-series 3D image data A collected in advance is extracted from the time-series 3D image data B obtained by scanning However, there are cases where it is not appropriate to extract observation regions in exactly the same range. For example, when an image of the heart is extracted, there may be a deviation in the extracted observation area as shown in FIG. 14 due to the contraction rate of the heart. That is, since the size of the three-dimensional image data B obtained by scanning is smaller than the range of the three-dimensional image data A collected in advance at the times t1 and t2, applying the past observation region as it is. Deviation occurs. Specifically, the image data g41 is smaller than the image data g31 at time (time phase) t1, and the image data g42 is smaller than the image data g32 at time (time phase) t2. Therefore, if the mask (region extraction information) applied to the three-dimensional image data A is applied to the three-dimensional image data B as it is, the extraction region shifts.

  In this case, the region control unit 35 generates a mask (region extraction information) suitable for the three-dimensional image data B (step S22), and the generated mask (region extraction information) is obtained by scanning. A three-dimensional image of a desired observation area is extracted by applying to the image data B (step S23). For example, the region control unit 35 finely adjusts the range (mask shape) of the extraction region following the change (for example, contraction rate) of the heart that is the observation target, and a mask that matches the 3D image data B ( Region extraction information). For example, by performing morphological filter processing, the region control unit 35 finely adjusts (shrinks or expands) the range (mask shape) of the extraction region as shown in FIG. 15 based on the contraction rate of the heart and the like. Thus, a mask (region extraction information) suitable for the three-dimensional image data B is generated. Then, the region control unit 35 extracts the three-dimensional image data included in the desired observation region by applying the finely adjusted range of the extraction region to the three-dimensional image data B obtained by scanning. In this way, by finely adjusting the region extraction information to correct the deviation, and applying the corrected region extraction information to the three-dimensional image data B, the image included in the observation region suitable for the three-dimensional image data B Can be extracted.

  As described above, the region control unit 35 obtains the time series 3 obtained by scanning the region corresponding to the region extraction range applied to the time series three-dimensional image data A collected in advance. By extracting the three-dimensional image data by applying to the three-dimensional image data B, it is possible to compare the three-dimensional images included in the observation region at each time (time phase).

  The 3D image data extracted by the region control unit 35 is output to the operation control unit 36. The control by the operation control unit 36 will be described with reference to FIGS. FIG. 16 is a flowchart showing, in order, processing for matching operation states by the image processing apparatus according to the first embodiment of the present invention. FIG. 17 is a diagram for explaining processing for matching the operation states by the image processing apparatus according to the first embodiment of the present invention.

  The operation control unit 36 reads information (operation information) stored in the operation information storage unit 45 indicating the content of the operation performed on the time-series 3D image data A collected in advance (step). S30). Further, the operation control unit 36 reads time-series three-dimensional image data B obtained by scanning (step S31). The operation information and image processing parameters are time-sequential and differ from time to time.

  Then, the operation control unit 36 changes the display on the image display unit 22 by reflecting the read operation information on the three-dimensional image data A collected in advance (step S32). Further, the operation control unit 36 reflects the read operation information in the three-dimensional image data B obtained by scanning, and changes the display on the image display unit 22 (step S33). The operation information includes, for example, information for rotating the image on the screen of the image display unit 22, and the operation control unit 35 receives, for example, the information for rotating the three-dimensional image collected in advance. The rotation operation is reflected in the data A and the three-dimensional image data B obtained by scanning. The 3D image data A and 3D image data B output from the operation control unit 35 are output to the image display unit 22 via the system control unit 31, and the 3D image data A and 3D image data B are , It is displayed in the same way on the screen. Thus, by comparing the direction and position by applying the same operation to each three-dimensional image data, comparison and observation of images are facilitated.

  Further, as shown in FIG. 17, operation information such as rotation, movement, or zoom input from the operation input unit 21 may be reflected in the three-dimensional image data A and B. For example, when the three-dimensional image data A is rotated, the operation control unit 35 performs the rotation operation on the three-dimensional image data B, and similarly rotates the three-dimensional image data B. In this way, by applying the same operation for each three-dimensional image data at each time, comparison and observation of images are facilitated.

  When matching the contents of the image processing of the three-dimensional image data A and B, the operation control unit 36 stores the time-series three-dimensional image data A collected in advance stored in the operation information storage unit 45. The image processing parameters (opacity, WL / WW, etc.) performed on the image data are read, and image processing is performed on the three-dimensional image data A and B using the parameters. This unifies the opacity, contrast, brightness, and the like of the three-dimensional image data A and B, so that comparison and observation of images are facilitated. In addition, the operation control unit 36 may perform image processing performed on the three-dimensional image data A obtained by scanning, on the three-dimensional image data A collected in advance. That is, image processing may be performed on the three-dimensional image data A and B using the same parameters.

  As described above, the time series 3D image data A that has been processed by each control unit and collected in advance, and the time series 3D image data B obtained by scanning are image display units. 22 is output. The contents displayed on the image display unit 22 are shown in FIG. In FIG. 18, on the screen of the image display unit 22, the three-dimensional image data A collected in advance and the three-dimensional image data B obtained by scanning are simultaneously displayed for each time. This display content is the same in the local coordinate system by the processing by the position control unit 33 described above. Further, by the processing by the time control unit 34, for example, 3D image data synchronized with an electrocardiogram signal (ECG signal) is displayed, and 3D image data collected at the same time (time phase) is displayed at the same time. In addition, an image in which regions corresponding to each other are extracted by the processing by the region control unit 35 is displayed. In this example, a three-dimensional image of the heart is displayed as the region of interest. Further, the processing by the operation control unit 36 makes the rotation direction, zoom magnification, and the like the same.

  On the screen of the image display unit 22, a slightly different three-dimensional image is displayed for each time, and the operator can observe as a moving image and compare both images. As described above, according to the image processing apparatus 20 according to this embodiment, the three-dimensional image data included in the desired observation region is extracted from the time-series three-dimensional image data in a so-called real time during the scan, and the biological information is obtained. Based on the time-series 3D image data, and further, display operations such as the orientation of the 3D image included in the extracted observation area, the viewing angle and magnification of the displayed 3D image, By sharing the content of image processing between time-series 3D image data to be compared, it is easy to compare time-series 3D image data and confirms changes over time in the affected area, treatment effects, etc. Can also be facilitated. This makes it possible to provide new information effective for diagnosis to doctors and the like.

  In addition, when an ultrasonic diagnostic apparatus is used as a medical image diagnostic apparatus and an image is collected by the stress echo method, it is possible to provide effective information to a doctor by using the image processing apparatus 20 according to this embodiment. Become. For example, the stress echo method is used to jog the subject and collect time-series 3D image data of the myocardium before and after jogging to obtain time-series 3D images before and after jogging. In the case of comparison, by processing time-series three-dimensional image data before and after jogging by the image processing device 20, it becomes possible to easily compare time-series three-dimensional images of the heart.

  Conventionally, when images are collected by the stress echo method, it is impossible to extract in real time 3D image data included in a desired observation area from time-series 3D image data during scanning. While collecting serial 3D image data, it was not possible to compare time-series 3D images before and after jogging in real time. However, when the image processing apparatus 20 according to this embodiment is used, three-dimensional image data included in a desired observation area can be extracted in real time from time-series three-dimensional image data during scanning. It is possible to compare the time-series three-dimensional images in real time.

  Further, the system control unit 31 may individually release the processing of the image control unit 32, the position control unit 33, the time control unit 34, the region control unit 35, and the operation control unit 36 in accordance with an instruction from the operation input unit 21. it can. In this way, it is possible to display an image on the image display unit 22 by synchronizing only the designated process without performing all the processes. For example, only the time control unit 34 and the region control unit 35 are processed to temporally synchronize the time-series 3D image data A and B, and further, the 3D image data included in the desired observation region It can also be extracted and displayed on the image display unit 22. Even if processing is performed in this way, three-dimensional images included in the observation region can be compared in time series.

  In addition to the three-dimensional image, a waveform of biological information is displayed on the screen of the image display unit 22. A marker indicating the time (time phase) when the currently displayed image is collected may be displayed on the waveform of the biological information. Further, an input unit for turning on / off the processing of each control unit is displayed on the screen of the image display unit 22 as a GUI (graphical user interface), and is turned on by a mouse or the like installed in the operation input unit 21. It may be possible to select switching of / OFF. Also, the three-dimensional position information used in the position control unit 33, the biological information used in the time control unit 34, the region extraction information used in the region control unit 35, and the operation information used in the operation control unit 36 Specific contents may be displayed so as to be selectable. For example, when the position “feature point of bone region” is selected by a mouse or the like installed in the operation input unit 21, the feature point (anatomical feature point) of the bone region is 3 as in the above-described processing. When the time “ECG” is selected, the electrocardiogram signal (ECG signal) is used as biological information. Furthermore, an operation panel may be displayed on the screen, and operations such as enlargement, movement, rotation, or measurement of a three-dimensional image may be selected. Further, an opacity curve as an image processing parameter and a WL / WW curve as a parameter for determining contrast and brightness may be displayed on the screen.

  In addition, when the image processing apparatus according to this embodiment is applied, it is possible to compare time-series three-dimensional image data collected by different medical image diagnostic apparatuses, and superimpose a plurality of three-dimensional image data. By displaying on the image display unit 22, a time-series composite image (so-called Fusion image) can be displayed. For example, based on region extraction and biological information for time-series 3D image data collected by an ultrasound diagnostic apparatus and time-series 3D image data collected by an X-ray CT apparatus By performing the synchronization process or the like, it becomes easy to compare three-dimensional images included in a desired observation area at each time. In addition, it is possible to easily compare the time-series three-dimensional image data of the subject with a typical case that changes with time. In this manner, new information effective for diagnosis can be provided to a doctor or the like.

[Second Embodiment]
Next, an image processing system including an image processing apparatus according to a second embodiment of the present invention will be described with reference to FIG. FIG. 19 is a block diagram showing a schematic configuration of an image processing system including an image processing apparatus according to the second embodiment of the present invention. The image processing apparatus 20 according to the second embodiment has the same configuration as that of the image processing apparatus 20 according to the first embodiment, but in this embodiment, as shown in FIG. The medical image diagnostic apparatus 1 (10) is not connected, and three-dimensional image data is not input from the medical image diagnostic apparatus 1 in a so-called real time. In this case, a plurality of time-series three-dimensional image data is stored in the storage device 40, and the above-described processing is performed on the stored three-dimensional image data.

  As described above, for a plurality of time-series three-dimensional image data stored in the storage device 40, a three-dimensional image synchronized with biological information is displayed on the image display unit 22, and one time series is further displayed. When 3D image data included in a desired observation area is extracted from typical 3D image data, the range of the observation area is also applied to the other time-series 3D image data to obtain the desired observation. Three-dimensional image data included in the region is extracted and displayed. At this time, if there is a deviation in the range of the observation region, the observation region is contracted or expanded by fine adjustment, and the finely adjusted observation region is applied to the other three-dimensional image data and included in the desired observation region. Three-dimensional image data is extracted. In addition, a three-dimensional image is displayed on the image display unit 22 in a state where the same display operation is performed by synchronizing operation information and the like.

  This makes it possible to compare time-series three-dimensional image data collected by different medical image diagnostic apparatuses. By superimposing a plurality of three-dimensional image data and displaying them on the image display unit 22, A time-series composite image (a so-called Fusion image) can be displayed. For example, based on region extraction and biological information for time-series 3D image data collected by an ultrasound diagnostic apparatus and time-series 3D image data collected by an X-ray CT apparatus By performing the synchronization process or the like, it becomes easy to compare three-dimensional images included in a desired observation area at each time. In addition, it is possible to easily compare the time-series three-dimensional image data of the subject with a typical case that changes with time. In this manner, new information effective for diagnosis can be provided to a doctor or the like.

[Third Embodiment]
Next, an image processing system according to a third embodiment of the present invention will be described with reference to FIG. The image processing system according to this embodiment corresponds to an intraoperative navigation system that assumes robotic surgery or the like, and can be realized by including the image processing apparatus according to the first embodiment.

  As shown in FIG. 20, an intraoperative navigation device 50 is connected to the image processing device 20. Further, the image processing apparatus 20 is provided with an edited image output unit 37. The edited image output unit 37 outputs time-series three-dimensional image data subjected to region extraction and synchronization processing based on biological information in the image processing device 20 to the intraoperative navigation device 50.

  The intraoperative navigation device 50 includes an operation input unit 51, a real image collection unit 52, a real image output unit 53, an image display unit 54, a control unit 55, and a superimposed image processing unit 56.

  The operation input unit 51 operates a surgical instrument 57 such as a scalpel necessary for surgery by an operation by an operator. Further, the operation input unit 51 can also operate a robot arm linked to the surgical instrument 57 by an operation of the surgeon.

  The real image collection unit 52 captures a real image of the subject P that is the imaging target of the medical image diagnostic apparatus 1, and can also follow the operator's viewpoint, and the operator's behavior (viewpoint A real image can be taken following the behavior. The image output unit 53 performs image processing on the real image collected by the real image collection unit 52, converts the collected real image into a clear image, and outputs the image to the superimposed image processing unit 56.

  The superimposed image processing unit 56 and the real image output from the image output unit 53 and the time-series three-dimensional image output from the edited image output unit 37 of the image processing apparatus 20 and subjected to processing such as region extraction. Receive data and superimpose them.

  The control unit 55 controls the intraoperative navigation device 50 as a whole. Mainly, the operation information output by the operator from the operation input unit 51 and the real image and time series output from the superimposed image processing unit 56 are used. A process of transferring image data or the like superimposed with typical three-dimensional image data to the image display unit 54 is performed.

  The image display unit 54 collects the actual image following the movement of the operator's viewpoint and the time series 3 collected by the medical image diagnostic apparatus 1 and subjected to region extraction, synchronization processing, and the like by the image processing apparatus 20. Dimensional image data is displayed superimposed. An operator (operating robot operator) operates the surgical instrument 57 via the operation input unit 51 while observing the superimposed image displayed on the image display unit 54.

  As described above, the time-series three-dimensional image subjected to region extraction or the like by the image processing device 20 and the real image are displayed in an overlapping manner, so that an organ that is not obtained in the real image and exists inside the subject. It is possible for the surgeon to acquire changes over time such as movement and deformation of the body in real time during the operation. For example, by displaying the time-series 3D image data during surgery and the time-series 3D image data before surgery in synchronization with each other, or by extracting and displaying an image of a portion to be observed It becomes possible to conduct comparative studies during surgery.

1 is a block diagram illustrating a schematic configuration of a medical image system including an image processing apparatus according to a first embodiment of the present invention. 1 is a block diagram illustrating a schematic configuration of a medical image system including an image processing apparatus according to a first embodiment of the present invention. It is a flowchart which shows the coordinate transformation process by the image processing apparatus which concerns on 1st Embodiment of this invention in order. It is a figure for demonstrating the coordinate transformation process by the image processing apparatus which concerns on 1st Embodiment of this invention. 4 is a flowchart illustrating, in order, processing for displaying two time-series three-dimensional images in synchronization by the image processing apparatus according to the first embodiment of the present invention. It is a figure which shows an electrocardiogram signal (ECG signal). FIG. 6 is a diagram for explaining processing for displaying two time-series three-dimensional images in synchronization by the image processing apparatus according to the first embodiment of the present invention. It is a figure for demonstrating the image interpolation process by the image processing apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the image interpolation process by the image processing apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the image interpolation process by the image processing apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the image interpolation process by the image processing apparatus which concerns on 1st Embodiment of this invention. 4 is a flowchart showing, in order, processing for extracting a desired observation region from a time-series three-dimensional image by the image processing apparatus according to the first embodiment of the present invention. It is a figure for demonstrating the process which extracts a desired observation area | region from a time-sequential 3D image by the image processing apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the process which extracts a desired observation area | region from a time-sequential 3D image by the image processing apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the process which extracts a desired observation area | region from a time-sequential 3D image by the image processing apparatus which concerns on 1st Embodiment of this invention. 4 is a flowchart showing, in order, processes for matching operation states by the image processing apparatus according to the first embodiment of the present invention. It is a figure for demonstrating the process which makes an operation state correspond by the image processing apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the image displayed on the image display part installed in the image processing apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows schematic structure of the image processing system provided with the image processing apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows schematic structure of the image processing system provided with the image processing apparatus which concerns on 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 10 Medical image diagnostic apparatus 2 Irradiation part 3 Detector 4 Data collection part 5 Image reconstruction part 6 Position information collection part 7 Biometric information collection part 11 Ultrasonic probe 12 Transmission / reception part 13 Signal processing part 14 DSC (digital scan converter)
DESCRIPTION OF SYMBOLS 20 Image processing apparatus 21 Operation input part 22 Image display part 30 Control part 31 System control part 32 Image control part 33 Position control part 34 Time control part 35 Area control part 36 Operation control part 40 Storage device 41 Image storage part 42 Position information storage Unit 43 Biological information storage unit 44 Area information storage unit 45 Operation information storage unit

Claims (9)

First time-series three-dimensional image data collected every predetermined time together with biological information of the subject that changes with time, and second time-series collected in advance along with the biological information every predetermined time Time synchronization means for receiving the three-dimensional image data and synchronizing the first time-series three-dimensional image data and the second time-series three-dimensional image data based on the biological information. When,
Three-dimensional image data included in the second observation region that differs for each time phase is extracted from the second time-series three-dimensional image data collected in advance for each time phase, and the different for each time phase Based on the second observation area, a first observation area for the first time-series three-dimensional image data is obtained for each time phase, and the time phase is calculated from the first time-series three-dimensional image data. Region extracting means for sequentially extracting, for each time phase , three-dimensional image data included in the first observation region that differs for each time ;
Display means for synchronizing and displaying the three-dimensional image data extracted for each time phase from the first time-series three-dimensional image data and the second time-series three-dimensional image data ;
An image processing apparatus. Said region extraction means, to follow the temporal change of the observation target by changing the range of the first observation region, the first observation region after the change from the first time-series three-dimensional image data 3D image data contained in
The image processing apparatus according to claim 1 . The time synchronization means is based on the biological information, and the three-dimensional data collected in the same time phase among the first time-series three-dimensional image data and the second time-series three-dimensional image data. Synchronize image data,
The image processing apparatus according to claim 1 . The biological information includes electrocardiographic signals, myoelectric signals of the subject, or joint motion information that indicates joint motions in time series.
The image processing apparatus according to any one of claims 1 to 3. The three-dimensional image data lacking the predetermined time of each of the first time-series three-dimensional image data or the second time-series three-dimensional image data is collected before and after the predetermined time. It further has an image interpolation means for generating by interpolation using the dimensional image data.
The image processing apparatus according to any one of claims 1 to 4. The first time-series three-dimensional image data and the second time-series three-dimensional image data are collected together with the three-dimensional position information of the subject that changes with time,
Based on the 3D position information, at least one of the coordinate system of the first time-series 3D image data or the coordinate system of the second time-series 3D image data is changed. And further has position control means for matching with the other coordinate system,
The image processing apparatus according to any one of claims 1 to 5. Time-series three-dimensional position information that changes with time is extracted from the first time-series three-dimensional image data and the second time-series three-dimensional image data, and the three-dimensional position information is extracted. And changing the coordinate system of the first time-series 3D image data or the coordinate system of the second time-series 3D image data to change the other coordinate system. Further comprising position control means for matching the system,
The image processing apparatus according to any one of claims 1 to 5. An operation or image processing performed on any one of the first time-series three-dimensional image data and the second time-series three-dimensional image data. Further has an operation control means for performing the other 3D image data.
The image processing apparatus according to any one of claims 1 to 7. Operation information indicating the content of the operation performed on the second time-series 3D image data, or the content of the image processing performed on the second time-series 3D image data Storage means for storing image processing information indicating
Operation indicated by the operation information stored in said storage means, or the image processing by the image processing information indicates, further having an operation control means for performing relative to the first time-series three-dimensional image data,
The image processing apparatus according to any one of claims 1 to 7.

Images