JP2008078959A - Image processor, and imaging type image display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively suppress occurrence of moire, and clearly display even a CG image, etc. when a composite image of a photographed image and an image other than the photographed image (CG image, etc.), is displayed. <P>SOLUTION: An image processor 100 is provided with a processing means 122L which performs low pass filter processing for a composite image which is composed of a first image created by photographing and imaging and a second image created by a method except photographing and imaging. The processing means performs the low pass filter processing for a first image region and a second image region among the composite image by cut-off frequencies which are different each other. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that synthesizes an image obtained by imaging and an image generated by computer graphics (CG) or the like, and a head mount type or hunt-held type image display apparatus including the same.

  There is an image display device such as a so-called video see-through type head-mounted display or hand-held display that displays a composite image of an image captured at a position corresponding to an observer's eye and a CG image on a display element to allow the observer to observe the image. Various proposals have been made (see Patent Document 1).

In such a video see-through type image display device, a captured image generated using an image sensor such as a CCD sensor or a CMOS sensor is output to a processing device such as a personal computer. In the processing device, an image produced by CG or the like (hereinafter referred to as a CG image) is synthesized with the captured image and output again to the image display device. In the image display device, the input composite image is subjected to resolution conversion, frame frequency conversion processing, and the like, and then low-pass filter processing is performed in order to reduce the moiré of the observed image.
JP 2006-171737 A

  However, in the conventional video see-through type image display device, the cut-off frequency of the low-pass filter processing for the composite image is made uniform over the entire screen.

  In this case, if the cut-off frequency is set high, moire often occurs in a captured image region that generally occupies a large portion of the screen. In addition, when the cut-off frequency is set to a low value, there is a problem that an edge portion of a CG image or a character drawn with high definition is blurred and the image becomes difficult to see.

  The present invention effectively suppresses the occurrence of moiré when displaying a composite image of a captured image and an image other than the captured image (such as a CG image), and enables the display of a CG image or the like clearly. An object of the present invention is to provide a processing device and an imaging type image display device.

  An image processing apparatus according to an aspect of the present invention performs low-pass filter processing on a composite image obtained by combining a first image generated by imaging and a second image generated by a method other than imaging. Have means. The processing means performs low-pass filter processing with different cutoff frequencies on the first image region and the second image region of the composite image.

  An image processing apparatus according to another aspect of the present invention includes a combining unit that combines a first image generated by imaging and a second image generated by a method other than imaging, and a low pass for the image. Processing means for performing filter processing. The processing means performs low-pass filter processing with different cutoff frequencies on the first image and the second image, and the synthesizing means is a first image that has been subjected to low-pass filter processing by the processing means. And the second image.

  An image processing apparatus according to another aspect of the present invention is a low-pass filter process for a synthesized image obtained by synthesizing a first image generated by imaging and a second image generated by a method other than imaging. The processing means for performing The processing means is characterized in that the low-pass filter processing is not performed on the region of the second image in the composite image, but the low-pass filter processing is performed on the region of the first image.

  An image processing apparatus according to another aspect of the present invention includes a combining unit that combines a first image generated by imaging and a second image generated by a method other than imaging, and a low pass for the image. Processing means for performing filter processing. The processing means performs low-pass filter processing on the first image and does not perform low-pass filter processing on the second image, and the synthesizing means performs low-pass filter processing by the processing means. The first image and the second image not subjected to the low-pass filter processing are synthesized.

  Further, an image pickup type image display device having an image pickup system for picking up a subject to generate a first image, the image processing device, and a display element for displaying a composite image subjected to low-pass filter processing by the processing means. Constitutes another aspect of the present invention.

  According to the present invention, it is possible to set an optimum low-pass filter processing cutoff frequency for each of a captured image region and a region of an image generated by CG or the like in a composite image or only for a captured image region. . Therefore, it is possible to suppress the occurrence of moire in the captured image area and to clearly display an area such as a CG image.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 5 shows a configuration of a video see-through type head mounted display main body (hereinafter referred to as an HMD main body) 1000 that is Embodiment 1 of the present invention.

  In FIG. 5, reference numeral 110 denotes an image pickup element constituted by a CCD sensor, a CMOS sensor, or the like. Reference numeral 611 denotes an imaging prism, which folds the light beam 610 from the outside (subject) using multiple reflections so that the position of the imaging element 110 is as close as possible to the position of the eyeball 622 of the observer, and is compact. The necessary optical path length is secured. Reference numeral 612 denotes an imaging lens for imaging the light flux from the imaging prism 611 on the imaging device 110. An imaging optical system is configured by the imaging prism and the imaging lens 612, and an imaging unit (imaging system) is configured by the imaging optical system and the imaging element 110.

  A display element 125 includes an LCD, a CRT, an EL element, or the like. Reference numeral 620 denotes an observation prism (display optical system) for guiding the light beam 621 emitted from the display element 125 to the eyeball 622 of the observer while expanding. The observer can observe the virtual image of the image displayed on the display element 125 by looking into the observation prism 620. The display element 125 and the observation prism 620 constitute a display unit.

  The HMD main body 1000 of this embodiment has a pair of left and right imaging display units that incorporate the imaging unit and the display unit. Thereby, the observer can observe the picked-up image (and the synthesized image mentioned later) acquired by the left and right imaging units with the left and right eyes via the left and right display units.

  FIG. 6 shows the appearance of the HMD main body 100 of this embodiment and the control unit (image processing apparatus) connected thereto. In FIG. 6, reference numeral 701 denotes an observer's head. 101L and 101R are left and right imaging display units. These imaging display units 101L and 101R are fixed to the head-mounted device 103. By mounting the head-mounted device 103 on the observer's head 701, the imaging display units 101L and 101R are placed in front of the observer's eyes. Can be arranged. As a result, it is possible to observe an image that is substantially the same as the observer actually sees with the eyes. However, in the present embodiment, a synthesized video in which a video generated by a method other than imaging such as a CG video or a character video is synthesized with the captured video is observed.

  Reference numeral 100 denotes a control unit for controlling the HMD main body 1000, and reference numeral 102 denotes a cable for connecting the HMD main body 1000 and the control unit 100. The HMD main body 1000 and the control unit 100 are inseparable from each other in use, and the image display device referred to in the present invention means that the HMD main body 1000 and the control unit 100 are included.

  Reference numeral 200 denotes a computer such as a personal computer or a workstation connected to the control unit 100 via a cable or wireless. The computer 200 can create a CG video, and also creates a composite video obtained by synthesizing a captured video and a CG video input from the control unit 100 as will be described later. The created composite video is output to the control unit 100.

  In FIG. 1, the structure of the electric circuit of the HMD main body 1000 and the control unit 100 of a present Example is shown. In FIG. 1, the letters R and L attached after each symbol (number) mean for the right eye and for the left eye. However, since the circuit configurations for the right eye and the left eye are common, R and L are omitted in the following description. In FIG. 1, the same components as those shown in FIGS. 5 and 6 are denoted by the same reference numerals.

  In FIG. 1, reference numeral 111 denotes an analog image processing unit that performs AGC (automatic gain control), CDS (correlated double sampling), ADC (analog-digital conversion), and the like on an analog signal output from the image sensor 110.

  A digital image processing unit 112 performs image processing such as γ (gamma) correction, hue correction, contour correction, and distortion correction for the imaging optical system on the digital signal output from the analog image processing unit 111.

  Reference numeral 113 denotes a video output unit such as an IEEE1394 interface that converts the signal processed by the digital image processing unit 112 into a predetermined video format and outputs the converted video signal. The captured video output from the video output unit 113 is input to the computer 200 described above.

  Reference numeral 114 denotes a captured video frame memory that stores the captured video image-processed by the digital image processing unit 112 for each of a plurality of predetermined frame images.

  Reference numeral 120 denotes a video input unit such as a DVI (digital video interface) receiver that captures a composite video of a captured video and a CG video that is a video other than the captured video, created by the computer 200.

  Reference numeral 121 denotes a video separation unit that compares the composite video input from the video input unit 120 and the frame image stored in the captured video frame memory 114 to separate the captured image and the CG image.

  A video conversion unit 122 converts the resolution and frame frequency of the captured image and CG image separated by the video separation unit 121 according to the display element 125, and further performs distortion correction processing on the display optical system. is there. The video conversion unit 122 further performs low-pass filter processing for the purpose of removing moire and the like. This low-pass filter process will be described in detail later.

  Reference numeral 123 denotes a video re-synthesizing unit that re-synthesizes the captured image and the CG image that have undergone each process in the video converting unit 122.

  A display element driving unit 124 receives the input of the recombined video from the video recombining unit 123 and drives the display element 125. Thereby, the video (synthesized image) re-synthesized by the video re-synthesis unit 123 is displayed on the display element 125, and the observer can observe the synthesized video.

  Reference numeral 130 denotes a control unit such as a CPU that controls operation of the control unit 100 and the HMD main body 1000.

  Reference numeral 131 denotes a distortion correction table unit that stores distortion correction data for the imaging optical system, and reference numeral 132 denotes a distortion correction table unit that stores distortion correction data for the display optical system. Note that the distortion of the image generated in the optical system differs due to the influence of individual differences of optical parts, assembly tolerances, and the like. Therefore, the distortion correction table unit 131 for the imaging optical system and the distortion correction table unit 132 for the display optical system measure the distortion data for each imaging unit and display unit in each HMD main body, and write the respective correction data.

  FIG. 2A shows a captured video (captured frame image). FIG. 2B shows a synthesized video (synthesized frame image) of a captured video and a CG video (CG frame image) input from the computer 200. Further, FIGS. 2C and 2D respectively show a captured frame image region and a CG frame image region separated from the synthesized video.

  As described above, the captured video (FIG. 2A) captured by the imaging unit of the HMD main body 1000 and subjected to various types of image processing in the control unit 100 is sent to the computer 200. The computer 200 generates a composite video (FIG. 2B) obtained by superimposing (combining) the CG video shown in FIG. 2D on the input captured video. The generated composite video is output from the standard video output terminal of the computer 200 in a predetermined format such as DVI. The composite video from the computer 200 is received by the DVI receiver or the like in the video input unit 120 of the control unit 100 and converted into a digital video signal that can be used in the processing circuit at the subsequent stage.

  The video separation unit 121 compares the synthesized video (synthesized frame image) from the video input unit 120 with the captured video (captured frame image) stored in the captured video frame memory 114 by the time of the synthesis process. Thereby, the imaging frame image region (FIG. 2C) and the CG frame image region (FIG. 2D) in the composite frame image are discriminated.

  Here, the synthesis processing time is the time from when the captured frame image is generated in the control unit 100 to when it is sent to the computer 200, synthesized with the CG image, and re-input to the control unit 100 as a synthesized image. . Therefore, the captured frame image to be compared with the synthesized frame image (image stored in the frame memory 114) matches the captured frame image used for generating the synthesized frame image. Thereby, the captured frame image region and the CG frame image region in the composite frame image can be accurately specified.

  The video separation unit 121 separates the captured frame image area and the CG frame image area from the synthesized frame image. Then, a separated captured frame image that is an image of only the captured frame image region and a separated CG frame image that is an image of only the CG frame image region are input to the video conversion unit 122.

  The video conversion unit 122 performs resolution conversion and frame frequency conversion according to the display element 125 for each of the separated captured frame image and the separated CG frame image. Further, based on the data stored in the distortion correction table unit 132 corresponding to the display optical system, a process for giving the display image a distortion in a direction opposite to the image distortion generated in the display optical system is performed.

  Further, the video conversion unit 122 performs low-pass filter processing on each of the separated captured frame image and the separated CG frame image on which the image processing has been performed. At this time, for the separated captured frame image, for example, low-pass filter processing is normally performed at a first cutoff frequency set for removing moire in the captured image. On the other hand, for the separated CG frame image, low-pass filter processing is performed at a second cutoff frequency set higher than the first cutoff frequency. That is, the low-pass filter process is performed on each of the separated captured frame image and the separated CG frame image at different cutoff frequencies.

  Thereby, when the synthesized video re-synthesized by the video re-synthesizing unit 123 is displayed on the display element 125, it is possible to suppress the occurrence of moiré that often appears in the area of the captured video, and the CG video. The area can be clearly displayed.

  The separated captured frame image and the separated CG frame image that have been subjected to various types of image processing including low-pass filter processing are sent to the video re-synthesizing unit 123, where they are combined to generate one combined frame image again. The composite frame image is sequentially processed into a signal suitable for driving the display element 125 in the display element driving unit 124 and output to the display element 125. As a result, a composite image is displayed and can be observed by an observer.

  3A and 3B and FIGS. 4A and 4B show an image separation method in the video separation unit 121 when the above-described synthesis processing time is unknown or fluctuates.

  3A and 3B show an example in which frame number data is written in an image data string of a captured frame image. FIG. 3A shows an image data string of a captured frame image subjected to image processing in the digital image processing unit 112. Here, only the top part of the image data string is shown for easy understanding.

  FIG. 3B shows an example in which the head of the image data sequence shown in FIG. 3A is replaced with frame number data (F.No).

  4A and 4B show an example in which frame number data is written in the image data string of the composite frame image. FIG. 4A shows an image data string of a composite frame image input from the computer 200. Here, only the top part of the image data string is shown for easy understanding.

  FIG. 4B shows an image data sequence in a state where frame number data is removed from the video data sequence shown in FIG. 4A.

  An operational difference from the first embodiment when using the frame number data shown in FIGS. 3A and 3B and FIGS. 4A and 4B will be described.

First, regarding the generation of the captured image, the operations up to the digital image processing unit 112 are the same as those in the first embodiment. The video output unit 113 performs the following processing on the image data sequence of the captured frame image shown in FIG. 3A that has been subjected to image processing by the digital image processing unit 112. First, as shown in FIG. 3B, the first 1-byte data at the head of the image data sequence, that is, “U _0.5 ” data in FIG. 3A is rewritten to frame number data (F.No.). The captured image frame memory 114 stores the captured frame image (image data string) shown in FIG. 3A and manages it in association with the frame number data.

  Next, the captured frame image (image data string) with frame number data shown in FIG. 3B output from the video output unit 113 is sent to the computer 200. The computer 200 converts the image data string into RGB data with the frame number data removed, and synthesizes it with the CG frame image.

After the composite frame image is generated, the computer 200, as shown in FIG. 4A, uses the frame number data obtained by removing the first byte of R (red) data in the composite frame image, that is, “R ₀ ” first. Rewrite to Then, the composite frame image with the frame number data is sent to the video input unit 120 of the control unit 100.

  The composite frame image with frame number data shown in FIG. 4A input to the video input unit 120 from the computer 200 is sent to the video separation unit 121. The video separation unit 121 reads the frame number data written in the first byte of the R data, and corresponds to the read frame number data among the plurality of captured frame images stored in the captured video frame memory 114. A captured frame image is read out. Then, the captured frame image and the synthesized frame image are compared, and the captured frame image region and the CG frame image region are separated.

At this time, a process of returning the first byte of the R data to “R ₀ ” data is performed, and converted to the composite frame image (image data string) shown in FIG. 4B.

This conversion method is as follows. First, data of the first two pixels in the image data sequence in the captured frame image corresponding to the frame number data stored in the captured video frame memory 114 is calculated. That is, “U _0.5 ”, “Y ₀ ”, “V _0.5 ”, “Y ₁ ” to “R ₀ ”, “R ₁ ”, “G ₀ ”, “G ₁ ” in FIG. “B ₀ ” and “B ₁ ” are calculated.

Next, among the calculated data of the first two pixels, the data values of the other five colors excluding “R ₀ ” and the image data string of the composite frame image input from the computer 200 are “R ₀ ”. The data values of the other five colors except for are compared. If all the data values of these five colors are equal (or the difference is within a predetermined range), it is determined that the first pixel is data of the captured frame image, and the calculated “R ₀ ” value is used as it is. Replace “F.No” written at the original “R ₀ ” position.

On the other hand, when at least one of the data values of the five colors is not equal, it is determined that the first pixel is CG frame image data, and the “R ₁ ” value of the image data string input from the computer 200 is originally set. This is replaced with “F.No” written at the “R ₀ ” position. Note that the video separation unit 121 and later perform the same operation as in the first embodiment.

  That is, this embodiment includes a memory that stores at least one captured frame image (first image) and means for writing frame number data to the captured frame image. Furthermore, it has means for writing the frame number data into the composite frame image when generating the composite frame image. Then, based on the frame number data written in the composite frame image, an imaging frame image to be compared with the composite frame image is selected from the memory.

  In the present embodiment, whether the image data is image data of a captured frame image or image data of a composite frame image with respect to the image data of the composite frame image erased by writing the frame number data to the composite frame image. It has a means to judge. If it is determined that the image data is a captured frame image, the erased image data is generated based on the selected captured frame image. If it is determined that the image data is a composite frame image, the image data of the pixel adjacent to the pixel from which the image data has been deleted is used instead of the deleted image data.

  In this embodiment, the frame number data is written in the first byte of the image data string or the first byte of the R data. However, the present invention is not limited to this. For example, the frame number data may be written in the last byte, the middle byte, or G (green) data or B (blue) data.

  In the present embodiment, the captured frame image is described as YUV422 data, and the synthesized frame image is described as RGB parallel data. However, the present invention is not limited to this, and other image data formats may be used.

In the present embodiment, the case of comparing all the data of the five colors when reproducing the data of “R ₀ ” has been described. However, the present invention is not limited to this, and for example, “G ₀ ”, “B” _The comparison may be made using two colors of “ ₀ ” or other colors.

  In the first and second embodiments, the case where the video output signal to the computer 200 is performed by IEEE 1394 and the video input signal from the computer 200 is performed by DVI has been described. However, the present invention is not limited to this. For example, a digital interface USB or other signals may be used.

  In the first and second embodiments, the binocular HMD provided with the imaging display unit for the right eye and the imaging display unit for the left eye has been described. However, the present invention is a single-eye HMD or handheld type. The present invention can also be applied to other image display devices.

  FIG. 7 shows a part of the control unit 100 ′ according to the third embodiment of the present invention. In the first and second embodiments, the case where the captured image is once sent from the control unit 100 to the computer 200, the captured image and the CG image are combined by the computer 200, and the combined image is input to the control unit 100 has been described. .

  On the other hand, the present embodiment is an example where only a CG image is input from the computer 200 to the control unit 100 '. In this case, the control unit 100 ′ does not need to output the captured image to the computer 200, and stores the captured frame image in the captured image frame memory 114 as it is. In the present embodiment, the video separation unit 121 described in the first and second embodiments is not necessary.

  The video conversion unit 122 converts the resolution of the CG video (CG frame image) input from the computer 200 to the video input unit 120 and the captured frame image stored in the captured video frame memory 114 according to the display element 125. And frame frequency conversion is performed. Further, as in the first embodiment, based on the data stored in the distortion correction table unit 132 corresponding to the display optical system, a process for giving the display image distortion in a direction opposite to the image distortion generated in the display optical system. Is done.

  Further, the video conversion unit 122 performs low-pass filter processing on each of the captured frame image and CG frame image on which the image processing has been performed. The cut-off frequency in the low-pass filter processing for each frame image is the same as in the first embodiment.

  The captured frame image and the CG frame image that have been subjected to various types of image processing including low-pass filter processing are sent to the video synthesis unit 143, where they are synthesized to generate a synthesized frame image. The composite frame image is sequentially processed into a signal suitable for driving the display element 125 in the display element driving unit 124 shown in FIG. As a result, a composite image is displayed and can be observed by an observer.

  Also in the present embodiment, when the synthesized video synthesized by the video synthesizing unit 143 is displayed on the display element 125, it is possible to suppress the occurrence of moiré that often appears in the area of the captured video, and the CG video. This area can be clearly displayed.

  In the first to third embodiments, the captured frame image is subjected to the low-pass filter processing at the first cutoff frequency, and the CG frame image is also processed at the second cutoff frequency higher than the first cutoff frequency. The case where the low-pass filter process is performed has been described. However, the low-pass filter process may not be performed on the CG frame image.

  Since the configuration of the control unit 100 in this case is the same as that of the first embodiment (FIG. 1), this embodiment will be described with reference to FIG.

  In the present embodiment, the separated captured frame image and the separated CG frame image separated by the video separation unit 121 are input to the video conversion unit 122.

  The video conversion unit 122 performs resolution conversion and frame frequency conversion according to the display element 125 for each of the separated captured frame image and the separated CG frame image. Further, based on the data stored in the distortion correction table unit 132 corresponding to the display optical system, a process for giving the display image a distortion in a direction opposite to the image distortion generated in the display optical system is performed.

  Further, in the video conversion unit 122, the separated imaging frame image out of the separated imaging frame image and the separated CG frame image subjected to these image processes is low-passed at the first cutoff frequency described in the first embodiment. Perform filtering. Then, the separated captured frame image after the low-pass filter processing is output to the video re-synthesis unit 123. On the other hand, the separated CG frame image is output to the video re-synthesis unit 123 without being subjected to low-pass filter processing.

  The video re-synthesizing unit 123 re-synthesizes the separated captured frame image in which the generation of moire is suppressed by performing low-pass filter processing, and the separated CG frame image that remains clear without performing low-pass filter processing. An image is displayed on the display element 125.

  Also in this embodiment, it is possible to suppress the occurrence of moire that often appears in the captured video region, and to display the CG video region clearly.

  As described in the third embodiment, even when only the CG video is input from the computer to the control unit, as described in the fourth embodiment, the low-pass filter process is performed on the captured video and the CG video is processed. May not perform low-pass filtering.

  Since the configuration of the control unit 100 ′ in this case is the same as that of the third embodiment (FIG. 7), this embodiment will be described with reference to FIG.

  The video conversion unit 122 converts the resolution of the CG video (CG frame image) input from the computer 200 to the video input unit 120 and the captured frame image stored in the captured video frame memory 114 according to the display element 125. And frame frequency conversion is performed. Further, as in the first embodiment, based on the data stored in the distortion correction table unit 132 corresponding to the display optical system, a process for giving the display image distortion in a direction opposite to the image distortion generated in the display optical system. Is done.

  Further, the video conversion unit 122 performs low-pass filter processing only on the captured frame image. That is, the low-pass filter process is not performed on the CG frame image.

  The captured frame image subjected to various image processing including low-pass filter processing and the CG frame image subjected to various image processing excluding low-pass filter processing are sent to the video synthesis unit 143 and synthesized there. A composite frame image is generated.

  The composite frame image is sequentially processed into a signal suitable for driving the display element 125 in the display element driving unit 124 shown in FIG. As a result, a composite image is displayed and can be observed by an observer.

  Also in the present embodiment, when the synthesized video synthesized by the video synthesizing unit 143 is displayed on the display element 125, it is possible to suppress the occurrence of moiré that often appears in the area of the captured video, and the CG video. This area can be clearly displayed.

The block diagram which shows the electric constitution of HMD which is Example 1 (and Example 4) of this invention, and a control unit. FIG. 3 is a diagram illustrating an example of a captured image in the first embodiment. FIG. 3 is a diagram illustrating an example of a composite image in the first embodiment. FIG. 3 is a diagram illustrating a captured video region in a composite image according to the first embodiment. FIG. 3 is a diagram illustrating a CG video area in the composite image according to the first embodiment. The figure which shows the image data sequence of the imaging frame image in Example 2 of this invention. The figure which shows a mode that the frame number data were written in the image data sequence of FIG. 3A. FIG. 10 is a diagram illustrating a state in which frame number data is written in an image data string of a composite frame image in the second embodiment. The figure which shows a mode that the frame number data of the image data sequence of FIG. 4A were rewritten by image data. FIG. 3 is a cross-sectional view illustrating a configuration of the HMD main body according to the first embodiment. FIG. 3 is a diagram illustrating an appearance of an HMD main body and a control unit according to the first embodiment. The block diagram which shows a part of electrical structure of the control unit which is Example 3 (and Example 5) of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Control unit 1000 HMD main body 101R Right-eye imaging display unit 101L Left-eye imaging display unit 110R, 110L Image sensor 111R, 111L Analog image processing part 112R, 112L Digital image processing part 113R, 113L Video output part 114R, 114L Frame memory 120R, 120L video input unit 121R, 121L video separation unit 122R, 122L video conversion unit 123R, 123L video recombination unit 124R, 124L display element driving unit 125R, 125L display element 130 control unit 131R, 131L, 132R, 132L Distortion correction table unit 143L Video composition unit

Claims (9)

Processing means for performing low-pass filter processing on a composite image obtained by combining the first image generated by imaging and the second image generated by a method other than imaging;
The processing means performs low-pass filter processing with mutually different cutoff frequencies on the first image region and the second image region of the composite image. Combining means for combining the first image generated by imaging and the second image generated by a method other than imaging;
Processing means for performing low-pass filter processing on the image,
The processing means performs low-pass filter processing with different cutoff frequencies on the first image and the second image,
The image synthesizing device synthesizes the first image and the second image, each of which has been subjected to low-pass filter processing by the processing means.   The processing means sets a cut-off frequency of low-pass filter processing for the first image region to a first cut-off frequency, and sets the cut-off frequency for the second image region to the first cut-off frequency. The image processing apparatus according to claim 1, wherein the second cutoff frequency is set higher than the frequency. Separating means for separating the region of the first image and the region of the second image from the composite image;
2. The image processing according to claim 1, further comprising: a recombining unit that recombines the regions of the first and second images subjected to the low-pass filter processing by the processing unit to generate a composite image. apparatus. Processing means for performing low-pass filter processing on a composite image obtained by combining the first image generated by imaging and the second image generated by a method other than imaging;
The processing means does not perform low-pass filter processing on the region of the second image in the composite image, and performs low-pass filter processing on the region of the first image. . Separating means for separating the region of the first image and the region of the second image from the composite image;
Recombining means for generating a composite image by recombining the region of the first image and the region of the separated second image subjected to low-pass filter processing by the processing unit. The image processing apparatus according to claim 5. Combining means for combining the first image generated by imaging and the second image generated by a method other than imaging;
Processing means for performing low-pass filter processing on the image,
The processing means performs low-pass filter processing on the first image and does not perform low-pass filter processing on the second image.
The image processing apparatus, wherein the synthesizing unit synthesizes the first image that has been subjected to the low-pass filter processing by the processing unit and the second image that has not been subjected to the low-pass filter processing. A memory for storing the first image;
The separation means separates the first image region and the second image region by comparing the first image stored in the memory with the composite image. Item 7. The image processing apparatus according to Item 4 or 6. An image processing device according to any one of claims 1 to 8,
An imaging system for performing imaging for generating the first image;
An image display device comprising: a display element that displays the composite image.