JP2009021771A - Video adjustment method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To smoothly connect correction data of photographic areas adjacent to each other and to obtain a smooth video on a screen. <P>SOLUTION: In this video adjustment method for projecting the video on the screen using at least three or more projection parts, photographing a displayed test pattern by a photography part to acquire the photographed image, and calculating the correction data for correcting display characteristics of a video display device based on the acquired photographed image, one or more parts whose projection ranges are adjacent to each other are photographed, respectively, pieces of temporary correction data corresponding to the respective photography parts are generated, the parts whose photographic ranges are adjacent to each other and video projection areas around them are divided into either of at least two kinds of an area for selecting either of pieces of the temporary correction data to be considered as the correction data or an area for separately generating the correction data, and the correction data of the area for separating the correction data is generated based on the temporary correction data. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image adjustment method, and more particularly to an adjustment method and an image display device for smoothly joining luminance, color, and geometry on a screen in an image display device that projects images on a screen with a plurality of projectors.

2. Description of the Related Art As a video display device that projects video with a projector, a video display device that uses a plurality of projectors and arranges the projected video of each projector on a screen and displays it on a large high-definition screen is known. In such a video display device, there are individual differences among a plurality of projectors, and the characteristics of the video projected on the screen and the like are different because each region has different characteristics even within one projector. There is a difference in brightness. Therefore, it is important to correct this difference in order to improve video quality.
As one of the methods, as disclosed in Patent Document 1, it is known that correction data is automatically generated using a result obtained by photographing a test pattern with a camera.

In recent years, video display devices in various forms such as curved screens have been demanded, and due to physical restrictions on the installation of projectors and cameras, shooting data obtained by shooting a test pattern is an image related to the test pattern data. In some cases, not all of the above could be included normally.
For example, Patent Document 2 discloses correction data that can calculate correction data related to an entire image even when shooting data obtained by shooting a test pattern cannot normally include all of the images related to the test pattern data. A calculation method is disclosed.

International Publication Number WO99 / 31877 JP 2005-20314 A

By the way, if the shooting data obtained by shooting the test pattern cannot include all of the images related to the test pattern data, or if you want to reduce the shooting area in order to improve the accuracy of the shooting data, the shooting is divided into multiple areas. It needs to be done separately.
However, when shooting is performed in a plurality of areas, the correction data characteristics of two or more shooting areas may greatly differ due to differences in the shooting posture of the camera. In such a case, there has been a problem that the correction becomes unnatural at a portion where the plurality of photographing regions are adjacent to each other and the periphery thereof.

SUMMARY OF THE INVENTION An object of the present invention is to provide an adjustment method and a video display apparatus that smoothly and naturally correct a portion where the photographing regions are adjacent to each other and its periphery when photographing is performed in a plurality of regions.
Another object of the present invention is to provide an adjustment method and an image display device that reduce a difference in correction data characteristics for each imaging region.

The video adjustment method according to the present invention projects video on a screen using at least three or more projection units, captures the displayed test pattern with the imaging unit, acquires the captured image, and based on the acquired captured image A video adjustment method for calculating correction data for correcting display characteristics of the video display device, wherein one or more locations where video projection ranges are adjacent to each other are captured and provisional correction data corresponding to each shooting location Produces
Divide the area where the shooting range is adjacent and the surrounding video projection range into at least two types of areas: the area for selecting correction data by selecting any temporary correction data and the area for generating correction data separately And
Based on the provisional correction data, correction data for a region to be generated separately is generated.
The video adjustment method according to the present invention is such that when two or more shooting ranges are required, the hues of the target colors targeted in different shooting ranges are determined so as not to change from each other.

  The projection display device according to the present invention projects a video on a screen using at least three or more projection units, captures the displayed test pattern with the imaging unit, acquires a captured image, and based on the acquired captured image In the projection display device that corrects the display characteristics of the video display device and projects an image according to the corrected display characteristics, each of the video projection ranges is photographed to correspond to each photographing location. Means for generating provisional correction data, at least a region where the shooting range is adjacent and a video projection range in the vicinity thereof, and selecting any one of the provisional correction data as correction data and a region for generating correction data separately; A means for dividing the image into one of two types of areas; a means for generating correction data for a separately generated area based on the temporary correction data; and an image corrected according to the correction data. A projection unit for projection of, configured as an image display device having a.

  According to the present invention, when shooting is performed in a plurality of areas, different correction data are smoothly connected, and the difference between different correction data characteristics is reduced, so that the image projected on the screen is smoothly connected. There are advantages.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Example 1]
FIG. 1 is a video adjustment system according to an embodiment of the present invention.
The video projection ranges 211, 212, and 213 are video projection ranges that the video projection units 201, 202, and 203 project on the screen 100, respectively. The video projection units 211, 212, and 213 and the screen 100 are arranged so that the video projection ranges 211 and 212 overlap each other on the screen 100 and the video projection ranges 212 and 213 overlap each other on the screen 100. .

Each of the video projection units 201 to 203 has a function of holding correction data and projecting a video corrected based on the correction data. Each video projection unit may be a single projector with a correction function, or may be a combination of a video generator that outputs a video signal corrected based on correction data and a normal projector.
Here, the screen 100 is assumed to be of a cylindrical surface type, but may be of another curved surface type such as a spherical surface or a flat surface type. Further, it may be a type that projects an image from the front or a type that projects from the back.

The shooting ranges 311 and 312 are ranges where the shooting units 301 and 302 shoot, respectively.
The photographing units 301 and 302 each include a lens and have a function of saving a photographed image obtained by photographing as digital data. For example, a single-lens reflex digital camera. The photographing unit 301 and the photographing unit 302 may be disposed when performing photographing for generating correction data, and may be removed from the system configuration after photographing. In addition, the photographing unit 302 may move the photographing unit 301 to be the photographing unit 302, or the photographing unit 302 may be configured by preparing a photographing unit different from the photographing unit 301.

  The data processing unit 40 is a processing device that performs processing such as generation of correction data, and includes, for example, a personal computer. The data processing unit 40 is connected to the video projection units 201, 202, 203 and the imaging units 301, 302 via a network, a bus, or the like. The data processing unit 40 may be configured as a single unit, or may be configured by incorporating the function in any one or a plurality of units of the video projection units 201, 202, and 203 and the imaging units 301 and 302. .

FIG. 2 shows a processing flow of video adjustment in one embodiment.
The video adjustment processing includes processing (S100) of shooting the first location where the video projection range is adjacent and generating first temporary correction data corresponding to the location, and the second location where the video projection range is adjacent. At least two types of processing (S200) for generating second provisional correction data corresponding to the location by photographing and which of the provisional correction data is used for each region or generation of correction data separately And a process (S400) of generating correction data relating to a region classified as separately generated based on correction data of the other classified region (S400).
The first temporary correction data generation process S100 includes a process of performing a geometric adjustment process in the photographing unit 301 (S110) and a process of performing a brightness / color adjustment process in the photographing unit 301 (S120).
The classification process S300 and the correction data generation process S400 for each region are realized by executing a predetermined application program by the processor of the data processing unit 40.
Hereinafter, each process will be described in detail.

First, the geometric adjustment process (S110) in the photographing unit 301 will be described with reference to FIG.
As shown in FIG. 3, the size of the video projected by the video projection unit 201 is the horizontal w _P1 pixel and the vertical h _P1 pixel, and the size of the video projected by the video projection unit 202 is the horizontal w _P2 pixel, the vertical h _P2 pixel, and the video projection. The size of the image projected by the unit 203 is assumed to be horizontal w _P3 pixels and vertical h _P3 pixels. For example, when the image size is XGA (1024 horizontal pixels and 768 vertical pixels), Equation 1 is obtained.

映像投映範囲２１１と２１２が互いに隣り合う領域を、撮影部３０１の撮影範囲３１１が含むように、撮影部３０１の撮影姿勢を決める。撮影姿勢とは、カメラの設置位置、方向、レンズの画角などのことである。

The shooting posture of the shooting unit 301 is determined so that the shooting range 311 of the shooting unit 301 includes a region where the video projection ranges 211 and 212 are adjacent to each other. The shooting posture refers to the installation position and direction of the camera, the angle of view of the lens, and the like.

As illustrated in FIG. 3, the size of the image captured by the imaging unit 301 is assumed to be horizontal w _C1 pixels and vertical h _C1 pixels. For example, in the case of an image size of 2000 pixels wide and 1312 pixels long, Equation 2 is obtained.

なお、ここで、撮影部３０１が撮影する一連の画像は、レンズ特有の周辺減光を補正した結果の画像であってもよい。同補正を行う場合は、一連の撮影を通じて、同じ特性の補正を用いるものとする。
映像投映部が投映する映像や、撮影部が撮影する画像の色は色（ｒ，ｇ，ｂ）と表し、ｒ（赤），ｇ（緑），ｂ（青）の値の範囲は、それぞれ［ｒ_ＭＩＮ，ｒ_ＭＡＸ］、［g_ＭＩＮ，g_ＭＡＸ］、［b_ＭＩＮ，b_ＭＡＸ］とし、例えば、数３のような値とする。

Here, the series of images photographed by the photographing unit 301 may be images obtained as a result of correcting the peripheral darkening specific to the lens. When the correction is performed, correction with the same characteristics is used throughout a series of photographing.
The color of the video projected by the video projection unit and the color of the image captured by the imaging unit are represented as colors (r, g, b), and the range of values of r (red), g (green), and b (blue) are respectively [R _MIN , r _MAX ], [g _MIN , g _MAX ], [b _MIN , b _MAX ], for example, values as shown in Equation 3.

映像投映部２０１が投映する映像の座標をＰ_１（ｘ_Ｐ１，ｙ_Ｐ１）、映像投映部２０２が投映する映像の座標をＰ_２（ｘ_Ｐ２，ｙ_Ｐ２）、撮影部３０１が撮影する画像の座標をＣ_１（ｘ_Ｃ１，ｙ_Ｃ１）と表す。
映像投映範囲２１１と２１２がスクリーン１００上で互いに隣り合っており、撮影範囲３１１がその互いに隣り合う領域を含んでいるので、Ｐ_１の集合とＰ_２の集合とＣ_１の集合には、互いに写像の関係がある。

The coordinates of the video projected by the video projection unit 201 are P ₁ (x _P1 , y _P1 ), the coordinates of the video projected by the video projection unit 202 are P ₂ (x _P2 , y _P2 ), and the image captured by the imaging unit 301 The coordinates are represented as C ₁ (x _C1 , y _C1 ).
Since the image projection ranges 211 and 212 are adjacent to each other on the screen 100 and the shooting range 311 includes the adjacent regions, the set of P ₁ , the set of P ₂ , and the set of C ₁ include There is a mapping relationship.

The mapping from a set of P ₁ to a collection of _{C 1 f} C1P1, represents a set of _{P 2} the mapping to a set of _{C 1} and _{f C1P2,} a mapping from the set of _{C 1} to the set of _{P 1} ^{f -1} _C1P1, represents a collection of _{C 1} mapping to a set of _{P 2} and ^f _{-1 C1P2.}
The relationship between these mappings and coordinates is expressed as in Expression 4.

図４のように、撮影領域３１１の内、映像投映範囲２１１が占める領域をマスク３２１とし、撮影領域３１１の内、映像投映範囲２１２が占める範囲をマスク３２２とする。
ここで、マスク３２１と３２２を撮影部３０１の座標Ｃ_１（ｘ_Ｃ１，ｙ_Ｃ１）の関数ｍ_Ｃ１Ｐ１、ｍ_Ｃ１Ｐ２でそれぞれ数５のように定義する。

As shown in FIG. 4, the area occupied by the video projection range 211 in the shooting area 311 is set as a mask 321, and the range occupied by the video projection range 212 in the shooting area 311 is set as a mask 322.
Here, the masks 321 and 322 are defined by the functions m _C1P1 and m _C1P2 of the coordinates C ₁ (x _C1 , y _C1 ) of the imaging unit 301 as _shown in _Equation 5, respectively.

例えば、特開２００５−５１３７５公報に記載された方法を用いて、撮影部２０１から投映したテストパターンを撮影部３０１で撮影し、データ処理部４００で処理すると、ｆ_Ｃ１Ｐ１とｆ^−１ _Ｃ１Ｐ１とｍ_Ｃ１Ｐ１を定義できる。同様に、同公報に記載の方法を用いて、撮影部２０２から投映したテストパターンを撮影部３０１で撮影し、データ処理部４００で処理すると、ｆ_Ｃ１Ｐ２とｆ^−１ _Ｃ１Ｐ２とｍ_Ｃ１Ｐ２を定義できる。

For example, when a test pattern projected from the photographing unit 201 is photographed by the photographing unit 301 and processed by the data processing unit 400 using the method described in Japanese Patent Application Laid-Open No. 2005-51375, f _C1P1 and f ⁻¹ _C1P1 and m _C1P1 can be defined. Similarly, when the test pattern projected from the photographing unit 202 is photographed by the photographing unit 301 and processed by the data processing unit 400 using the method described in the publication, f _C1P2 , f ⁻¹ _C1P2 and m _C1P2 can be defined. .

Note that the OR region of the masks 321 and 322 is a mask 325, the AND region is a mask 326, and the region obtained by removing the mask 326 from the mask 325 is a mask 327 for use in later calculations.
When the masks ₃₂₅ , ₃₂₆ , and ₃₂₇ are represented by functions m ₃₂₅ , m ₃₂₆ , and m ₃₂₇ of the coordinates C ₁ (x _C1 , y _C1 ) of the imaging unit 301, the following expression 6 is obtained.

撮影部３０１での明るさ・色調整処理を行うＳ１２０について、図５により説明する。
ここで、合わせたい色を（ｒ_Ｔ，ｇ_Ｔ，ｂ_Ｔ）とする。なお、合わせたい色の数は２つ以上でも構わない。また、合わせたい色の数やそれぞれの合わせたい色の値は、予め決めておいてもよいし、ユーザに入力させてもよい。
処理Ｓ１２１００で、合わせたい色が残っている場合、処理Ｓ１２２００に進み、残っていない場合には終了する。

S120 for performing brightness / color adjustment processing in the photographing unit 301 will be described with reference to FIG.
Here, it is assumed that the color to be matched is (r _T , g _T , b _T ). Note that the number of colors to be matched may be two or more. The number of colors to be matched and the value of each color to be matched may be determined in advance or may be input by the user.
If it is determined in step S12100 that the color to be matched remains, the process proceeds to step S12200, and if not, the process ends.

  In step S12200, exposure setting processing is performed. The exposure (aperture and shutter speed) of the photographing unit 301 is fixed while maintaining the same photographing posture as in S110. During the processing of S120, it is desirable to make the exposure so that the image to be photographed does not saturate or become completely black.

Next, the first target color determination process (S12300) will be described with reference to FIG.
If it is determined in S12310 that the color to be matched is black (0, 0, 0), the process proceeds to S12330, and if it is not black, the process proceeds to S12320.
In S12320, if the color to be matched is white (255, 255, 255), the process proceeds to S12340, and if it is other than white, the process proceeds to S12350.
Here, the first target color determination image is an image 330, and the color of the coordinates C ₁ (x _C1 , y _C1 ) of the image 330 is (r ₃₃₀ (x _C1 , y _C1 ), g ₃₃₀ (x _C1 , y _C1 ), b ₃₃₀ (x _C1 , y _C1 )). The values of r ₃₃₀ , g ₃₃₀ , and b ₃₃₀ include the imaging results in S12330, 12340, or 12350.

The first target color determination process (S12330) in the case of black will be described.
A black (0, 0, 0) solid image is displayed from each of the video projection units 201 and 202, the projection areas 211 and 212 projected on the screen are photographed by the photographing unit 301, and the photographed image 330 is stored. To do.
m ₃₂₅ (x, y) of all the coordinates satisfying = 1 (x, y) relates, in descending order of the value of the green component _g 330 (x, y), color _{(r 330 (x, y)} , g 330 (x , Y), b ₃₃₀ (x, y)) are sorted, and (x, y) = (x, y) = (0.05), the magnitude of the value of the green component g ₃₃₀ (x, y) is the upper 0.05%. (X ₃₃₀ , y ₃₃₀ ).
The colors (r ₃₃₀ (x ₃₃₀ , y ₃₃₀ ), g ₃₃₀ (x ₃₃₀ , y ₃₃₀ ), b ₃₃₀ (x ₃₃₀ , y ₃₃₀ )) at this time are set as target colors.
According to the above procedure, among the colors in the area where the video projection range exists in the image 330, the color having the highest brightness of 0.05% is the target color.

  For example, if the area occupied by the mask 325 in the image 330 is 60% of the whole, the upper 0.05% pixels are the upper 788th pixels as shown in Equation 7.

一番上位の色ではなく、上位０．０５％の色を用いているのは、撮影した画像のノイズや、撮影部の素子不良などによる輝点の悪影響を避けるためである。もちろん、０．０５％の値は違う値を用いてもよい。

The reason why the upper 0.05% color is used instead of the uppermost color is to avoid the adverse effect of bright spots due to noise in the photographed image and defective elements in the photographing section. Of course, a different value may be used for the value of 0.05%.

Next, the first target color determination process (S12340) in the case of white will be described.
White (255, 255, 255) solid images are displayed from the video projection units 201 and 202, respectively, and the imaging unit 301 captures the projection areas 211 and 212 on the screen, and stores the captured image 330. To do.
m ₃₂₇ (x, y) of all the coordinates satisfying = 1 (x, y) relates, in descending order of the value of the green component _g 330 (x, y), color _{(r 330 (x, y)} , g 330 (x , Y), b ₃₃₀ (x, y)), and the value of the green component g ₃₃₀ (x, y) is the lower 0.5% of (x, y) (x, y) = (X ₃₃₀ , y ₃₃₀ ).
The colors (r ₃₃₀ (x ₃₃₀ , y ₃₃₀ ), g ₃₃₀ (x ₃₃₀ , y ₃₃₀ ), b ₃₃₀ (x ₃₃₀ , y ₃₃₀ )) at this time are set as target colors.
According to the above procedure, among the colors in the area where the video projection range exists in the image 330 and the video projection range does not overlap, the target color is the color whose green brightness is lower 0.5%.

  The part that overlaps the video projection range is not included because the correction range is wider than the part that does not overlap the video projection range, and the wider correction range can be adjusted to the narrower one. This is because it may not be possible to match the narrower range to the wider range.

  For example, if the area occupied by the mask 327 in the image 330 is 50% of the entire area, the lower 0.5% pixel is the lower 6560th pixel as shown in Equation 8.

一番下位の色ではなく、下位０．５％の色を用いているのは、撮影した画像のノイズや、撮影部の素子不良などによる暗点の悪影響を避けるためである。もちろん、０．５％の値は違う値を用いてもよい。

The reason why the lower 0.5% color is used instead of the lowest color is to avoid the adverse effects of dark spots due to noise in the captured image and element defects in the imaging unit. Of course, a different value may be used for the value of 0.5%.

Next, the first target color determination process (S12350) in the normal case will be described.
Solid images of color (r, g, b) are displayed from the video projection units 201 and 202, respectively, and the imaging unit 301 captures the projection ranges 211 and 212 reflected on the screen, and stores the captured image 330. To do.
m ₃₂₇ (x, y) of all the coordinates satisfying = 1 (x, y) relates, in descending order of the value of the green component _g 330 (x, y), color _{(r 330 (x, y)} , g 330 (x , Y), b ₃₃₀ (x, y)) are sorted, and (x, y) = (x, y) where the magnitude of the value of the green component g ₃₃₀ (x, y) is the top 50% ₃₃₀ , _y330 ). The colors (r ₃₃₀ (x ₃₃₀ , y ₃₃₀ ), g ₃₃₀ (x ₃₃₀ , y ₃₃₀ ), b ₃₃₀ (x ₃₃₀ , y ₃₃₀ )) at this time are set as target colors.

According to the above procedure, among the colors in the range where the projection area in the image 330 exists and the projection areas do not overlap, the color having the highest brightness of green is the target color. Of course, a different value may be used for the value of 50%.
The part where the video projection range overlaps is not included because the part where the video projection range overlaps is wider than the part where the video projection range does not overlap, and it is possible to match the wider correction range to the narrower one, This is because it may not be possible to match the narrower correction range to the wider correction range.

The reason why sorting is performed using only the green component in S12330, S12340, and S12350 is that the green value is most effective for luminance. In addition, as a method different from the method of sorting using only the green component, the colors (r ₃₃₀ , g ₃₃₀ , b ₃₃₀ ) are converted into a light quantity proportional space, and further converted into an HSV color space, in order of increasing brightness. It is also possible to use colors with higher or lower brightness. In S12350, instead of using the top 50% colors, the target colors for white and black are determined first, and the target colors of the colors (r _T , g _T , b _T ) to be combined are interpolated from the results. You may ask for it. As yet another method, the target color may be specified by the user in S12330, S12340, and S12350.

When the target color is represented as (r _G1 , g _G1 , b _G1 ), the following equation 9 is obtained.

次に、図７を参照して、フィードバックループ処理（Ｓ１２４００）について説明する。
Ｓ１２４１０で、整数ｉを変数とし、ｉの値を１とする。Ｓ１２４２０で、数列ｓのｊ番目の要素をｓ（ｊ）と表すとき、数１０を満たす数列ｓを定義する。

Next, the feedback loop process (S12400) will be described with reference to FIG.
In step S12410, the integer i is set as a variable, and the value of i is set to 1. In S12420, when the j-th element of the sequence s is expressed as s (j), the sequence s satisfying Equation 10 is defined.

ここで、数列ｓと要素数Ｎを数１１とする。

Here, the sequence s and the number N of elements are represented by Formula 11.

横ｗ_Ｐ１ピクセル、縦ｈ_Ｐ１ピクセルのサイズのフィードバックループ用投映画像を画像２２１とし、画像２２１の座標（ｘ_Ｐ１，ｙ_Ｐ１）の色を（ｒ_２２１（ｘ_Ｐ１，ｙ_Ｐ１），ｇ_２２１（ｘ_Ｐ１，ｙ_Ｐ１），ｂ_２２１（ｘ_Ｐ１，ｙ_Ｐ１））と定義する。また、横ｗ_Ｐ２ピクセル、縦ｈ_Ｐ２ピクセルのサイズのフィードバックループ用投映画像を画像２２２とし、画像２２２の座標（ｘ_Ｐ２，ｙ_Ｐ２）の色を（ｒ_２２２（ｘ_Ｐ２，ｙ_Ｐ２），ｇ_２２２（ｘ_Ｐ２，ｙ_Ｐ２），ｂ_２２２（ｘ_Ｐ２，ｙ_Ｐ２））と定義する。

The projected image for the feedback loop having the size of the horizontal w _P1 pixel and the vertical h _P1 pixel is set as an image 221, and the color of the coordinates (x _P1 , y _P1 ) of the image 221 is (r ₂₂₁ (x _P1 , y _P1 ), g ₂₂₁ ( _xP1 , _yP1 ), _b221 ( _xP1 , _yP1 )). A projected image for feedback loop having a size of horizontal w _P2 pixels and vertical h _P2 pixels is set as an image 222, and the color of the coordinates (x _P2 , y _P2 ) of the image 222 is set to (r ₂₂₂ (x _P2 , y _P2 ), g ₂₂₂ ( _xP2 , _yP2 ), _b222 ( _xP2 , _yP2 )).

In S12430, initial values of the projected images for the feedback loop are set.
The images 221 and 222 are black images, and initial values are set as shown in Equation 12.

Ｓ１２４４０で、画像２２１を投映部２０１から表示し、画像２２２を投映部２０２から表示し、スクリーンに映った投映範囲２１１、２１２の様子を撮影部３０１で撮影して、撮影されたフィードバックループ用画像である画像３４０を保存する。
画像３４０は横ｗ_Ｃ１ピクセル、縦ｈ_Ｃ１ピクセルのサイズであり、画像３４０の座標（ｘ_Ｃ１，ｙ_Ｃ１）の色を（ｒ_３４０（ｘ_Ｃ１，ｙ_Ｃ１），ｇ_３４０（ｘ_Ｃ１，ｙ_Ｃ１），ｂ_３４０（ｘ_Ｃ１，ｙ_Ｃ１））と表す。ｒ_３４０、ｇ_３４０、ｂ_３４０の値に関し、横ｗ_Ｃ１ピクセル、縦ｈ_Ｃ１ピクセルのサイズを超える部分について、数１３のように定義する。

In step S12440, the image 221 is displayed from the projection unit 201, the image 222 is displayed from the projection unit 202, the state of the projection ranges 211 and 212 on the screen is captured by the imaging unit 301, and the captured feedback loop image The image 340 is saved.
The image 340 has a size of horizontal w _C1 pixels and vertical h _C1 pixels, and the coordinates (x _C1 , y _C1 ) of the image 340 are changed to (r ₃₄₀ (x _C1 , y _C1 ), g ₃₄₀ (x _C1 , y _C1). ), B ₃₄₀ (x _C1 , y _C1 )). With respect to the values of r ₃₄₀ , g ₃₄₀ , and b ₃₄₀ , a portion exceeding the size of the horizontal w _C1 pixel and the vertical h _C1 pixel is defined as in Expression 13.

Ｓ１２４５０で、画像３４０に映っている映像投映範囲２１１、２１２の画素の比較・更新をする。映像投映範囲２１１に関して、以下のように、画素比較・更新をする。
０≦ｘ_２２１＜ｗ_Ｐ１、０≦ｙ_２２１＜ｈ_Ｐ１を満たす整数ｘ_２２１、ｙ_２２１に関し、（ｘ_２２１，ｙ_２２１）に対応する撮影範囲座標を（ｘ_３４０，ｙ_３４０）とすると、図８のように、数１４の関係がある。

In step S12450, the pixels in the video projection ranges 211 and 212 shown in the image 340 are compared and updated. For the video projection range 211, pixel comparison / update is performed as follows.
With respect to the integers x ₂₂₁ and y ₂₂₁ that satisfy 0 ≦ x ₂₂₁ <w _P1 and 0 ≦ y ₂₂₁ <h _P1 , assuming that the shooting range coordinates corresponding to (x ₂₂₁ , y ₂₂₁ ) are (x ₃₄₀ , y ₃₄₀ ), FIG. As shown in FIG.

ｘ_３４０やｙ_３４０は常に整数となるとは限らないので、画像３４０の座標（ｘ_３４０，ｙ_３４０）の色（ｒ，ｇ，ｂ）を、バイリニア方式で、図１４のように４近傍の画素を用いて、数１５のように定義する。

Since x ₃₄₀ and y ₃₄₀ are not always integers, the colors (r, g, b) of the coordinates (x ₃₄₀ , y ₃₄₀ ) of the image 340 are represented by four neighboring pixels as shown in FIG. Is defined as shown in Equation 15.

色（ｒ，ｇ，ｂ）と目標色（ｒ_Ｇ１，ｇ_Ｇ１，ｂ_Ｇ１）を比較する。
ｒ＜ｒ_Ｇ１のとき、ｒ_２２１（ｘ_２２１，ｙ_２２１）を変分ｓ（ｉ）だけ増やす。増やした後にｒ_２２１（ｘ_２２１，ｙ_２２１）がｒ_ＭＡＸを超えたときはｒ_ＭＡＸとする。
ｒ＞ｒ_Ｇ１のとき、ｒ_２２１（ｘ_２２１，ｙ_２２１）を変分ｓ（ｉ）だけ減らす。減らした後にｒ_２２１（ｘ_２２１，ｙ_２２１）がｒ_ＭＩＮを下回るときはｒ_ＭＩＮとする。
ｇ＜ｇ_Ｇ１のとき、ｇ_２２１（ｘ_２２１，ｙ_２２１）を変分ｓ（ｉ）だけ増やす。増やした後にｇ_２２１（ｘ_２２１，ｙ_２２１）がｇ_ＭＡＸを超えたときはｇ_ＭＡＸとする。
ｇ＞ｇ_Ｇ１のとき、ｇ_２２１（ｘ_２２１，ｙ_２２１）を変分ｓ（ｉ）だけ減らす。減らした後にｇ_２２１（ｘ_２２１，ｙ_２２１）がｇ_ＭＩＮを下回るときはｇ_ＭＩＮとする。
ｂ＜ｂ_Ｇ１のとき、ｂ_２２１（ｘ_２２１，ｙ_２２１）を変分ｓ（ｉ）だけ増やす。増やした後にｂ_２２１（ｘ_２２１，ｙ_２２１）がｂ_ＭＡＸを超えたときはｂ_ＭＡＸとする。
ｂ＞ｂ_Ｇ１のとき、ｂ_２２１（ｘ_２２１，ｙ_２２１）を変分ｓ（ｉ）だけ減らす。減らした後にｂ_２２１（ｘ_２２１，ｙ_２２１）がｂ_ＭＩＮを下回るときはｂ_ＭＩＮとする。

The color (r, g, b) is compared with the target color (r _G1 , g _G1 , b _G1 ).
When r <r _G1 , r ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) is increased by the variation s (i). When r ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) exceeds r _MAX after the increase, it is set as r _MAX .
When r> r _G1 , r ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) is reduced by the variation s (i). If r ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) falls below r _MIN after the reduction, it is set as r _MIN .
When g <g _G1 , g ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) is increased by the variation s (i). When g ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) exceeds g _MAX after increasing, it is set as g _MAX .
When g> g _G1 , g ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) is reduced by the variation s (i). When g ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) falls below g _MIN after the reduction, it is set as g _MIN .
When b <b _G1 , b ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) is increased by the variation s (i). When b ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) exceeds b _MAX after the increase, b _MAX is set.
When b> b _G1 , b ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) is reduced by the variation s (i). If b ₂₂₁ (x ₂₂₁ , y ₂₂₁ ) falls below b _MIN after the reduction, b _MIN is set.

When the coordinates (x ₂₂₁ , y ₂₂₁ ) are on the mask 326 where the video projection range overlaps, ρ _L s (i) (ρ _L is 0 <ρ instead of the variation s (i). _A real number satisfying _L <1) may be used. The value of [rho _L is, for example, a [rho _{L =} 0.5, or the like may be used value that varies according to the distance from the point where the range movies PICTURE began overlap.

As for the video projection range 212, the pixel comparison / update is performed as follows in the same manner as the video projection range 211.
_Assuming that the shooting range coordinates corresponding to (x ₂₂₂ , y ₂₂₂ ) satisfying 0 ≦ x ₂₂₂ <w _P2 and 0 ≦ y ₂₂₂ <h _P2 with respect to the integers x ₂₂₂ and y ₂₂₂ are (x ₃₄₀ , y ₃₄₀ ), FIG. As shown in FIG.

画像３４０の座標（ｘ_３４０，ｙ_３４０）の色（ｒ，ｇ，ｂ）は、数１５で定まる。
色（ｒ，ｇ，ｂ）と目標色（ｒ_Ｇ１，ｇ_Ｇ１，ｂ_Ｇ１）を比較する。
ｒ＜ｒ_Ｇ１のとき、ｒ_２２２（ｘ_２２２，ｙ_２２２）を変分ｓ（ｉ）だけ増やす。増やした後にｒ_２２２（ｘ_２２２，ｙ_２２２）がｒ_ＭＡＸを超えたときはｒ_ＭＡＸとする。
ｒ＞ｒ_Ｇ１のとき、ｒ_２２２（ｘ_２２２，ｙ_２２２）を変分ｓ（ｉ）だけ減らす。減らした後にｒ_２２２（ｘ_２２２，ｙ_２２２）がｒ_ＭＩＮを下回るときはｒ_ＭＩＮとする。
ｇ＜ｇ_Ｇ１のとき、ｇ_２２２（ｘ_２２２，ｙ_２２２）を変分ｓ（ｉ）だけ増やす。増やした後にｇ_２２２（ｘ_２２２，ｙ_２２２）がｇ_ＭＡＸを超えたときはｇ_ＭＡＸとする。
ｇ＞ｇ_Ｇ１のとき、ｇ_２２２（ｘ_２２２，ｙ_２２２）を変分ｓ（ｉ）だけ減らす。減らした後にｇ_２２２（ｘ_２２２，ｙ_２２２）がｇ_ＭＩＮを下回るときはｇ_ＭＩＮとする。
ｂ＜ｂ_Ｇ１のとき、ｂ_２２２（ｘ_２２２，ｙ_２２２）を変分ｓ（ｉ）だけ増やす。増やした後にｂ_２２２（ｘ_２２２，ｙ_２２２）がｂ_ＭＡＸを超えたときはｂ_ＭＡＸとする。
ｂ＞ｂ_Ｇ１のとき、ｂ_２２２（ｘ_２２２，ｙ_２２２）を変分ｓ（ｉ）だけ減らす。減らした後にｂ_２２２（ｘ_２２２，ｙ_２２２）がｂ_ＭＩＮを下回るときはｂ_ＭＩＮとする。

The color (r, g, b) of the coordinates (x ₃₄₀ , y ₃₄₀ ) of the image 340 is determined by Equation 15.
The color (r, g, b) is compared with the target color (r _G1 , g _G1 , b _G1 ).
When r <r _G1 , r ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) is increased by the variation s (i). When r ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) exceeds r _MAX after the increase, it is set as r _MAX .
When r> r _G1 , r ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) is reduced by the variation s (i). If r ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) falls below r _MIN after the decrease, it is set as r _MIN .
When g <g _G1 , g ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) is increased by the variation s (i). When g ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) exceeds g _MAX after the increase, it is set as g _MAX .
When g> g _G1 , g ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) is reduced by the variation s (i). If g ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) falls below g _MIN after the decrease, it is set as g _MIN .
When b <b _G1 , b ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) is increased by the variation s (i). If b ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) exceeds b _MAX after the increase, b _MAX is set.
When b> b _G1 , b ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) is reduced by the variation s (i). If b ₂₂₂ (x ₂₂₂ , y ₂₂₂ ) falls below b _MIN after the reduction, b _MIN is set.

When the coordinates (x ₂₂₂ , y ₂₂₂ ) are on the mask 326 where the video projection range overlaps, instead of the variation s (i), σ _L s (i) (σ _L is 0 <σ _A real number satisfying _L <1) may be used. The value of sigma _L is for example the use of σ _L = 1-ρ _L.

In S12470, the sizes of i and N are compared. When i <N, the process proceeds to S12480, and when i <N is not satisfied, S12400 is terminated. In S12480, the value of i is incremented by 1, and the process returns to the next S12440 to loop. When returning to S12440, a smoothing filter may be applied to the images 221 and 222, respectively.
By repeating the procedure of S12440 to 12480, the color of the area in which the video projection ranges 211 and 212 are captured in the captured image 340 approaches the target color (r _G1 , g _G1 , b _G1 ).
Images 221 and 222 at the end of S12400 are provisional correction data of the colors (r _T , g _T , b _T ) to be matched in the photographing unit 301.

Next, the geometric adjustment process (S210) in the imaging unit 302 will be described.
The shooting posture of the shooting unit 302 is determined so that the shooting range 312 of the shooting unit 302 includes a region where the video projection ranges 212 and 213 are adjacent to each other. It is desirable that the photographing unit 302 has characteristics similar to those of the photographing unit 301. Here, the photographing unit 301 is moved to be the photographing unit 302.

As shown in FIG. 3, the size of the image captured by the imaging unit 302 is assumed to be horizontal w _C2 pixels and vertical h _C2 pixels.
Since the photographing unit 301 is moved to the photographing unit 302, Expression 17 is obtained.

撮影部３０２が撮影する一連の画像は、レンズ特有の周辺減光を補正した結果の画像であってもよい。同補正を行う場合は、一連の撮影を通じて、撮影部３０１での撮影と同じ特性の補正を用いるものとする。
映像投映部２０３が投映する映像の座標をＰ_３（ｘ_Ｐ３，ｙ_Ｐ３）、撮影部３０２が撮影する画像の座標をＣ_２（ｘ_Ｃ２，ｙ_Ｃ２）と表す。映像投映範囲２１２と２１３がスクリーン１００上で互いに隣り合っており、撮影範囲３１２がその互いに隣り合う領域を含んでいるので、Ｐ_２の集合とＰ_３の集合とＣ_２の集合には、互いに写像の関係がある。

The series of images photographed by the photographing unit 302 may be an image obtained as a result of correcting the peripheral darkening specific to the lens. In the case of performing the correction, it is assumed that correction with the same characteristics as the shooting by the shooting unit 301 is used through a series of shootings.
The coordinates of the video projected by the video projection unit 203 are represented as P ₃ (x _P3 , y _P3 ), and the coordinates of the image captured by the imaging unit 302 are represented as C ₂ (x _C2 , y _C2 ). Since the image projection ranges 212 and 213 are adjacent to each other on the screen 100 and the shooting range 312 includes the adjacent regions, the set of P ₂ , the set of P ₃ , and the set of C ₂ include There is a mapping relationship.

The mapping from a set of P ₂ to a collection of _{C 2 f} C2P2, represents a set of _{P 3} a mapping to a set of _{C 2} and _{f C2P3,} a mapping from the set of _{C 2} to a set of _{P 2} ^{f -1} _C2P2, represents a collection of _{C 2} to mapping to a set of _{P 3} and ^f _{-1 C2P3.}
Expression 18 represents the relationship between these maps and coordinates.

図９のように、撮影領域３１２の内、映像投映範囲２１２が占める領域をマスク３５２とし、撮影領域３１２の内、映像投映範囲２１３が占める範囲をマスク３５３とする。
マスク３５２と３５３を撮影部３０２の座標Ｃ_２（ｘ_Ｃ２，ｙ_Ｃ２）の関数ｍ_Ｃ２Ｐ２、ｍ_Ｃ２Ｐ３でそれぞれ数１９のように定義する。

As shown in FIG. 9, the area occupied by the video projection range 212 in the shooting area 312 is referred to as a mask 352, and the area occupied by the video projection range 213 in the shooting area 312 is referred to as a mask 353.
The masks 352 and 353 are defined by the functions m _C2P2 and m _C2P3 of the coordinates C ₂ (x _C2 , y _C2 ) of the imaging unit 302 as _shown in _Equation 19, respectively.

例えば、特開２００５−５１３７５公報に記載された方法を用いて、撮影部２０２から投映したテストパターンを撮影部３０２で撮影し、データ処理部４００で処理すると、ｆ_Ｃ２Ｐ２とｆ^−１ _Ｃ２Ｐ２とｍ_Ｃ２Ｐ２を定義できる。同様に、同公報に記載の方法を用いて、撮影部２０３から投映したテストパターンを撮影部３０２で撮影し、データ処理部４００で処理すると、ｆ_Ｃ２Ｐ３とｆ^−１ _Ｃ２Ｐ３とｍ_Ｃ２Ｐ３を定義できる。

For example, when a test pattern projected from the photographing unit 202 is photographed by the photographing unit 302 and processed by the data processing unit 400 using the method described in Japanese Patent Laid-Open No. 2005-51375, f _C2P2 , f ⁻¹ _C2P2 and m _C2P2 can be defined. Similarly, when the test pattern projected from the photographing unit 203 is photographed by the photographing unit 302 and processed by the data processing unit 400 using the method described in the publication, f _C2P3 , f ⁻¹ _C2P3, and m _C2P3 can be defined. .

Here, the OR region of the masks 352 and 353 is referred to as a mask 355, the AND region is referred to as a mask 356, and the region obtained by removing the mask 356 from the mask 355 is referred to as a mask 357 for use in later calculations.
When the masks ₃₅₅ , ₃₅₆ , and ₃₅₇ are expressed by functions m ₃₅₅ , m ₃₅₆ , and m ₃₅₇ of the coordinates C ₂ (x _C2 , y _C2 ) of the imaging unit 302, the following Expression 20 is obtained.

次に、図１０を参照して、撮影部３０２での明るさ・色調整処理（Ｓ２２０）について説明する。
なお、合わせたい色の数とそれぞれの合わせたい色の値は、Ｓ１２０と同じとする。この例では、合わせたい色は１つで、色（ｒ_Ｔ，ｇ_Ｔ，ｂ_Ｔ）となる。
Ｓ２２１００で、合わせたい色が残っている場合にはＳ２２２００に進み、残っていない場合には終了する。
Ｓ２２２００では、露出設定処理をする。Ｓ２１０００と同じ撮影姿勢に保ったまま、撮影部３０１の露出（絞り、シャッタースピード）を、Ｓ１２２００と同じ値に固定する。

Next, the brightness / color adjustment processing (S220) in the photographing unit 302 will be described with reference to FIG.
Note that the number of colors to be matched and the value of each color to be matched are the same as in S120. In this example, there is one color to be combined, and the color (r _T , g _T , b _T ).
If it is determined in S22100 that the color to be matched remains, the process proceeds to S22200, and if not, the process ends.
In S22200, exposure setting processing is performed. The exposure (aperture and shutter speed) of the photographing unit 301 is fixed to the same value as in S12200 while maintaining the same photographing posture as in S21000.

Here, the second target color determination process (S22300) will be described with reference to FIG.
If the color to be matched is black (0, 0, 0) in S22310, the process proceeds to S22330, and if it is not black, the process proceeds to S22320.
In S22320, if the color to be matched is white (255, 255, 255), the process proceeds to S22340, and if it is not white, the process proceeds to S22350.

The image for determining the second target color is an image 360, and the colors of the coordinates C ₂ (x _C2 , y _C2 ) of the image 360 are (r ₃₆₀ (x _C2 , y _C2 ), g ₃₆₀ (x _C2 , y _C2 ), b ₃₆₀ (x _C2 , y _C2 )). Any of the imaging results of S22330, 22340, or 22350 is entered in the values of r ₃₆₀ , g ₃₆₀ , and b ₃₆₀ .

The second target color determination process (S22330) in the case of black will be described below.
A black (0, 0, 0) solid image is displayed from each of the video projection units 202 and 203, and the state of the projection ranges 212 and 213 reflected on the screen is photographed by the photographing unit 302, and the photographed target color is determined. The image 360 that is an image is stored. Here, considering the conversion 1 for converting the value (r, g, b) of the image photographed by the photographing unit 201 or 202 into a value (R, G, B) proportional to the amount of light, there is a relationship of Equation 21.

ただし、Ｒ、Ｇ、Ｂの値の範囲は数２２とする。

However, the range of values of R, G, and B is represented by Equation 22.

撮影部２０１、２０２が色空間の国際標準規格であるｓＲＧＢに準拠していれば、変換ｌとｌ^−１を数２３のように定義できる。

If the photographing units 201 and 202 conform to sRGB, which is an international standard for color space, the conversions l and l ⁻¹ can be defined as in Expression 23.

数２３の変換を使う代わりに、予め撮影部２０１と２０２をキャリブレーションして変換ｌを求めておいてもよい。
数２４のように、先のＳ１２３３０で定まった、合わせたい色である黒色に対する第一の目標色（ｒ_Ｇ１，ｇ_Ｇ１，ｂ_Ｇ１）を変換ｌで変換し、その値を（Ｒ_Ｇ１，Ｇ_Ｇ１，Ｂ_Ｇ１）とする。

Instead of using the transformation of Equation 23, the imaging units 201 and 202 may be calibrated in advance to obtain the transformation l.
As shown in Equation 24, the first target color (r _G1 , g _G1 , b _G1 ) for black, which is the color to be matched, determined in the previous S12330 is converted by the conversion l, and the value is converted to (R _G1 , G _G1 , B _G1 ).

画像３６０を画素ごとに変換ｌで変換した画像を画像３７０とし、画像３７０の座標Ｃ_２（ｘ_Ｃ２，ｙ_Ｃ２）の値を（Ｒ_３７０（ｘ_Ｃ２，ｙ_Ｃ２），Ｇ_３７０（ｘ_Ｃ２，ｙ_Ｃ２），Ｂ_３７０（ｘ_Ｃ２，ｙ_Ｃ２））とすると、数２５の関係がある。

An image obtained by converting the image 360 by pixel conversion 1 is an image 370, and the values of the coordinates C ₂ (x _C2 , y _C2 ) of the image 370 are (R ₃₇₀ (x _C2 , y _C2 ), G ₃₇₀ (x _C2 , y _C2 ), B ₃₇₀ (x _C2 , y _C2 )), there is a relationship of Equation 25.

ｍ_３５５（ｘ，ｙ）＝１を満たす全ての座標（ｘ，ｙ）の色のうち、緑成分ｇ_３６０（ｘ，ｙ）の値の大きさが上位０．０５％に含まれる色を除いた色全てに対して、数２６を満たす最小のαを求める。

Of the colors of all coordinates (x, y) satisfying m ₃₅₅ (x, y) = 1, except for the color whose magnitude of the value of the green component g ₃₆₀ (x, y) is included in the upper 0.05% For all the colors, the minimum α satisfying Equation 26 is obtained.

上位０．０５％に含まれる色を除いているのは、撮影した画像のノイズや撮影部の素子不足などによる輝点の悪影響を避けるためである。もちろん、０．０５％の値は違う値でもよい。
数２７のように、このときの値（αＲ_Ｇ１，αＧ_Ｇ１，αＢ_Ｇ１）を変換ｌ^−１で変換した色を第二の目標色（ｒ_Ｇ２，ｇ_Ｇ２，ｂ_Ｇ２）とする。

The reason why the colors included in the upper 0.05% are excluded is to avoid adverse effects of bright spots due to noise in the captured image and insufficient elements in the imaging unit. Of course, a value of 0.05% may be different.
As shown in Equation 27, a value obtained by converting the values at this time (αR _G1 , αG _G1 , αB _G1 ) by the conversion l ⁻¹ is set as a second target color (r _G2 , g _G2 , b _G2 ).

以上の手順により、第一の目標色と明るさは異なっても色相を保ったまま第二の目標色を決定することができる。人の目は、色相の変化には敏感でも、明るさの変化には鈍いので、本方法は有効である。

By the above procedure, the second target color can be determined while maintaining the hue even if the brightness is different from the first target color. This method is effective because the human eye is sensitive to changes in hue but is not sensitive to changes in brightness.

Next, the second target color determination process (S22340) in the case of white will be described.
White (255, 255, 255) solid images are respectively displayed from the video projection units 202 and 203, and the projection areas 212 and 213 reflected on the screen are photographed by the photographing unit 302, and the photographed target color is determined. The image 360 that is an image is stored.
As shown in Equation 24, the first target color (r _G1 , g _G1 , b _G1 ) for white that is the color to be matched, determined in the previous S12340, is converted by the conversion l, and the value is converted to (R _G1 , G _G1 , B _G1 ).
Assuming that an image obtained by converting the image 360 by the conversion 1 for each pixel is an image 370, the values of the coordinates C ₂ (x _C2 , y _C2 ) of the image 370 (R ₃₇₀ (x _C2 , y _C2 ), G ₃₇₀ (x _C2 , y _C2 ), B ₃₇₀ (x _C2 , y _C2 )) are as shown in Equation 25.
Of the colors of all coordinates (x, y) satisfying m ₃₅₅ (x, y) = 1, the color whose magnitude of the value of the green component g ₃₆₀ (x, y) is included in the lower 0.5% is excluded. For all the colors, the maximum β satisfying Equation 28 is obtained.

上位０．５％に含まれる色を除いているのは、撮影した画像のノイズや撮影部の素子不足などによる暗点の悪影響を避けるためである。もちろん、０．５％の値は違う値でもよい。
数２８のように、このときの値（βＲ_Ｇ１，βＧ_Ｇ１，βＢ_Ｇ１）を変換ｌ^−１で変換した色を第二の目標色（ｒ_Ｇ２，ｇ_Ｇ２，ｂ_Ｇ２）とする。

The reason why the colors included in the upper 0.5% are excluded is to avoid the adverse effects of dark spots due to noise in the captured image and insufficient elements in the imaging unit. Of course, a value of 0.5% may be different.
As in Equation 28, the value of this time _{(βR G1, βG G1, βB} G1) color converted by converting ^{l -1} a second target color _{(r G2, g G2, b} G2) and.

以上の手順により、第一の目標色と明るさは異なっても色相を保ったまま第二の目標色を決定することができる。

By the above procedure, the second target color can be determined while maintaining the hue even if the brightness is different from the first target color.

Next, the second target color determination process (S2350) in the normal case will be described.
When the first target color for the color to be matched, which is determined in the previous S12350, is (r _G1 , g _G1 , b _G1 ), the second target color (r _G2 , g _G2 , b _G2 ) is expressed by Equation 30. Determined by

次に、図１２を参照して、フィードバックループ処理（Ｓ２２４００）について説明する。
Ｓ２２４１０で、整数ｉを変数とし、ｉの値を１とする。Ｓ２２４２０で、数列ｓをＳ１２４２０と同じく数１１とする。
横ｗ_Ｐ２ピクセル、縦ｈ_Ｐ２ピクセルのサイズのフィードバックループ用投映画像を画像２３２とし、画像２３２の座標（ｘ_Ｐ２，ｙ_Ｐ２）の色を（ｒ_２３２（ｘ_Ｐ２，ｙ_Ｐ２），ｇ_２３２（ｘ_Ｐ２，ｙ_Ｐ２），ｂ_２３２（ｘ_Ｐ２，ｙ_Ｐ２））と定義する。また、横ｗ_Ｐ３ピクセル、縦ｈ_Ｐ３ピクセルのサイズのフィードバックループ用投映画像を画像２３３とし、画像２３３の座標（ｘ_Ｐ３，ｙ_Ｐ３）の色を（ｒ_２３３（ｘ_Ｐ３，ｙ_Ｐ３），ｇ_２３３（ｘ_Ｐ３，ｙ_Ｐ３），ｂ_２３３（ｘ_Ｐ３，ｙ_Ｐ３））と定義する。

Next, the feedback loop process (S22400) will be described with reference to FIG.
In S22410, the integer i is a variable and the value of i is 1. In S22420, the number sequence s is set to Equation 11 as in S12420.
A projected image for a feedback loop having a size of horizontal w _P2 pixels and vertical h _P2 pixels is an image 232, and the color of the coordinates (x _P2 , y _P2 ) of the image 232 is (r ₂₃₂ (x _P2 , y _P2 ), g ₂₃₂ ( x _P2 , y _P2 ), b ₂₃₂ (x _P2 , y _P2 )). Also, a _projected image for feedback loop having a size of horizontal w _P3 pixels and vertical h _P3 pixels is set as an image 233, and the color of the coordinates (x _P3 , y _P3 ) of the image 233 is set to (r ₂₃₃ (x _P3 , y _P3 ), g ₂₃₃ (x _P3 , y _P3 ), b ₂₃₃ (x _P3 , y _P3 )).

In S22430, initial values of the projected images for the feedback loop are set.
The images 232 and 233 are black images, and initial values are set as shown in Equation 31.

Ｓ２２４４０で、画像２３２を投映部２０２から表示し、画像２３３を投映部２０３から表示し、スクリーンに映った投映範囲２１２、２１３の様子を撮影部３０２で撮影して、撮影されたフィードバックループ用画像である画像３８０を保存する。
画像３８０は横ｗ_Ｃ２ピクセル、縦ｈ_Ｃ２ピクセルのサイズであり、画像３８０の座標（ｘ_Ｃ２，ｙ_Ｃ２）の色を（ｒ_３８０（ｘ_Ｃ２，ｙ_Ｃ２），ｇ_３８０（ｘ_Ｃ２，ｙ_Ｃ２），ｂ_３８０（ｘ_Ｃ２，ｙ_Ｃ２））と表す。ｒ_３８０、ｇ_３８０、ｂ_３８０の値に関し、横ｗ_Ｃ２ピクセル、縦ｈ_Ｃ２ピクセルのサイズを超える部分について、数３２のように定義する。

In S22440, the image 232 is displayed from the projection unit 202, the image 233 is displayed from the projection unit 203, and the state of the projection ranges 212 and 213 reflected on the screen is photographed by the photographing unit 302. The image 380 is saved.
The image 380 has a size of horizontal w _C2 pixel and vertical h _C2 pixel, and the color of the coordinates (x _C2 , y _C2 ) of the image 380 is (r ₃₈₀ (x _C2 , y _C2 ), g ₃₈₀ (x _C2 , y _C2). ), B ₃₈₀ (x _C2 , y _C2 )). Regarding the values of r ₃₈₀ , g ₃₈₀ , and b ₃₈₀ , the portion exceeding the size of the horizontal w _C2 pixel and the vertical h _C2 pixel is defined as in Expression 32.

Ｓ２２４５０で、画像３８０に映っている映像投映範囲２１２、２１３の画素の比較・更新をする。
映像投映範囲２１２に関して、以下のように、画素比較・更新をする。
０≦ｘ_２３２＜ｗ_Ｐ２、０≦ｙ_２３２＜ｈ_Ｐ２を満たす整数ｘ_２３２、ｙ_２３２に関し、（ｘ_２３２，ｙ_２３２）に対応する撮影範囲座標を（ｘ_３８０，ｙ_３８０）とすると、図１３のように数３３の関係がある。

In step S22450, the pixels in the video projection ranges 212 and 213 shown in the image 380 are compared and updated.
As for the video projection range 212, pixel comparison / update is performed as follows.
With respect to the integers x ₂₃₂ and y ₂₃₂ that satisfy 0 ≦ x ₂₃₂ <w _P2 and 0 ≦ y ₂₃₂ <h _P2 , assuming that the shooting range coordinates corresponding to (x ₂₃₂ , y ₂₃₂ ) are (x ₃₈₀ , y ₃₈₀ ), FIG. As shown in FIG.

ｘ_３８０やｙ_３８０は常に整数となるとは限らないので、画像３８０の座標（ｘ_３８０，ｙ_３８０）の色（ｒ，ｇ，ｂ）を、バイリニア方式で、図１４のように４近傍の画素を用いて、数３４のように定義する。

Since x ₃₈₀ and y ₃₈₀ are not always integers, the colors (r, g, b) of the coordinates (x ₃₈₀ , y ₃₈₀ ) of the image 380 are represented by four neighboring pixels as shown in FIG. Is defined as shown in Equation 34.

色（ｒ，ｇ，ｂ）と目標色（ｒ_Ｇ２，ｇ_Ｇ２，ｂ_Ｇ２）を比較する。
ｒ＜ｒ_Ｇ２のとき、ｒ_２３２（ｘ_２３２，ｙ_２３２）を変分ｓ（ｉ）だけ増やす。増やした後にｒ_２３２（ｘ_２３２，ｙ_２３２）がｒ_ＭＡＸを超えたときはｒ_ＭＡＸとする。
ｒ＞ｒ_Ｇ２のとき、ｒ_２３２（ｘ_２３２，ｙ_２３２）を変分ｓ（ｉ）だけ減らす。減らした後にｒ_２３２（ｘ_２３２，ｙ_２３２）がｒ_ＭＩＮを下回るときはｒ_ＭＩＮとする。
ｇ＜ｇ_Ｇ２のとき、ｇ_２３２（ｘ_２３２，ｙ_２３２）を変分ｓ（ｉ）だけ増やす。増やした後にｇ_２３２（ｘ_２３２，ｙ_２３２）がｇ_ＭＡＸを超えたときはｇ_ＭＡＸとする。
ｇ＞ｇ_Ｇ２のとき、ｇ_２３２（ｘ_２３２，ｙ_２３２）を変分ｓ（ｉ）だけ減らす。減らした後にｇ_２３２（ｘ_２３２，ｙ_２３２）がｇ_ＭＩＮを下回るときはｇ_ＭＩＮとする。
ｂ＜ｂ_Ｇ２のとき、ｂ_２３２（ｘ_２３２，ｙ_２３２）を変分ｓ（ｉ）だけ増やす。増やした後にｂ_２３２（ｘ_２３２，ｙ_２３２）がｂ_ＭＡＸを超えたときはｂ_ＭＡＸとする。
ｂ＞ｂ_Ｇ２のとき、ｂ_２３２（ｘ_２３２，ｙ_２３２）を変分ｓ（ｉ）だけ減らす。減らした後にｂ_２３２（ｘ_２３２，ｙ_２３２）がｂ_ＭＩＮを下回るときはｂ_ＭＩＮとする。

The color (r, g, b) is compared with the target color (r _G2 , g _G2 , b _G2 ).
When r <r _G2 , r ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) is increased by the variation s (i). When r ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) exceeds r _MAX after the increase, it is set as r _MAX .
When r> r _G2 , r ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) is reduced by the variation s (i). If r ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) falls below r _MIN after the decrease, it is set to r _MIN .
When g <g _G2 , g ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) is increased by the variation s (i). When g ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) exceeds g _MAX after increasing, it is set as g _MAX .
When g> g _G2 , g ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) is reduced by the variation s (i). When g ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) falls below g _MIN after the reduction, it is set as g _MIN .
When b <b _G2 , b ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) is increased by the variation s (i). If b ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) exceeds b _MAX after the increase, b _MAX is set.
When b> b _G2 , b ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) is reduced by the variation s (i). If b ₂₃₂ (x ₂₃₂ , y ₂₃₂ ) falls below b _MIN after the reduction, b _MIN is set.

When the coordinates (x ₂₃₂ , y ₂₃₂ ) are on the portion where the video projection range overlaps, ρ _R s (i) (ρ _R is 0 <ρ _R <1 instead of the variation s (i). (Real number to satisfy) may be used. The value of [rho _R is, for example [rho _{R =} 0.5 or the like value that varies according to the distance from the point where the range began overlap movies PICTURE.

Regarding the video projection range 213, pixel comparison / update is performed as follows in the same manner as the video projection range 212.
_Assuming that the shooting range coordinates corresponding to (x ₂₃₃ , y ₂₃₃ ) satisfying 0 ≦ x ₂₃₃ <w _P2 and 0 ≦ y ₂₃₃ <h _P2 regarding the integers x ₂₃₃ and y ₂₃₃ are (x ₃₈₀ , y ₃₈₀ ), FIG. As shown in FIG.

画像３８０の座標（ｘ_３８０，ｙ_３８０）の色（ｒ，ｇ，ｂ）は、数３４で定まる。
色（ｒ，ｇ，ｂ）と目標色（ｒ_Ｇ２，ｇ_Ｇ２，ｂ_Ｇ２）を比較する。
ｒ＜ｒ_Ｇ２のとき、ｒ_２３３（ｘ_２３３，ｙ_２３３）を変分ｓ（ｉ）だけ増やす。増やした後にｒ_２３３（ｘ_２３３，ｙ_２３３）がｒ_ＭＡＸを超えたときはｒ_ＭＡＸとする。
ｒ＞ｒ_Ｇ２のとき、ｒ_２３３（ｘ_２３３，ｙ_２３３）を変分ｓ（ｉ）だけ減らす。減らした後にｒ_２３３（ｘ_２３３，ｙ_２３３）がｒ_ＭＩＮを下回るときはｒ_ＭＩＮとする。
ｇ＜ｇ_Ｇ２のとき、ｇ_２３３（ｘ_２３３，ｙ_２３３）を変分ｓ（ｉ）だけ増やす。増やした後にｇ_２３３（ｘ_２３３，ｙ_２３３）がｇ_ＭＡＸを超えたときはｇ_ＭＡＸとする。
ｇ＞ｇ_Ｇ２のとき、ｇ_２３３（ｘ_２３３，ｙ_２３３）を変分ｓ（ｉ）だけ減らす。減らした後にｇ_２３３（ｘ_２３３，ｙ_２３３）がｇ_ＭＩＮを下回るときはｇ_ＭＩＮとする。
ｂ＜ｂ_Ｇ２のとき、ｂ_２３３（ｘ_２３３，ｙ_２３３）を変分ｓ（ｉ）だけ増やす。増やした後にｂ_２３３（ｘ_２３３，ｙ_２３３）がｂ_ＭＡＸを超えたときはｂ_ＭＡＸとする。
ｂ＞ｂ_Ｇ２のとき、ｂ_２３３（ｘ_２３３，ｙ_２３３）を変分ｓ（ｉ）だけ減らす。減らした後にｂ_２３３（ｘ_２３３，ｙ_２３３）がｂ_ＭＩＮを下回るときはｂ_ＭＩＮとする。

The color (r, g, b) of the coordinates (x ₃₈₀ , y ₃₈₀ ) of the image 380 is determined by Equation 34.
The color (r, g, b) is compared with the target color (r _G2 , g _G2 , b _G2 ).
When r <r _G2 , r ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) is increased by the variation s (i). When r ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) exceeds r _MAX after the increase, it is set as r _MAX .
When r> r _G2 , r ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) is reduced by the variation s (i). If r ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) falls below r _MIN after the decrease, it is set to r _MIN .
When g <g _G2 , g ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) is increased by the variation s (i). When g ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) exceeds g _MAX after increasing, it is set as g _MAX .
When g> g _G2 , g ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) is reduced by the variation s (i). If g ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) falls below g _MIN after the reduction, it is set as g _MIN .
When b <b _G2 , b ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) is increased by the variation s (i). When b ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) exceeds b _MAX after the increase, b _MAX is set.
When b> b _G2 , b ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) is reduced by the variation s (i). If b ₂₃₃ (x ₂₃₃ , y ₂₃₃ ) falls below b _MIN after the reduction, b _MIN is set.

When the coordinates (x ₂₃₃ , y ₂₃₃ ) are on the portion where the video projection range overlaps, σ _R s (i) (σ _R is 0 <σ _R <1 instead of the variation s (i). (Real number to satisfy) may be used. The value of sigma _R may, for example the use of σ _R = 1-ρ _R.
In S22470, the sizes of i and N are compared. If i <N, the process proceeds to S22480, and if i <N, S22400 is terminated. In S22480, the value of i is incremented by 1, and the process returns to the next S22440 to loop. When returning to S22440, a smoothing filter may be applied to each of the images 232 and 233.

In this manner, by repeating the procedure of S22440 to 22480, the color of the area in which the video projection ranges 212 and 213 are captured in the captured image 380 approaches the target color (r _G2 , g _G2 , b _G2 ). . The images 232 and 233 at the time when S22400 ends are temporary correction data of the colors (r _T , g _T , b _T ) to be matched in the photographing unit 302.
Through the above procedure, provisional correction data corresponding to each photographing location was obtained. Thereby, the image 222 and the image 232, which are temporary correction data of the projection unit 202, which is one feature of the present invention, can be smoothly connected.

Next, the region classification processing (S300) will be described with reference to FIG.
The mask conversion process (S31000) will be described. In this process, as shown in FIG. 16, when the area mapped from the mask 321 with f ⁻¹ _C1P2 is the mask 421, the mask 421 occupies the video projection range 211 that was captured in the shooting range 301 in the video projection range 212. Represents an area.
_Assuming that an area _obtained by mapping the mask 322 with f ⁻¹ _C1P2 is a mask 422, the mask 422 represents an area occupied by the video projection range 212 in the imaging range 301 in the video projection range 212. _Assuming that an area _obtained by mapping the mask 332 with f ⁻¹ _C2P2 is a mask 432, the mask 432 represents an area occupied by the video projection range 212 that was captured in the shooting range 302 in the video projection range 212. _Assuming that an area _obtained by mapping the mask 333 with f ⁻¹ _C2P2 is a mask 433, the mask 433 represents an area occupied by the video projection range 213 in the imaging range 302 in the video projection range 212.
When the masks ₄₂₁ , ₄₂₂ , ₄₃₂ , and ₄₃₃ are expressed by functions m ₄₂₁ , m ₄₂₂ , m ₄₃₂ , and m ₄₃₃ of the coordinates P ₂ (x _P2 , y _P2 ) of the video projection unit 202, respectively, _Expression 36 is obtained.

次に、第一の基準線算出の処理（Ｓ３２０００）について説明する。
第一の基準線を、映像投映範囲２１２のｙ座標ごとに、ｘ座標の値がひとつ決まる関数ｚ_Ｌ（ｙ）で表す。関数ｚ_Ｌ（ｙ）を決める手順を図１７に示す。
Ｓ３２１００で、ｙの値を０とする。Ｓ３２２００で、ｘの値をｗ_Ｐ２−１とする。Ｓ３２３００で、（ｘ，ｙ）がマスク４２２上にあるかどうかを判定し、マスク４２２上にあるときＳ３２４００に進み、マスク４２２上にないときＳ３２５００に進む。

Next, the first reference line calculation process (S32000) will be described.
The first reference line is represented by a function z _L (y) in which one x-coordinate value is determined for each y-coordinate of the video projection range 212. The procedure for determining the function z _L (y) is shown in FIG.
In S32100, the value of y is set to 0. In S32200, the value of x is set to w _P2 −1. In step S32300, it is determined whether (x, y) is on the mask 422. If it is on the mask 422, the process proceeds to step S32400, and if it is not on the mask 422, the process proceeds to step S32500.

When the value of m ₄₂₂ (x, y) is 1, (x, y) is on the mask 422, and when the value of m ₄₂₂ (x, y) is 0, (x, y) is not on the mask 422. And In S32400, the value of z _L (y) is set to x, and the process proceeds to S32800. In S32500, the value of x is decremented by one. In S32600, it is checked whether x ≧ 0. If x ≧ 0, the process returns to S32300, and if x ≧ 0, the process proceeds to S32700.
In S32700, the value of z _L (y) is set to zero. In S32800, the value of y is incremented by one. In S32900, it is checked whether y <h _P2 or not. If y <h _P2 , the flow returns to S32200, and if y <h _P2 , the flow ends S32000.

Next, the second reference line calculation process (S33000) will be described.
The second reference line is represented by a function z _R (y) in which one x-coordinate value is determined for each y-coordinate of the video projection range 212. The procedure for determining the function z _R (y) is shown in FIG.
In S33100, the value of y is set to 0. In S33200, the value of x is set to 0. In S33300, it is determined whether (x, y) is on the mask 432. If it is on the mask 432, the process proceeds to S33400, and if it is not on the mask 432, the process proceeds to S33500.
When the value of m ₄₃₂ (x, y) is 1, (x, y) is on the mask 432, and when the value of m ₄₃₂ (x, y) is 0, (x, y) is not on the mask 432. And

In S33400, the value of z _R (y) is set to x, and the process proceeds to S33800. In S33500, the value of x is incremented by one. In S33600, examined whether x _{<w P2,} return to S33300 is when it is x _{<w P2,} when it is not x _{<w P2} proceeds to S33700. In S33700, the value of z _R (y) is set to w _P2 −1. In S33800, the value of y is incremented by one.
In S33900, it is checked whether y <h _P2 or not. If y <h _P2 , the process returns to S33200, and if y <h _P2 , the process ends S33000.

Next, the center line calculation process (S34000) will be described.
The center line is represented by a function z (y) that determines one x-coordinate value for each y-coordinate of the video projection range 212, and the points (z _L (y), y) and (z _R ( y), defined by the middle point of y).

次に、内挿確定範囲の左端の決定をする処理（Ｓ３５０００）について説明する。
内挿確定範囲の左端を、映像投映範囲２１２のｙ座標ごとに、ｘ座標の値がひとつ決まる関数ｚ_ＭＩＮ（ｙ）で表す。関数ｚ_ＭＩＮ（ｙ）を決める手順を図１９に示す。
Ｓ３５１００で、ｙの値を０とする。ａ_ｆを負でない整数として、強制的に内挿を行う幅を２ａ_ｆピクセルとする。２ａ_ｆピクセルの幅を取ることで、画像２２２と２３２を滑らかにつなぐことができる。
ａ_ｆの値は、例えば数３８とする。

Next, the process (S35000) for determining the left end of the interpolation confirmation range will be described.
The left end of the interpolation fixed range is represented by a function z _MIN (y) that determines one x-coordinate value for each y-coordinate of the video projection range 212. The procedure for determining the function z _MIN (y) is shown in FIG.
In S35100, the value of y is set to 0. As non-negative integer to a _f, to force the width of performing interpolation with 2a _f pixels. By taking a width of 2a _f pixels, the images 222 and 232 can be smoothly connected.
The value of a _f is, for example, Equation 38.

なお、ａ_ｆの値は、ユーザに入力させても構わない。

The value of a _f are, it may be input to the user.

In S35200, the values of x z (y) -a _f. In S35300, it is determined whether (x, y) is on the mask 422. If it is on the mask 422, the process proceeds to S35400, and if it is not on the mask 422, the process proceeds to S35500. In S35400, the value of z _MIN (y) is set to x, and the process proceeds to S35800. In S35500, the value of x is decremented by 1.
In S35600, it is checked whether x ≧ 0. If x ≧ 0, the process returns to S35300, and if x ≧ 0, the process proceeds to S35700. In S35700, the value of z _MIN (y) is set to 0. In S35800, the value of y is incremented by one.
In S35900, it is checked whether y <h _P2 or not. If y <h _P2 , the process returns to S35200, and if y <h _P2 , the process S35000 is terminated.

Next, the process (S36000) for determining the right end of the interpolation confirmation range will be described.
The right end of the interpolation fixed range is represented by a function z _MAX (y) that determines one x-coordinate value for each y-coordinate of the video projection range 212. The procedure for determining the function z _MAX (y) is shown in FIG.
In S36100, the value of y is set to 0. In S36200, the values of _{x z (y) + a f} . In S36300, it is determined whether (x, y) is on the mask 432. If it is on the mask 432, the process proceeds to S36400, and if it is not on the mask 432, the process proceeds to S36500. In S36400, the value of z _MAX (y) is set to x.

In S36500, the value of x is incremented by one. In S36600, examined whether x _{<w P2,} return to S36300 is when it is x _{<w P2,} when it is not x _{<w P2} proceeds to S36700. In S36700, the value of z _MAX (y) is set to w _P2 −1. In S36800, the value of y is incremented by one.
In S36900, it is checked whether y <h _P2 or not. If y <h _P2 , the process returns to S36200, and if y <h _P2 , the process ends S36000.

Next, the area classification process (S37000) will be described.
The policy area is divided into a policy area that is not re-corrected, a policy area that is re-corrected, and a policy area that changes whether to re-correct according to the situation.
Since it is desirable to use the correction data obtained as a result of photographing by the photographing unit as it is for the region where the projection region overlaps on the screen, the region where the projection region overlaps on the screen is set as a region where re-correction is not performed. Expressing region of the re-corrected non policies in mask m _M, so that the number 39.

内挿確定範囲の左端と右端に挟まれた領域をマスクｍ_ＩＮと表すと、数４０のようになる。

When a region between the left and right ends of the interpolation definite ranges representing a mask m _IN, so the number 40.

ここで、再補正する方針の領域をマスクｍ_Ｉで表し、数４１で定義する。

Here, the area of the policy to be recorrected is represented by a mask m _I and is defined by Equation 41.

再補正しない方針の領域と再補正する方針の領域以外の領域を、再補正するかどうかを状況に応じて変える方針の領域とし、マスクｍ_Ｓで表すと、数４２のようになる。

When the area other than the policy area to be re-corrected and the area other than the policy area to be re-corrected are policy areas to change whether to re-correct according to the situation, and expressed by the mask m _S , the following equation 42 is obtained.

このとき、マスクｍ_Ｍ、ｍ_Ｉ、ｍ_Ｓは図２１のようになる。

At this time, the masks m _M , m _I and m _S are as shown in FIG.

When the mask m _S is divided into three or more areas because there is an obstacle between the video projection part and the screen, only the two areas with a large area are left as the mask m _{S. May} be moved to mm.
According to the above procedure, the area m _M and m _{S where} the provisional correction data is selected and the correction data is generated are the areas m _M and m _S in which the shooting range is adjacent and the surrounding video projection range are selected as correction data. _I was divided into any one of at least two areas.

Next, correction data generation processing (S400) will be described with reference to FIG.
The color of the pixel at the coordinates (x, y) of the corrected image 442 is (r ₄₄₂ (x, y), g ₄₄₂ (x, y), b ₄₄₂ (x, y)).
First, the initial value generation process (S41000) of the corrected image 442 will be described with reference to FIG.
In S41100, the value of y is set to 0. In S41200, the value of x is set to 0. In S41300, the coordinates (x, y) is checked whether on the mask _{m M} or on the mask _{m S,} in certain cases S41400, if not, the process proceeds to S41500.

In S41400, it is checked whether or not the value of x is smaller than z (y). If smaller, the process proceeds to S41600, and if not smaller, the process proceeds to S41700. In S41500, the colors (r ₄₄₂ (x, y), g ₄₄₂ (x, y), b ₄₄₂ (x, y)) are set to (0, 0, 0), and the process proceeds to S41800. In S41600, the colors (r ₄₄₂ (x, y), g ₄₄₂ (x, y), b ₄₄₂ (x, y)) are changed to (r ₂₂₂ (x, y), g ₂₂₂ (x, y), b ₂₂₂ ( x, y)), and the process proceeds to S41800. In S41700, the colors (r ₄₄₂ (x, y), g ₄₄₂ (x, y), b ₄₄₂ (x, y)) are changed to (r ₂₃₂ (x, y), g ₂₃₂ (x, y), b ₂₃₂ ( x, y)), and the process proceeds to S41800.

In S41800, x is incremented by one. In S41850, x is examined whether smaller than _{w P2,} if small returns to S41300, if not less, the process proceeds to S41900. In S41900, y is incremented by one. In S41950, y is examined whether smaller than _{h P2,} if small returns to S41200, if not less ends the S41000.

Next, the interpolation process (S42000) will be described with reference to FIG.
First, in S42100, the corrected image 442 is converted into a light quantity proportional value by conversion 1 for each pixel, and the image is set to 452. Assuming that the value of the image 452 is (R ₄₅₂ (x, y), G ₄₅₂ (x, y), B ₄₅₂ (x, y)), Expression 43 is obtained.

Ｓ４２２００でｙの値を０とする。

The value of y is set to 0 in S42200.

The procedure for determining the control point for the Y coordinate y in S42300 will be described with reference to FIG.
Assume that four control points are determined, and the X coordinates of the four points are expressed as c ₀ , c ₁ , c ₂ , and c _{3 in} order from the smallest value. As Integer a _m non-negative, a margin and a _m pixels, for example, a value of a few 44.

マージンの値は、ユーザに入力させてもよい。
Ｓ４２３１０で、ｓ_Ｌ、ｓ_Ｒ、ｃ_１、ｃ_２の初期値を、数４５とする。

The margin value may be input by the user.
In S42310, initial values of s _L , s _R , c ₁ , and c ₂ are set to Expression 45.

Ｓ４２３２０で、ｃ_０の値を、数４６とする。

In S42320, the value of c ₀ is set to Equation 46.

Ｓ４２３３０で、座標（ｃ_０，ｙ）がマスクｍ_Ｓ上にあるかどうか調べ、あるときはＳ４２３５０に進み、ないときは４２３４０に進む。Ｓ４２３４０で、ｓ_Ｌを０．８倍し、Ｓ４２３２０に戻る。なお、０．８倍の代わりに、（０，１）の範囲の実数を用いてもよい。
Ｓ４２３５０で、ｃ_３の値を、数４７とする。

In S42330, it is checked whether or not the coordinates (c ₀ , y) are on the mask m _{S. If} there is, the process proceeds to S42350, and if not, the process proceeds to 42340. In S42340, s _L is multiplied by 0.8, and the flow returns to S42320. A real number in the range of (0, 1) may be used instead of 0.8.
In S42350, the values of _{c 3,} and the number 47.

Ｓ４２３６０で、座標（ｃ_３，ｙ）がマスクｍ_Ｓ上にあるかどうか調べ、あるときはＳ４２３００を終了し、ないときは４２３７０に進む。Ｓ４２３７０で、ｓ_Ｒを０．８倍し、Ｓ４２３５０に戻る。なお、０．８倍の代わりに、（０，１）の範囲の実数を用いてもよい。

In S42360, it is checked whether or not the coordinates (c ₃ , y) are on the mask m _{S. If} there is, the process ends S42300, and if not, the process proceeds to 42370. In S42370, the _{s R} 0.8 multiplied by, returns to S42350. A real number in the range of (0, 1) may be used instead of 0.8.

The procedure for interpolating the Y coordinate y (S42400) will be described with reference to FIG.
In S42410, four points (c ₀ , R ₄₅₂ (c ₀ , y)), (c ₁ , R ₄₅₂ (c ₁ , y)), (c ₂ , R ₄₅₂ (c ₂ , y)), (c ₃ , R ₄₅₂ (c ₃ , y)) is assumed to be (x, S _R (x)) for the cubic spline interpolation of the natural boundary.
4 points (c ₀ , G ₄₅₂ (c ₀ , y)), (c ₁ , G ₄₅₂ (c ₁ , y)), (c ₂ , G ₄₅₂ (c ₂ , y)), (c ₃ , G ₄₅₂ Let (x, S _G (x)) be the cubic spline interpolation of the natural boundary passing through (c ₃ , y)).
4 points (c ₀ , B ₄₅₂ (c ₀ , y)), (c ₁ , B ₄₅₂ (c ₁ , y)), (c ₂ , B ₄₅₂ (c ₂ , y)), (c ₃ , B ₄₅₂ Let (x, S _B (x)) be the cubic spline interpolation of the natural boundary passing through (c ₃ , y)).

Further, linear interpolation passing through two points (c ₁ , R ₄₅₂ (c ₁ , y)) and (c ₂ , R ₄₅₂ (c ₂ , y)) is defined as (x, L _R (x)).
Linear interpolation passing through two points (c ₁ , G ₄₅₂ (c ₁ , y)) and (c ₂ , G ₄₅₂ (c ₂ , y)) is defined as (x, L _G (x)).
A linear interpolation passing through two points (c ₁ , B ₄₅₂ (c ₁ , y)) and (c ₂ , B ₄₅₂ (c ₂ , y)) is defined as (x, L _B (x)).

In S44220, the value of x is set to c ₁ +1. In S42430, it is checked whether s _L ≧ s _MIN and s _R ≧ s _MIN . If both are satisfied, the process proceeds to S42440, and if both are not satisfied, the process proceeds to S42450. However, s _MIN is assumed to be 48.

数４８で、ｓ_ＭＩＮが１０より小さくなったときは、ｓ_ＭＩＮの値を１０とする。
Ｓ４２４４０で、値（Ｒ_４５２（ｘ，ｙ），Ｇ_４５２（ｘ，ｙ），Ｂ_４５２（ｘ，ｙ））の値を、数４９のように更新する。

When s _MIN becomes smaller than 10 in _Equation 48, the value of s _MIN is set to 10.
In S42440, the values (R ₄₅₂ (x, y), G ₄₅₂ (x, y), B ₄₅₂ (x, y)) are updated as shown in Equation 49.

Ｓ４２４５０で、値（Ｒ_４５２（ｘ，ｙ），Ｇ_４５２（ｘ，ｙ），Ｂ_４５２（ｘ，ｙ））の値を、数５０のように更新する。

In S42450, the values (R ₄₅₂ (x, y), G ₄₅₂ (x, y), B ₄₅₂ (x, y)) are updated as in Formula 50.

Ｓ４２４６０で、ｘの値をプラス１する。
Ｓ４２４７０で、ｘがｃ_２未満かどうかを調べ、未満のときはＳ４２４３０に戻り、そうでないときは、Ｓ４２４００を終了する。

In S42460, the value of x is incremented by one.
In S42470, x is checked whether or not less than _{c 2,} when less than return to S42430, and if not, to end the S42400.

As described above, visually interpolating can be performed by interpolating the image 452 using cubic spline interpolation that is continuous up to the second derivative. Also, by using linear interpolation when the distance between the control points c ₀ and c _{1 and} the distance between the control points c ₂ and c ₃ are close, it is possible to avoid unnaturally large fluctuations in the cubic spline interpolation. Can do.

Further, in this embodiment, the colors of the four points (c ₀ , y), (c ₁ , y), (c ₂ , y), and (c ₃ , y) where the provisional correction data exist are proportional to the light amount. Although values are interpolated, values such as geometric information can be interpolated in the same manner.

Next, the corrected image confirmation process (S43000) will be described.
The image 452 is returned to the corrected image 442 by conversion l ⁻¹ for each pixel (x, y) as shown in Equation 51.

補正画像４４２に戻す前に、平滑化フィルタを画像４５２にかけてもよい。
以上の手順により、別途生成する領域の補正データが仮補正データに基づいて生成され、滑らかにつながる補正画像４４２を求めることができる。
画像２２１や２３３の、補正データが求まっていない部分については、線形外挿などの公知技術で外挿して生成すればよいが、映像投映領域が重複する部分のデータの平均値を鏡像コピーした後に本特許の方法で内挿することで生成することもできる。

A smoothing filter may be applied to the image 452 before returning to the corrected image 442.
Through the above procedure, correction data for a region to be separately generated is generated based on the temporary correction data, and a correction image 442 that is smoothly connected can be obtained.
The portions of the images 221 and 233 for which correction data is not obtained may be generated by extrapolation by a known technique such as linear extrapolation, but after the average value of the data of the portion where the video projection areas overlap is copied as a mirror image It can also be generated by interpolation using the method of this patent.

[Example 2]
FIG. 27 shows a video adjustment system according to the second embodiment.
Video projection units 601 to 608 project video onto the screen 500. The photographing units 701, 703, 705, and 707 may be installed in the screen 500, and the photographing units 701, 703, 705, and 707 may be removed after photographing.

FIG. 28 shows the video projection range and the shooting range on the screen 500.
The screen 500 is developed in a plane, and the left and right sides 501 are the same location on the actual screen 500. Video projection ranges 611 to 618 are video projection ranges that the video projection units 601 to 608 project onto the screen 500, respectively.
Video projection unit 601 so that video projection ranges 611 and 612, 612 and 613, 613 and 614, 614 and 615, 615 and 616, 616 and 617, 617 and 618, and 618 and 611 overlap each other on screen 500, respectively. ˜608 and the screen 500 are arranged.

  The imaging unit 701 is installed so that the imaging range 711 covers the video projection range 611, the imaging unit 703 is installed so that the imaging range 713 covers the video projection range 613, and the imaging range 715 covers the video projection range 615. The shooting unit 705 is installed, the shooting unit 707 is installed so that the shooting range 717 covers the video projection range 617, and each measurement is performed, so that the colors for the video projection units 601 to 608 are the same as in the first embodiment. Correction data can be obtained.

The second embodiment shows an example in which geometric correction data is interpolated.
For example, by applying the present embodiment to a multi-projection video display device described in Patent Document 1 (International Publication No. WO99 / 31877), even in a loop structure such as a cylinder as in Embodiment 2, the geometry is smooth. To be connected.

The set of coordinates of the images projected by the image projection units _{601 to 608} are P _{601 to} P ₆₀₈ , respectively, and the set of coordinates of the shooting ranges ₇₀₁ , ₇₀₃ , ₇₀₅ , and ₇₀₇ are C ₇₀₁ , C ₇₀₃ , C ₇₀₅ , and C ₇₀₇ , respectively. To do.
_{F C701P608} a mapping to a set of _{C 701} from the set of P _608, mapping _f set from to the set of _{C 701} in the _{P 601 C701P601,} a mapping from a set of _{P 602} to a collection of _{C 701 f C701P602, P 602} set a mapping to mapping to a set of _{C 703} from the set of _{f C703P602, P 603} to a set of _{C 703} from the set of _{f C703P603, P 604} from the set mapping to a set of _{C 703} of _{f C703P604, P 604} of C a mapping to a set of _{C 705} from the set of _{f C705P606, P 606} a mapping to a set of _{C 705} from the set of _{f C705P605, P 606} a mapping to a set of _{C 705} from the set of _{f C705P604, P 605} from the mapping of the ₇₀₇ to a set from the set of _{f C707P606, P 607} A mapping ₇₀₇ to the set of the mapping from the set of _{f C707P607, P 608} to a set of _{C 707} and _{f C707P608.}

^F _{-1 C701P608} a mapping to a set of _{P 608} from the set of C _701, a mapping from the set of _{C 701} a mapping to a set of _{P 601} from the set of ^f _{-1 C701P601, C 701} to a set of _{P 602} f ^-1 _C701P602, a mapping of the mapping from the set of _{C 703} to a set of _{P 602} from the set of ^f _{-1 C703P602, C 703} to a set of _{P 603} from the set of ^f _{-1 C703P603, C 703} to a set of _{P 604} P a mapping mapping a mapping from a set of ^f _{-1 C703P604, C 705} to a set of _{P 604} from the set of ^f _{-1 C705P604, C 705} to a set of _{P 605} from the set of ^f _{-1 C705P605, C 705} mapping to a set of _{P 606} a mapping ₆₀₆ to the set from the set of ^f _{-1 C705P606, C 707} The mapping from the set of f ^-1 _{C707P606, C 707} a mapping to a set of _{P 607} from the set of ^f _{-1 C707P607, C 707} to a set of _{P 608} and ^f _{-1 C707P608.}

In the photographing of the photographing range 701, f _C701P608 , f _C701P601 , f _C701P602 , f ⁻¹ _C701P608 , f ⁻¹ _C701P601 , and f ⁻¹ _C701P602 can be obtained in the same manner as in the first embodiment.
In the photographing of the photographing range 703, f _C703P602 , f _C703P603 , f _C703P604 , f ⁻¹ _C703P602 , f ⁻¹ _C703P603 , and f ⁻¹ _C703P604 can be obtained in the same manner as in the first embodiment.
In the photographing of the photographing range 705, f _C705P604 , f _C705P605 , f _C705P606 , f ⁻¹ _C705P604 , f ⁻¹ _C705P605 , and f ⁻¹ _C705P606 can be obtained in the same manner as in the first embodiment.
In the shooting of the shooting range 707, f _C707P606 , f _C707P607 , f _C707P608 , f ⁻¹ _C707P606 , f ⁻¹ _C707P607 , and f ⁻¹ _C707P608 can be obtained in the same manner as in the first embodiment.

The mapping from a set of P ₆₀₁ to a set of _{P 602 q P601P602,} a mapping from a set of _{P 602} to a set of _{P 603 q P602P603,} a mapping from a set of _{P 603} to a set of _{P 604 q P603P604, P 604} set a mapping from a set of the set of _{P 605} a mapping from a set of _{q P604P605, P 605} to a set of _{P 606} from the set of _{q P605P606, P 606} a mapping to a set of _{P 607} of _{q P606P607, P 607} P a mapping to a set of _{P 608} from the set of _{q P601P608, P 602} a mapping to a set of _{P 601} from the set of _{q P608P601, P 601} a mapping to a set of _{P 608} from the set of _{q P607P608, P 608} from the mapping of the ₆₀₁ to a set from the set of _{q P602P601, P 603} A mapping ₆₀₂ to the set of the set of _{q P603P602, P 604} a mapping to mapping to a set of _{P 603} from the set of _{q P604P603, P 605} to a set of _{P 604} from the set of _{q P605P604, P 606} of _{P 605} If the mapping to the set _{q P606P605,} a mapping from a set of _{P 607} to a set of _{P 606 q P607P606, q} the mapping to a set of _{P 607} from the set of _{P 608 P608P607,} that can be defined as the number 52 .

映像投映部６０８、６０１、６０２、６０３、６０４の幾何の仮補正データをそれぞれｕ_Ａ８（ｘ，ｙ）、ｕ_Ａ１（ｘ，ｙ）、ｕ_Ａ２（ｘ，ｙ）、ｕ_Ａ３（ｘ，ｙ）、ｕ_Ａ４（ｘ，ｙ）とすると、ｑ_{Ｐ６０８Ｐ６０１}、ｑ_{Ｐ６０１Ｐ６０８}、ｑ_{Ｐ６０１Ｐ６０２}、ｑ_{Ｐ６０２Ｐ６０１}、ｑ_{Ｐ６０２Ｐ６０３}、ｑ_{Ｐ６０３Ｐ６０２}、ｑ_{Ｐ６０３Ｐ６０４}、ｑ_{Ｐ６０４Ｐ６０３}に対して国際公開番号ＷＯ９９／３１８７７の方法を用いることで、ｕ_Ａ８（ｘ，ｙ）、ｕ_Ａ１（ｘ，ｙ）、ｕ_Ａ２（ｘ，ｙ）、ｕ_Ａ３（ｘ，ｙ）、ｕ_Ａ４（ｘ，ｙ）が求まる。

The geometric temporary correction data of the video projection units 608, 601, 602, 603, and 604 are respectively converted to u _A8 (x, y), u _A1 (x, y), u _A2 (x, y), and u _A3 (x, y). ), U _A4 (x, y), q _P608P601 , _qP601P608 , _qP601P602 , _qP602P601 , _qP602P603 , _qP603P602 , _qP603P604 , _qP604P603 are used in the international publication number WO99 / 31877. , U _A8 (x, y), u _A1 (x, y), u _A2 (x, y), u _A3 (x, y), u _A4 (x, y) are obtained.

The geometric temporary correction data of the video projection units 604, 605, 606, 607, and 608 are respectively converted to u _B4 (x, y), u _B5 (x, y), u _B6 (x, y), and u _B7 (x, y). ), U _B8 (x, y), q _P604P605 , _qP605P604 , _qP605P606 , _qP606P605 , _qP606P607 , _qP607P606 , _qP607P608 , _qP608P607 are used in the international publication number WO99 / 31877. , U _B4 (x, y), u _B5 (x, y), u _B6 (x, y), u _B7 (x, y), and u _B8 (x, y) are obtained.
When the geometric correction data of the video projection unit 601 to 608 and _u 1 ~u ₈ respectively, except _{u 4} and _{u 8} is determined as number 53.

ｕ_４は、ｕ_Ａ４とｕ_Ｂ４から、本特許の方法で内挿して定まる。
ｕ_８は、ｕ_Ａ８とｕ_Ｂ８から、本特許の方法で内挿して定まる。
以上のように、実施例２によれば、円筒などループする構造でも幾何がつながるようになる。

u ₄ is determined by interpolating from u _A4 and u _{B4 by} the method of this patent.
u ₈ is determined by interpolating from u _A8 and u _{B8 by} the method of this patent.
As described above, according to the second embodiment, the geometry is connected even with a looping structure such as a cylinder.

[Example 3]
In this example, as shown in FIG. 29, the arrangement of video projection ranges on the screen is arranged vertically. In the first embodiment, correction data can be obtained also in this embodiment by appropriately replacing the X coordinate and the Y coordinate for convenience.
According to this embodiment, even when the video projection ranges are arranged vertically, the video is smoothly connected on the screen.

[Example 4]
This example is an example applied to a multi-projection video display device in which video projection ranges do not overlap each other as shown in FIG.
As in the first embodiment, the target color characteristics of the respective shooting ranges are brought close to each other, and the correction data is interpolated by the method of this patent with respect to the hatched video projection range. Images are smoothly connected on the screen.

It is a figure which shows the system for the image adjustment by a 1st Example. It is a flowchart of the image | video adjustment method in a 1st Example. It is explanatory drawing of the coordinate of an image, and a pixel. FIG. 6 is a diagram illustrating a mask in an imaging range 311. FIG. 6 is a flowchart of brightness / color adjustment processing in an imaging unit. It is a flowchart of a 1st target color determination process. It is a flowchart of a 1st feedback loop process. FIG. 6 is a relationship diagram between video projection ranges 211 and 212 and a shooting range 311. FIG. 6 is a diagram illustrating a mask of an imaging range 312. FIG. 6 is a flowchart of brightness / color adjustment processing in an imaging unit. It is a flowchart of the 2nd target color determination process. It is a flowchart of a 2nd feedback loop process. FIG. 6 is a relationship diagram between video projection ranges 212 and 213 and a shooting range 312. It is a figure of the pixel referred with a bilinear system. It is a flowchart of an area | region classification process. It is a figure which shows the mask of the image | video projection range. It is a flowchart of a 1st reference line calculation process. It is a flowchart of a 2nd reference line calculation process. It is a flowchart of the determination process of the left end of an interpolation confirmation range. It is a flowchart of the determination process of the interpolation end right end. It is a figure which shows the area | region classified into the centerline, the interpolation fixed range, and three types. It is a flowchart of the production | generation process of correction data. It is a flowchart of the initial value generation process of a correction image. It is a flowchart of interpolation. It is a flowchart of determination of a control point. It is a flowchart of an interpolation process. It is a figure which shows the system for the video adjustment by a 2nd Example. It is a figure which shows the relationship between a video projection range and an imaging | photography range. It is a figure which shows the system for the video adjustment by a 3rd Example. It is a figure which shows the system for the image adjustment by the 4th Example.

Explanation of symbols

100: screen,
201-203: Video projection part,
211-213: Image projection range,
221, 222: Projected image for feedback loop of first shooting,
232, 233: Projection image for feedback loop of second shooting,
301, 302: photographing unit,
311 and 312: shooting range,
321, 322, 325 to 327: first shooting mask 330: first shooting target color determination image,
340: Shooting image for feedback loop of first shooting,
352, 353, 355-357: mask for second imaging 360: target color determining image for second imaging,
370: An image obtained by converting 360 into a value proportional to the amount of light 380: a captured image for the feedback loop of the second imaging,
40: Data processing unit 500: Cylindrical screen 501: Cutting lines 601 to 608 of the development view of the cylindrical screen: Video projection unit,
611 to 618: video projection ranges 701, 703, 705, 707: photographing units 711, 713, 715, 717: photographing ranges.

Claims (6)

At least three projection units are used to project an image on a screen, and the displayed test pattern is photographed by the photographing unit to obtain a photographed image. Based on the obtained photographed image, the display characteristics of the video display device are obtained. A video adjustment method for calculating correction data for correction,
Shoot one or more locations where the video projection range is adjacent, and generate temporary correction data corresponding to each location,
Divide the area where the shooting range is adjacent and the surrounding video projection range into at least two types of areas: the area for selecting correction data by selecting any temporary correction data and the area for generating correction data separately And
A video adjustment method, wherein correction data for a separately generated region is generated based on the temporary correction data. 2. The image adjustment method according to claim 1, wherein the correction data includes at least one of color, luminance, and geometric information. 3. The image adjustment method according to claim 1, wherein the area is divided in units of pixels of the projection unit. 4. The video adjustment method according to claim 1, wherein the temporary correction data is generated by feeding back the captured image acquired by the imaging unit. At least three projection units are used to project an image on a screen, and the displayed test pattern is photographed by the photographing unit to obtain a photographed image. Based on the obtained photographed image, the display characteristics of the video display device are obtained. In a projection display device that corrects and projects an image according to the corrected display characteristics,
Means for capturing one or more locations where the video projection ranges are adjacent to each other and generating temporary correction data corresponding to each of the shooting locations; Means for dividing the correction data into at least two types of areas, ie, an area for selecting correction data and an area for generating correction data separately, and correction data for the area generated separately based on the provisional correction data A video display device comprising: means for generating a video image; and a projection unit that projects a video image corrected according to the correction data. A test pattern displayed on a video display device that projects video on at least one projector on the screen is shot by the shooting unit to obtain a shot image, and display characteristics of the video display device are corrected based on the acquired shot image. A video adjustment method for calculating correction data for performing
An image adjustment method characterized in that when two or more shooting ranges are required, a hue of a target color that is a target in different shooting ranges is determined so as not to change from each other.

Images