JP2021064848A - Control device, projection device, projection system, control method of the projection device, program, and storage medium - Google Patents

Abstract

To provide a control device, a projection device, a projection system, a projection method, a program, and a storage medium capable of suppressing an occurrence of uneven luminance in an area where a plurality of projected images are superimposed without installing an imaging apparatus for adjustment when performing multiple projections on a curved screen.SOLUTION: A control device 120 for controlling a projection device 100 that displays an image on a screen in multi-projection has a communication unit 233 that acquires transformation information regarding geometric correction processing for an input image input to the projection device 100, and a gain determining unit 235 that determines a blend curve used for attenuation processing applied by the projection device 100 to the input image based on the transformation information.SELECTED DRAWING: Figure 1

Description

The present invention relates to a control device, a projection device, a control method of the projection device, a program, and a storage medium in a projection system for displaying an image by multi-projection, and the projection system thereof.

There is a projection system that projects one image by superimposing at least a part of a plurality of projected images projected on a screen from a plurality of projection devices. In such a projection system, a dimming process (edge blending process) is applied to an area (superimposed area) of a projected image that overlaps with another projected image.

Patent Document 1 discloses a display device in which a plurality of projectors are continuously arranged to form a large screen. The display device of Patent Document 1 is adjusted so that the brightness between the overlapping start point and the end point of the overlapping portion of each screen detected from the image acquired by the imaging device is linearly reduced on the acquired image.

Further, when a curved screen is used, geometric deformation (geometric correction) processing may be applied to the projected image. The geometric transformation process is called warping or the like. When the geometric transformation process is applied to the projected image, the pixel size of the projected image on the screen is not always uniform.

Japanese Unexamined Patent Publication No. 11-98439

However, in the technique disclosed in Patent Document 1 described above, an imaging device is required for setting the edge blending process. Therefore, the user had to prepare an imaging device for adjustment in order to install the multi-projection system.

The present invention is a control device capable of suppressing the occurrence of uneven brightness in a region where a plurality of projected images are superimposed without providing an imaging device for adjustment when performing multi-projection on a curved screen. , Projection devices, projection systems, projection methods, programs, and storage media.

In order to achieve the above object, one embodiment of the present invention is in a multi-projection system in which a part of an image projected by each projection device is superimposed and a single image is displayed by a plurality of projection means, respectively. A deformation processing means for adjusting the position of the projected image, a blend processing means for performing edge blending based on a blend curve for determining the blend ratio of the superposed region, and a blend curve setting means for setting the shape of the blend curve. Further, the blend size acquisition means for acquiring the pixel size on the projection surface is further provided when the blending process is performed by the blending processing means after the deformation processing by the deformation processing means, and the blend curve setting means has the pixel size. The blend curve to be set in the blend curve setting means is determined according to the pixel size on the projection surface acquired by the acquisition means.

According to the control device, projection device, projection system, projection method, program, and storage medium of the present invention, when performing multi-projection on a curved screen, a plurality of projected images are performed without providing an image pickup device for adjustment. It is possible to suppress the occurrence of uneven brightness in the area where the above are superimposed.

It is a block diagram of a multi-projection system. It is a top view which shows the relationship between the projection apparatus and a screen in a projection system. It is a schematic diagram which shows each functional block of a projection device, an output device, and a control device. It is a block diagram which shows the functional block of the image processing part of a projection apparatus. It is a schematic diagram which shows the dimming processing applied by a dimming processing part. It is a schematic diagram which shows the functional block of the image processing part of an output device. It is a schematic diagram which shows the relationship between the original image and a partial image. It is a schematic diagram which shows the interpolation method of the pixel in the geometric correction processing executed by the geometric transformation processing part. It is a schematic diagram which shows the size of the pixel of the superimposition area of each projection image on a screen. It is a schematic diagram which shows the luminance distribution when the projected image is superposed on the screen. It is a flowchart which shows the position adjustment flow in a projection system. It is a schematic diagram which shows an example of a grid point pattern. It is a schematic diagram which shows the GUI of the position adjustment application displayed on the display part. It is a schematic diagram which shows the moving destination of a grid point. It is a schematic diagram which shows the size and boundary of the pixel group of the central row in the vertical direction of the projected image on a screen. It is a schematic diagram which shows the change of the size of a pixel with respect to the position of a pixel in a pixel array in a horizontal direction. It is a schematic diagram which shows the blend curve determined by the gain determination part.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a configuration diagram of a multi-projection system.

The multi-projection system includes a projection device 100a, a projection device 100b, an output device 110, a control device 120, and a LAN hub 130. The multi-projection system projects one image by superimposing the superposed region of the projected image 101a projected on the screen 140 by the projection device 100a and the superposed region of the projected image 101b projected on the screen 140 by the projection device 100b. It is assumed that the screen 140 is a screen having a concave curved surface toward the projection device 100a and the projection device 100b. In FIG. 1, the region where the overlapping regions overlap is the region indicated by hatching.

The projection devices 100a and 100b are projection devices (projectors) having a common configuration. Hereinafter, when the individual functions and operations of each projection device are described, they are referred to as projection devices 100a and 100b. On the other hand, when the functions and operations common to both the projection devices 100a and 100b are described, the term “projection device 100” is used.

The projection device 100 projects a projected image on the screen 140 based on the image output from the output device 110. Further, the projection device 100 applies the dimming process to the superimposed region of the image based on the control signal input from the control device 120.

The output device 110 is an output device that outputs image data to the projection device 100. The output device 110 generates an image (partial image) obtained by cutting out a portion projected by each projection device 100 from the image (original image) projected by the multi-projection system, and outputs each partial image to each projection device 100. ..

It is assumed that the output device 110 is a media server. The output device 110 may be any device capable of outputting images, such as a personal computer, a camera, a mobile phone, a smartphone, a hard disk recorder, and a game machine. Each of the plurality of partial images generated by the output device 110 has a portion (superimposed region) common to other partial images. The projection device 100 applies the dimming process to the superposed region of the partial image.

Further, the output device 110 applies a geometric transformation process to each partial image in advance so that the original image is projected on the screen 140 with a predetermined aspect ratio.

The control device 120 is a PC that is controlled via the projection devices 100a and 100b, the output device 110, and the LAN hub 130. The control device 120 may be a tablet or the like. The control device 120 determines the gain curve of the dimming process applied to the superposed region by each projection device 100 according to the curvature of the screen 140 and the size of the pixels of the projected image on the screen 140. The control device 120 outputs information indicating the determined gain curve to each projection device 100.

The LAN hub 130 is a hub terminal that controls communication between the projection device 100a, the projection device 100b, the output device 110, and the control device 120. It is assumed that the projection device 100a, the projection device 100b, the output device 110, and the control device 120 are each connected to the LAN hub 130 via a wired LAN. The projection device 100a, the projection device 100b, the output device 110, and the control device 120 may be partially or all connected to the LAN hub 130 via a wireless LAN.

FIG. 2 is a top view showing the relationship between the projection devices 100a and 100b in the projection system and the screen 140. The screen 140 is a curved screen having a constant curvature. Further, it is assumed that the screen 140 is a surface having a convex shape with respect to the projection direction of the projection device 100.

The curved surface formed by the screen 140 is a cylinder, and each projection device 100 is installed so that the optical axis of each projection device 100 passes through the center of the cylinder (circle) formed by the screen 140. In the projected image projected on the screen 140, the brightness of the central portion where the projection distance of the projection device 100 with respect to the screen 140 is long may be lower than that of the peripheral portion. Can be displayed. The projection distance is the distance from the tip of the lens of the projection optical system of the projection device 100 to the intersection when a perpendicular line is drawn with respect to the optical axis from the position of the pixel on the screen 140.

FIG. 3 is a schematic view showing each functional block of the projection devices 100a and 100b, the output device 110, and the control device 120 of the projection system. Since the structure of the projection device 100b is the same as that of the projection device 100a, the description thereof will be omitted.

The projection device 100 includes an input unit 201, a communication unit 202, a control unit 203, an image processing unit 204, a drive unit 205, an operation unit 206, a ROM 207, a light source 208, an illumination optical system 209, a liquid crystal display element 210, and a projection optical system 211. To be equipped.

The input unit 201 is an input unit for inputting an image input from the output device 110. The input unit 201 includes an input terminal and a processing unit that converts the input image into a format that can be processed by each functional block of the projection device 100.

The input unit 201 may be configured to include, for example, a composite, a component, DVI-D, HDMI (registered trademark), or the like. And so on. Furthermore, it is also possible to read an image recorded on a medium such as a USB flash memory or an SD card. The input unit 201 outputs the received image to the image processing unit 204.

The communication unit 202 is a communication interface for transmitting and receiving communication data with an external device. The communication unit 202 can communicate with the output device 110, the control device 120, and the projection device 100b via the LAN hub 130. What communication method can the communication unit 202 support? It may be wired or wireless. The communication unit 202 receives the control signal from the control device 120.

The control unit 203 is a processor that controls the operation of each unit in the projection device 10. The control unit 203 includes, for example, a microcomputer, a CPU, and the like. The control unit 203 receives the control signals input from the communication unit 202 and the operation unit 206, and controls each functional block of the projection device 100.

The image processing unit 204 is a processing circuit that applies image processing such as enlargement / reduction processing, color conversion processing, dynamic range conversion processing, deformation processing, and dimming processing (edge blending processing) to the image acquired from the input unit 201. .. It is assumed that the image processing unit 204 is, for example, a microprocessor (GPU) for image processing.

The image processing unit 204 can execute processing such as black level correction processing for matching the black luminance level of the superposed region and the other region (non-superimposed region) at the time of multi-projection.

The image processing unit 204 can also apply the above-mentioned image processing to the image reproduced by the control unit 203 in addition to the image acquired from the input unit 201.

Based on the instruction from the control unit 203, the image processing unit 204 can superimpose and output an arbitrary OSD or test pattern such as a GUI on the received image. Details will be described later.

The liquid crystal driving unit 205 controls the voltage applied to the light modulation element of the pixels of the liquid crystal display element 210 based on the image output from the image processing unit 204, and controls the light modulation rate of the liquid crystal display element 210.

The operation unit 206 receives the user's instruction and transmits an instruction signal to the control unit 203. It is assumed that the operation unit 206 is a switch, a dial, a cursor button, or the like provided in the projection device 100. Further, the operation unit 206 may have a touch panel. The operation unit 206 may have a reception unit that receives a signal from a remote control device such as a remote controller, and may transmit a control signal to the control unit 203 based on the received signal.

The ROM 207 is a storage medium for storing data such as a program used by the control unit 202.

The light source 208 supplies light to the liquid crystal display element 210, and may be, for example, a halogen lamp, a xenon lamp, a high-pressure mercury lamp, a laser, an LED, a phosphor, or a combination thereof.

The illumination optical system 209 parallelizes the light emitted from the light source 108, outputs the light as a luminous flux, and supplies the light to the liquid crystal display element 210.

The liquid crystal display element 210 is composed of one or a plurality of elements, and an image is formed on the liquid crystal display element 210. A digital micromirror device (DMD) may be used as a display element.

The projection optical system 110 is an optical system that projects the light (projected image) output from the liquid crystal display element 210 onto the screen 140. The projection optical system 110 includes a plurality of lenses and a drive unit for operating the lenses.

The output device 110 includes a control unit 221, a communication unit 222, a storage unit 223, an image processing unit 224, and an output unit 225.

The control unit 221 is a processor that controls the operation of each unit in the output device 110. The control unit 221 includes, for example, a microcomputer, a CPU, and the like. The control unit 221 receives a control signal input from the control device 120 via the communication unit 222, and controls each operation block of the output device 110.

The communication unit 222 is a communication interface for transmitting and receiving communication data with an external device. The communication unit 222 can communicate with the projection device 100a, the projection device 100b, and the control device 120 via the LAN hub 130. The communication method of the communication unit 222 may be either wired or wireless. The communication unit 222 receives the control signal from the control device 120. The output device 110 operates when the control unit 221 executes a process based on the control signal received from the control device 120.

The storage unit 223 is a storage medium that stores the original image that is the source of the projected image projected by the projection device 100. The storage unit 223 stores the video content transmitted via the communication unit 222 as an original image. The storage unit 223 outputs the original image to the image processing unit 224 based on the instruction of the control unit 221. The storage unit 223 is a non-volatile storage medium such as an SSD or an HDD.

The image processing unit 224 is an image processing circuit that generates a partial image to be output to each projection device 100 from the original image read from the storage unit 223. The image processing unit 224 cuts out a region corresponding to the projected image projected by each projection device 100 from the original image input from the storage unit 223 to generate a partial image. The image processing unit 224 can also perform resolution conversion, image division processing, color correction, and geometric deformation processing on the original image or the partial image.

The output unit 505 outputs the partial image generated by the image processing unit 224 to the corresponding projection device 100.

The control device 120 includes a control unit 231, an operation unit 232, a communication unit 233, a projection condition acquisition unit 234, a gain determination unit 235, a ROM 236, a RAM 237, and a display control unit 238.

The control unit 231 is a processor that controls the operation of each unit in the control device 120. The control unit 231 includes, for example, a microcomputer, a CPU, and the like. The control unit 231 executes the program stored in the ROM 236 and controls each operation block of the control device 120. The control unit 231 receives the control signal input from the operation unit 232 and controls each operation block of the control device 120.

The operation unit 232 receives an instruction from the user and transmits a control signal to the control unit 231. The operation unit 232 comprises at least one of, for example, a keyboard, a mouse, a dial, a button, and a touch panel.

The communication unit 233 is a communication interface for transmitting and receiving communication data with an external device. The communication unit 233 can communicate with the output device 110 and the projection devices 100a and 100b via the LAN hub 130. What communication methods can the communication unit 233 support? It may be wired or wireless. A control signal is transmitted from the control device 120 to the output device 110 and the projection devices 100a and 100b via the communication unit 233.

The projection condition acquisition unit 234 acquires the pixel size (pixel size) of the projected image on the screen 140. The projection condition acquisition unit 234 calculates the pixel size based on the deformation information stored in the RAM 237. The projection condition acquisition unit 234 outputs information indicating the pixel size to the gain determination unit 235.

The gain determination unit 235 determines the gain to be used for the dimming process applied to the superimposed region of the image by the image processing unit 204 of each projection device 100 based on the pixel size acquired by the projection condition acquisition unit 234. The gain is represented as a blend curve that gradually changes with respect to the direction in which the projected images are adjacent in the superimposed region. It is assumed that the blend curve is represented by a curve shape in which the gain for each pixel between the start end and the end end is tabulated in the superposed region. The information indicating the blend curve determined by the gain determination unit 235 (blend curve setting information) is stored in the RAM 237.

The ROM 236 is a storage medium that stores a program executed by the control unit 231, various setting data, and a table. The ROM 236 stores the deformation information used by the output device 110 to apply the geometric deformation process to the partial image and the application for determining the blend curve of the projection device 100.

The RAM 237 is a storage medium for storing temporary data and tables. The RAM 237 stores the deformation information used by the output device 110 to apply the geometric deformation process to the partial image and the blend curve of the projection device 100, respectively.

The display control unit 238 is a control unit that displays a Graphical User Interface image (GUI image) used by the user to control the control device 120, the output device 110, and the projection devices 100a and 100b on a display unit such as a liquid crystal display. is there. The display unit may be included in the control device 120, or may be provided outside the control device 120.

FIG. 4 is a block diagram showing a functional block of the image processing unit 204 of the projection device 100.

The image processing unit 204 includes a resolution conversion unit 401, a color correction unit 402, a dimming processing unit 403, a setting unit 404, and an output unit 205.

The resolution conversion unit 401 executes a resolution conversion process for converting the resolution of the input partial image into a preset resolution. The resolution conversion unit 401 outputs a partial image to which the resolution conversion process has been applied to the color correction unit 402. It is assumed that the set resolution is, for example, the resolution of the liquid crystal display element 210. Here, the resolution means the number of pixels of the image.

It is assumed that the resolution of the liquid crystal display element 210 is 1920 × 1080 and the resolution of the input partial image is 1280 × 720. In this case, the resolution conversion unit 401 enlarges the input partial image 1.5 times in both vertical and horizontal directions and converts it into an image having a resolution of 1920 × 1080. When the aspect ratio of the resolution of the liquid crystal display element and the resolution of the input partial image are the same as described above, the image may be simply enlarged.

When the resolution of the input partial image is 1024 × 768, the aspect ratio of the partial image is different from the aspect ratio of the liquid crystal display element. In this case, the resolution conversion unit 401 arranges a partial image enlarged according to the ratio of the vertical or horizontal width in the center, and adds a black image to the remaining pixels in the periphery.

The color correction unit 402 applies color correction processing such as color matrix conversion, chroma processing, gamma processing, sharpness, and color space conversion to the partial image acquired from the resolution conversion unit 401. The color correction unit 402 outputs a partial image to which the color correction processing has been applied to the dimming processing unit 403.

The dimming processing unit 403 applies the dimming processing (gain processing) to the superimposed region of the partial image acquired from the color correction unit 402 based on the blend curve information acquired from the setting unit 404.

FIG. 5 is a schematic view showing the dimming treatment applied by the dimming processing unit 403.

FIG. 5A is a schematic view showing a blend curve set for the projection device 100a. In FIG. 5A, the horizontal axis indicates the horizontal position of the image. It can be said that the projected image 101a of the projection device 100a is a position in a direction adjacent to the projected image 101b. The vertical axis shows the gain applied to the image by the dimming processing unit 403.

As shown in FIG. 5A, in the blend curve set for the projection device 100a, the gain is small from the start end (edge of the image) to the end (left end of the overlay region) of the superimposed region of the image. Become. The blend curve shown in FIG. 5A is set so that the gain changes linearly from the start point to the end of the superposed region. That is, the coordinates of the superposed region to be dimmed are 0 at the start, 1 at the end, and 0.5 at the center of the superposed region.

FIG. 5B is a schematic view showing a blend curve set for the projection device 100b.

As shown in FIG. 5B, in the blend curve set for the projection device 100b, the gain is small from the start end (edge of the image) to the end (right end of the overlay region) of the superimposed region of the image. Become. The blend curve shown in FIG. 5B is set so that the gain changes linearly from the start point to the end of the superposed region. That is, the coordinates of the superposed region to be dimmed are 0 at the start, 1 at the end, and 0.5 at the center of the superposed region.

5 (c) shows the brightness of the image on the screen 140 when each projection device 100 projects the image based on the image to which the blend curve shown in FIGS. 5 (a) and 5 (b) is applied. .. Here, it is assumed that the brightness of each partial image output from the output device 120 is constant.

When a projected image is projected based on a partial image to which a dimming process is applied to a superposed region, the entire projected image including the superposed region is projected on the screen 140 with a constant brightness. The blend curves shown in 5 (a) and (b) are preset. In other words, the blend curve is such that the brightness of the region where the projected image 101a and the projected image 101b overlap (superimposed region) is uniform with the brightness of the portion other than the superposed region of the projected image 101a and the projected image 101b. It is set.

The setting unit 404 sets the blend curve information to be applied by the dimming processing unit 403 based on the blend curve setting information received from the control device 120. The setting unit 404 outputs the blend curve information to the dimming processing unit 403. Specifically, the curve as shown in FIGS. 3A and 3B is calculated and set. This curve is set based on the width of the superposed region 40 and the blend curve setting information received from the control device 120.

The output unit 405 outputs either the partial image acquired from the dimming processing unit 404 or the pattern image determined based on the pattern information received from the control unit 202 to the drive unit 205 as an output image. The pattern image is, for example, a test pattern such as a raster image, a lamp image, or a color bar. The output unit 405 determines whether to output the partial image acquired from the dimming processing unit 404 or the pattern image based on the control signal received from the control unit 202.

FIG. 6 is a schematic view showing a functional block of the image processing unit 224 of the output device 110.

The image processing unit 224 includes a resolution conversion unit 601, a partial image generation unit 602, a color correction unit 603, a pattern generation unit 604, a geometric deformation processing unit 605, and a frame memory 606.

The resolution conversion unit 601 converts the resolution of the original image acquired from the storage unit 223 into an image having a preset resolution.

The partial image generation unit 602 generates a partial image to be output to the projection device 100a and the projection device 100b based on the original image output from the resolution conversion unit 601.

FIG. 7 is a schematic diagram showing the relationship between the original image and the partial image.

Image 701 shows the original image. The partial image generation unit 602 cuts out each partial image from the image 701 so that each partial image includes a superposed region as a common portion. The image 702 is a partial image to be output to the projection device 100a. The image 703 is a partial image to be output to the projection device 100b.

The partial image generation unit 602 determines the cutout position of each partial image based on the positions of the start end and the end of the superimposed region shown in the blend curve setting information acquired from the control unit 222.

The partial image generation unit 602 sets the left end of the image 701 as the start end of the cutout of the image 702, and the position on the right side of the width of the image input to the projection device 100a from the start end of the cutout as the end of the cutout of the image 702.

The partial image generation unit 602 sets the right end of the image 701 as the start end of the cutout of the image 703, and the position on the left side of the width of the image input to the projection device 100b from the start end of the cutout as the end of the cutout of the image 703.

The partial image generation unit 602 cuts out images 702 and 703 from the image 701 based on the start and end of the cutout of the image 702 and the start and end of the cutout of the image 703, and generates a partial image.

For example, the resolution of the image 701 is 3520 × 1200, and the resolution of the input image to the projection device 100a and the projection device 100b is 1920 × 1200. The horizontal coordinates of the start end of the superimposed region of the image 702 are 1919, the horizontal coordinates of the end are 1600, the horizontal coordinates of the start end of the superimposed region of the image 703 are 0, and the horizontal coordinates of the end are 319. In each case, the width of the superimposed region is set to 320 pixels. At this time, the image 702 cuts out the horizontal coordinates 0 to 1919 from the image 701. Image 703 cuts out horizontal coordinates 1600 to 3519 from image 701. When cut out in this way, 1600 to 1919 of the image 701 becomes a superposed region.

Since the operation of the color correction unit 603 is the same as that of the color correction unit 402, the description thereof will be omitted.

The pattern generation unit 604 generates and outputs a test pattern such as a raster image, a lamp image, and a color bar based on the pattern information received from the control unit 221. Further, the pattern generation unit 604 can also generate and output a grid point pattern necessary for the control unit 222 to calculate the deformation information given to the geometric deformation processing unit 605, which will be described later. When the control unit 221 issues an instruction to output the test pattern, the pattern generation unit 604 does not output the image output from the color correction unit 603, but outputs the test pattern. On the other hand, when the control unit 221 does not give an instruction to output the test pattern, the pattern generation unit 604 outputs the image output from the color correction unit 603 as it is.

The geometric deformation processing unit 605 transforms the image input from the color correction unit 603 into an arbitrary shape. The geometric deformation processing unit 605 deforms the image and writes it in the frame memory 606. The geometric deformation processing unit 605 reads the deformed image from the frame memory 606 and outputs it. The geometric deformation processing unit 605 obtains the coordinates of the pixels before and after the deformation based on a predetermined deformation formula, and outputs the image after the deformation. The geometric transformation processing unit 605 uses grid point interpolation as a method of transforming an image into an arbitrary shape.

FIG. 8 is a schematic diagram showing a pixel interpolation method in the geometric correction processing executed by the geometric deformation processing unit 605. FIG. 8A shows an image before deformation. FIG. 8B shows an image after deformation. The grid points P1, P2, P3, and P4 are grid points used for interpolating the pixel D in the deformed image.

The coordinates S of the pre-deformation image of FIG. 8 (a) correspond to the coordinates D of the post-deformation image of FIG. 8 (b) in which the grid point P1 is moved. P1'is an intersection of a straight line passing through the grid points P1 and the coordinates S and a line segment connecting the grid points P2 and P4. The position of the coordinates S of the image before deformation is defined as the position of α: 1-α of the line segment P1-P1'. The position of the point P1'is the position of β: 1-β of the lattice points P2 and P4.

The coordinates D of the deformed image can also be obtained from the coordinates of the grid points P1, P2, P3, and P4 and each ratio. At this time, if the coordinate D of the transformed image is an integer, this may be used as it is as the pixel value of the coordinate S of the untransformed image. However, the transformed coordinates obtained by interpolation are not always integers. In that case, the pixel value of the coordinate D of the deformed image is obtained by interpolating using the pixel values of the peripheral pixels of the coordinate D of the deformed image. As the interpolation method, bilinear, bicubic, or any other interpolation method may be used.

Further, it is assumed that the image deformed by the geometric deformation processing unit 605 is smaller than the image before deformation. When a region not included in the effective image region is generated on the liquid crystal display element 210 due to the deformation, the pixel value is black or the background color set by the user.

The method of obtaining the coordinates D of the movement destination in the deformed image of the coordinates S of the untransformed image is not limited to this.

The geometric deformation processing unit 605 creates the deformed image by obtaining the pixel values for all the coordinates of the deformed image by the above procedure.

The geometric deformation process is used to adjust the shape of the projected image on the screen 140 when the image is projected on the projection surface of a curved surface such as the screen 140. When a rectangular projected image is output on a horizontally curved projection surface such as the screen 140, the center of the projected image is projected into a barrel shape that is enlarged in the vertical direction (vertical direction) from the edges. It ends up. The projected image is projected onto the screen 140 in a rectangular shape by reducing the vertical width (height) of the central portion of the projected image in advance by the geometric transformation process.

In the case of the multi-projection system in the present embodiment shown in FIG. It is possible to display an image.

Next, the problems solved by the present invention will be described in detail.

When a projected image is projected on the screen 140, the distance from the output surface of the projection optical system 211 of each projection device 100 to the screen 140 differs depending on the position of the projected image in the horizontal direction. As shown in FIG. 2, in each projected image, the distance between the central portion and the projected optical system 211 in the horizontal direction is longer than the distance between the end portion of the projected image and the projected optical system 211. Therefore, in each projected image, the size of the pixel at the center in the horizontal direction is larger than the size of the pixel at the end of the projected image.

FIG. 9 is a schematic diagram showing the size of pixels in the superimposed region of each projected image on the screen 140. For the sake of simplicity, FIG. 9 shows only the central pixel sequence in the vertical direction in the superimposed region of each projected image. FIG. 9A shows the pixel size of the superimposed region of the projected image 101a. FIG. 9B shows the pixel size of the superimposed region of the projected image 101b.

Each quadrangle indicates a pixel on the projection surface, and the numerical value in the quadrangle is the coordinate of the pixel when the left end of the projected image is 0. In the projected image 31, the pixel at the beginning of the superimposed region is the 1919th pixel, and the pixel at the end is the 1913th pixel. In the projected image 32, the start end of the superimposed region is the 0th pixel and the end is the 6th pixel. As described above, assuming that the pixel size increases from the center to the edge of the projected image, the pixel size of the projected image 101a increases from the 1919th pixel to the 1913th pixel. Similarly, in the projection device 101b, the pixel size increases from the 0th pixel to the 6th pixel.

As shown in FIG. 5, it is assumed that the blend curve to be set in each projection device 100 is determined so that the brightness of the pixel in the center of the width of the overlapping region is 50%. In the projected image, the central pixel of the superimposed region is the 1916th pixel in the projected image 101a and the third pixel in the projected image 101b.

FIG. 9C is a schematic view showing pixels when the superimposed regions of the projected images 101 are overlapped on the screen 140. In FIG. 9C, the solid rectangle indicates the pixels of the projected image 101a. The broken line rectangle indicates the pixels of the projected image 101b.

As described above, since the pixel sizes differ in the horizontal direction, the central pixel (1916th pixel) of the superposed region of the projected image 101a and the central pixel (third pixel) of the superposed region of the projected image 101b are different. , Do not overlap on screen 140.

FIG. 10 is a schematic diagram showing the brightness distribution in the horizontal direction of the superimposed region when the projected image 101 to which the blend curve shown in FIG. 5 is applied is overlapped on the screen 140 as shown in FIG. 9 (c).

In FIG. 10, the distribution 1001 shown by the alternate long and short dash line indicates the luminance distribution of the superimposed region of the projected image 101a. As shown in FIG. 9A, the pixel size of the superimposed region of the projected image 101a gradually decreases from the central portion (left side in the figure) to the end portion (right side in the figure) of the projected image 101a.

In FIG. 10, the distribution 1002 shown by the dotted line indicates the luminance distribution of the superimposed region of the projected image 101b. As shown in FIG. 9B, the pixel size of the superimposed region of the projected image 101b gradually decreases from the central portion (right side in the figure) to the end portion (left side in the figure) of the projected image 101b.

In FIG. 10, the distribution 1003 shown by the thick line shows the luminance distribution when the superposed region of the projected image 101a and the superposed region of the projected image 101b are superposed. Each of the examples shown in FIG. 10 shows a case where the partial image input to each projection device 100 is a so-called solid image in which the brightness level is uniform over the plane.

The blend curve shown in FIG. 5 is set so that the brightness distribution on the screen 140 becomes uniform by superimposing the overlapping regions of the projected images 101 whose pixel sizes on the screen 140 are even in the horizontal direction. It is a characteristic.

When the pixel size on the screen 140 is not uniform as shown in FIG. 9C, the distribution 1002 of the projected image 101b superimposed on the pixel does not satisfy the interpolation relationship with respect to the distribution 1001 of the superimposed region of the projected image 101a. .. Therefore, the luminance distribution 1003 on the screen 140 becomes non-uniform in the superimposed region and may be recognized as luminance unevenness.

The control device 120 of this embodiment determines the blend curve setting information based on the result of estimating the brightness of the image projected on the screen 140 from the pixel size of the projected image on the screen 140. This makes it possible to apply the dimming process so that the superimposed regions of the projected image 101 on the screen 140 are interpolated according to the pixel size on the screen 140. Therefore, it is possible to suppress the occurrence of luminance unevenness in the image projected on the screen 140.

FIG. 11 is a flowchart showing a position adjustment flow in the projection system.

When the power is turned on to the output device 110, a video signal is output from the storage unit 223, image processing is performed by the image processing unit 224, and an image is output from the output unit 505 to the projection device 10 and the projection device 20. To. Further, when the power is turned on to the projection devices 10 and 20 and the control unit 203 detects that the input image has been input to the input unit 201, the input unit 201 inputs the image to the image processing unit 103. The control unit 102 drives the image processing unit 103 and the liquid crystal drive unit 104 to project an image. After that, the user aligns the main body of the projection device, and the operation unit 206 enables the edge blending process based on an instruction from the user via a GUI (not shown).

When the control unit 231 detects that the position adjustment application execution instruction has been given by the user, the control unit 231 starts processing the flowchart.

In S1101, the control unit 231 controls the projection device 100 and the output device 110 to project the grid point pattern onto at least one of the projection devices 100a and 100b. The control unit 231 outputs a control signal for generating a grid point pattern to at least one of the projection devices 100a and 100b to the control unit 221 of the output device 110 via the communication unit 233. The control unit 221 causes the pattern generation unit 604 to generate a grid point pattern. Further, the control unit 221 causes each projection device 100 to output the pattern image generated by the output unit 225.

FIG. 12 is a schematic diagram showing an example of a grid point pattern. It is assumed that the grid point pattern is composed of grid point groups P00 to P44 arranged in a matrix of 5 points in the horizontal direction and 5 points in the vertical direction. Assuming that the resolution of the liquid crystal display element 210 is 1920x1200, the coordinates of the lattice points (markers) are 0,480,960,1440,1919 in the horizontal direction and 0,300,600,900,1199 in the vertical direction. The coordinates of each grid point are represented by a combination of these.

The user can select the grid points to be adjusted (target grid points) from the projected grid point patterns and set the parameters (deformation information) of the projection image deformation processing by moving the target grid points. it can.

In S1102, the control unit 231 controls the display control unit 238 so that the GUI of the position adjustment application is displayed on the display unit 240. FIG. 13 is a schematic view showing the GUI of the position adjustment application displayed on the display unit 240. The GUI has a selection area 1301 for selecting a grid point to be adjusted by the user, and a movement area 1302 for designating a movement destination of the selected grid point. The GUI also has a button 1303 for outputting an instruction indicating that the designation of the moving position of the grid point is completed.

The display aspect ratio of the moving area 1302 is the same as the coordinate system handled by the geometric deformation processing unit 605, and is the same as the aspect ratio of the liquid crystal display element 210 of the projection device 100. For example, if the resolution of the liquid crystal display element 210 is 1920x1200 and the moving area 1302 is displayed at a resolution of 480x300, the movement area 1302 is reduced to a quarter.

In S1103, the control unit 231 determines whether or not the target grid point is selected by the user's operation on the GUI displayed in S1102. The selection area 1301 is a GUI for selecting a target grid point from a grid point group included in the grid point pattern, and has a radio button corresponding to each grid point. When the user selects one of the radio buttons in the selection area 1301 using the operation unit 232, the corresponding grid point is selected as the target grid point. When the control unit 231 determines that the target grid point has been selected by the user, the control unit 231 shifts to S1104. When the control unit 231 determines that the grid points have not been selected by the user, the control unit 231 repeats the process of S1103.

In S1104, the control unit 231 determines whether or not the movement position of the target grid point is specified by the user's operation on the GUI displayed in S1102. When the control unit 231 detects that the movement position of the grid point has been specified by the user, the control unit 231 shifts to S1105. If the control unit 231 cannot detect that the destination of the grid points has been specified by the user, the control unit 231 repeats the process of S1104.

In S1105, the control unit 231 acquires the moving position (coordinates) of the target grid point specified by the user in S1104. As described above, when the resolution of the moving area 1302 is reduced to 1/4 of the coordinate system of the geometric deformation processing unit 605, the coordinates (50,100) of the moving area 1302 are specified as the moving position. To do. In this case, the moving position of the target lattice point in the geometric deformation processing unit 605 is the coordinates (200, 400).

In S1106, the deformation information to be transmitted to the geometric deformation processing unit 605 of the output device 110 is updated by using the moving position of the grid point acquired in S1105.

In S1107, the control unit 231 determines whether or not the designation of the moving position of the grid point is completed. The instruction indicating that the designation of the moving position of the grid point is completed is input by the user pressing the button 1303 of the GUI. When the control unit 231 determines that the user has instructed that the movement of the grid points has been completed, the control unit 231 shifts the process to S1108. When it is determined that there is no instruction indicating that the movement of the grid points is completed, the process is shifted to S1103.

In S1108, the deformation information updated by the control unit 231 up to S1107 is transmitted to the output device 110 via the communication unit 233. The control unit 221 controls the geometric deformation processing unit 605 so as to apply the geometric deformation processing by using the deformation information received via the communication unit 222. The geometric deformation processing unit 605 applies the geometric deformation to the input image according to the deformation information received from the control unit 221.

FIG. 14 is a schematic diagram showing the destination of the movement of the grid points when the designation of the movement position of the grid points is completed in S1107. FIG. 14A shows the destination of the grid points in the coordinate system of the geometric deformation processing unit 605. FIG. 14B shows the destination of the grid points in the projected image 101a.

The frame 1401 shown by the solid line in FIG. 14A shows the shape of the grid point pattern (shape before deformation) before designating the movement destination of the grid points. The frame 1402 shown by the dotted line shows the shape (deformed shape) of the grid point pattern reflecting the movement of the grid points specified in the processing up to S1107.

The frame 1403 shown by the solid line in FIG. 14 (b) is an image projected on the screen 140 when the projection device 100a projects an image based on the grid point pattern of the shape before deformation shown in FIG. 14 (a). (Lattice point pattern) is shown. The frame 1404 shown by the dotted line in FIG. 14 (b) is an image projected on the screen 140 when the projection device 100a projects an image based on the grid point pattern of the deformed shape shown in FIG. 14 (a). (Lattice point pattern) is shown. The frame 1405 shown by the alternate long and short dash line in FIG. 14B shows the projection position of the projected image 101b when the geometric deformation treatment is similarly applied to the projection device 100b.

In S1109, the control unit 231 instructs the projection condition acquisition unit 234 to estimate the pixel size on the screen 140. In this embodiment, the screen 140 is a cylindrical screen having a constant curvature in the horizontal direction. Further, the superimposed region of each projected image is the right end or the left end in the horizontal direction of the projected image. Therefore, the projection condition acquisition unit 234 estimates the pixel size in the horizontal direction.

FIG. 15 is a schematic view showing the size and boundary of the pixel group of the central row in the vertical direction of the projected image on the screen.

The projection condition acquisition unit 234 calculates _{the boundary coordinates x i} between each pixel by performing tertiary spline interpolation using the moving positions (coordinates) of the grid points P20 to P24 in FIG. 14A. By this cubic spline interpolation, as shown in FIG. 15, the boundary coordinates x _i (i = 0 to 1920) after the movement between all the pixels in the horizontal direction can be obtained.

Since P21 and P23 are moved so as to approach P22, the geometric deformation processing unit 605 performs deformation so that the amount of movement increases as the outer pixel increases. Pixel size Xw _i of each pixel after the movement in the coordinate system of the geometric transformation processing section 605 as shown in FIG. 15 is the distance between the boundary coordinate x _i between the pixels is represented using Equation 1.

Xw _i = x _{i + 1} −x _i ... (Equation 1)
Those obtained by normalizing the maximum value of xw _i as 1 and xw _i '. Figure 16 (a) is a schematic diagram showing changes in Xw i _'(the distance between the boundary coordinates) size of the pixel relative to the position of pixels in the horizontal direction of the pixel rows in the coordinate system of the geometric transformation processing unit 605.

_{The pixel size XS i} on the projection surface is the reciprocal of the pixel size obtained in the coordinate system of the geometric deformation processing unit 605, and is expressed using Equation 2.

XS _i = 1 / Xw _i '・ ・ ・ (Equation 2)
FIG. 16B is a schematic diagram showing a change in _{pixel size (distance between boundary coordinates) XS i} with respect to a pixel position in a horizontal pixel array on the screen 140. In the coordinate system of the geometric deformation processing unit 605, the narrower the grid point spacing, the larger the pixel size on the screen 140.

In S1110, the gain determination unit 235 calculates a blend curve indicating the blend ratio of each pixel based _{on the pixel size XS i on the screen 140 and the blend width obtained in S1112.}

Gain determining unit 235 identifies the center pixel b _c of the overlapping region of the projection image on the screen 140. The gain determination unit 235 obtains a width Bw that is half the width of the superposed region. The width Bw is obtained based on Equation 3. Incidentally, b _s are the starting end pixel coordinates of the superimposed region (farthest end from the center of the image. Blend ratio 0%), b _e the end closer to the center of the end pixel coordinates (image of the superimposed region. Blend 100% ).

Obtain the pixel located closest to the half of the width Bw of the overlapping region as a b _c.

It is assumed that the pixel size distribution XS _i is the distribution shown in FIG. 16 (b), and the overlapping region is between 1630 pixels and 1919 pixels in the horizontal position. The pixel b _c is a pixel at a position that bisects the area of the shaded area in FIG. 16 (b).

Gain determining unit 235 generates a blend curve connecting the end from the beginning of the overlap region so that the blend ratio of 50% in the pixel b _c. For example, the blend ratio of 0% starting _{b s,} blend ratio of 50% _{b c,} the cubic spline interpolation three points of the blend of 100% termination _{b e,} can be calculated blending ratio of the pixels between points, the blend Can generate curves. Each blend curve of the projection device 10 and the projection device 20 is generated. The generated blend curve is stored in the RAM 237 as the blend curve setting information.

FIG. 17 is a schematic diagram showing a blend curve determined by the gain determination unit 235. FIG. 17A is a schematic view showing a blend curve set for the projection device 100a. In FIG. 17A, the horizontal axis indicates the horizontal position of the image. It can be said that the projected image 101a of the projection device 100a is a position in a direction adjacent to the projected image 101b. The vertical axis shows the gain applied to the image by the dimming processing unit 403.

As shown in FIG. 17A, in the blend curve set for the projection device 100a, the gain is small from the start end (the end portion of the image) to the end (the left end portion of the overlay region) of the superimposed region of the image. Become. The blend curve shown by the dotted line is a blend curve assuming the projection plane of the plane shown in FIG. 5 (a). By determining the blend curve based on the distance (pixel size) of each projected pixel on the screen 140, the superimposed region has a shape that is convex downward from the blend curve assuming a flat surface. That is, the gain in the superposed region is determined to be less than or equal to the gain when a plane is assumed.

As shown in FIG. 17B, in the blend curve set for the projection device 100b, the gain is small from the start end (edge of the image) to the end (left end of the overlay region) of the superimposed region of the image. Become. The blend curve shown by the dotted line is a blend curve assuming the projection plane of the plane shown in FIG. 5 (b). By determining the blend curve based on the distance (pixel size) of each projected pixel on the screen 140, the superimposed region has a shape that is convex downward from the blend curve assuming a flat surface. That is, the gain in the superposed region is determined to be less than or equal to the gain when a plane is assumed.

FIG. 17 (c) shows the brightness of the image on the screen 140 when each projection device 100 projects the image based on the image to which the blend curve shown in FIGS. 17 (a) and 17 (b) is applied. .. By determining the blend curve based on the distance (pixel size) of each projected pixel on the screen 140, the occurrence of luminance unevenness in the superimposed region superimposed on the screen 140 is suppressed.

In S1111, the gain determination unit 235 transmits the blend curve setting information generated in S1110 to each projection device 100 via the communication unit 233.

The control unit 203 transmits the blend curve setting information received via the communication unit 202 to the setting unit 404, and the dimming processing unit 403 applies the dimming processing with the blend curve based on the blend curve setting information.

In S1112, the control unit 231 determines whether or not the user has been instructed to complete a series of operations via the operation unit 232. When the control unit 231 determines that there is an instruction to complete the operation, the control unit 231 terminates the position adjustment application and ends the processing of this flowchart. When the control unit 231 determines that there is no instruction to complete the operation and there is an instruction to perform readjustment or the like, the process returns to S1103.

The method of adjusting the blend curve of the superposed region has been described above. In the present embodiment, the blend curve is calculated based on the position where the blend ratio is 50%. However, if the blend curve can be calculated so that the brightness of the superimposed region becomes uniform, it is within the gist of the present invention. It can be changed with.

In this embodiment, the blend curve calculates the blend rate of pixels between representative points (coordinates of start end, end point, and blend rate of 50%) by cubic spline interpolation. For example, the relationship between the overlapping pixels of the projected image 101a and the projected image 101b may be obtained based on the pixel size calculation result, and the blend curve may be determined so that the total blending ratio for the overlapping pixels is 100%.

Further, based on the pixel size acquired by the projection condition acquisition unit 234, the pixel size ratio normalized by the pixel size of the end (terminal) closer to the center of the image among the edges of the superposed region is used as a reference blend curve. The shape of the blend curve may be calculated by multiplying by. At this time, the reference blend curve may be determined so that when the pixel size ratio of all the pixels is 1, the total blending ratio of the overlapping pixels on the projection surface is 100%.

Further, in this embodiment, the method of calculating the pixel size from the deformation information for deformation by the geometric deformation processing unit 605 has been described, but if the information can estimate the pixel size on the projection surface, it can be calculated. Not limited. For example, the pixel size on the screen 140 may be calculated from the radius of curvature on the projection surface and the projection distance.

Further, in this embodiment, the configurations of the projection devices 100a and 100b, the output device 110, and the control device 120 have been described, but the output device 110 and the control device 120 may be configured as one device. The projection condition acquisition unit 234 and the gain determination unit 235 may be built in other devices (for example, projection devices 10 and 20) instead of the control device 120. These configurations can be modified and modified in various ways within the scope of the gist.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and modifications can be made within the scope of the gist thereof.

100 Projection device 110 Output device 120 Control device 140 Screen 201 Input unit 202 Communication unit 203 Control unit 204 Image processing unit 204
208 Light source 209 Illumination optical system 210 Liquid crystal display element 211 Projection optical system 221 Control unit 222 Communication unit 224 Image processing unit 225 Output unit 231 Control unit 233 Communication unit 234 Projection condition acquisition unit 235 Gain determination unit

Claims (14)

A control device that controls a projection device that displays an image on the screen in multi-projection.
An acquisition means for acquiring information on geometric correction processing for an input image input to the projection device, and
Based on the information, a determining means for determining the gain used for the dimming process applied to the input image by the projection device, and
Control device with. The control device according to claim 1, wherein the acquisition means acquires the information from an output device that generates the input image and outputs the input image to the projection device. The determination means acquires the size of a pixel in a superposed region where images projected by a plurality of projection devices overlap among the images projected on the screen based on the information, and the gain is based on the size of the pixel. The control device according to claim 1 or 2, wherein the control device is determined. The acquisition means acquires the shape of the screen and the projection distance, and obtains the image.
Based on the shape of the screen and the projection distance, the determination means acquires the size of pixels in a superposed region in which images projected by a plurality of projection devices overlap among the images projected on the screen, and the pixels. The control device according to claim 1 or 2, wherein the gain is determined based on the size of the above. The determination means is
Based on the pixel size, the pixel size ratio normalized by the pixel size of the edge of the superposed region closer to the center of the image is calculated.
When the blend curve predetermined so that the total blend ratio for the overlapping pixels on the screen is 100% when the pixel size ratio of all the pixels is 1, the reference blend curve is defined as the reference blend curve.
The control device according to claim 3 or 4, wherein the shape of the blend curve is set by multiplying the reference blend curve by the pixel size ratio. It is a projection device that displays an image on the screen by multi-projection based on the input image acquired from the output device.
An acquisition means for acquiring a gain determined from information on the geometric correction process applied to the input image by the output device.
A processing means for applying dimming processing to a region of the input image that overlaps with an image projected by another projection device based on the gain.
Projection device with. It is a projection system that displays an image on the screen with multi-projection.
With multiple projection devices
An output device that outputs an input image to which geometric correction processing is applied to the plurality of projection devices, and an output device.
It has a control device for controlling the plurality of projection devices and the output device.
The projection device applies a dimming process to a region of the input image on which images projected from a plurality of projection devices are superimposed, based on information on the geometric correction process applied to the input image. A projection system characterized by that. It is a control method of a projection device that displays an image on the screen by multi-projection.
An acquisition process for acquiring information on geometric correction processing for an input image input to the projection device, and
A determination step of determining the gain used for the dimming process applied to the input image by the projection device based on the information.
Control method having. The control method according to claim 8, wherein the acquisition step acquires the information from an output device that generates the input image and outputs the input image to the projection device. In the determination step, the pixel size in the superposed region where the images projected by the plurality of projection devices overlap among the images projected on the screen based on the information is acquired, and the gain is obtained based on the pixel size. The control method according to claim 8 or 9, wherein the control method is determined. In the acquisition step, the shape of the screen and the projection distance are acquired.
In the determination step, based on the shape of the screen and the projection distance, the size of the pixel in the superposed region where the images projected by the plurality of projection devices overlap among the images projected on the screen is acquired, and the pixel size is obtained. The control method according to claim 8 or 9, wherein the gain is determined based on the size of the above. The determination step is
Based on the pixel size, the pixel size ratio normalized by the pixel size of the edge of the superposed region closer to the center of the image is calculated.
When the blend curve predetermined so that the total blend ratio for the overlapping pixels on the screen is 100% when the pixel size ratio of all the pixels is 1, the reference blend curve is defined as the reference blend curve.
The control method according to claim 10 or 11, wherein the shape of the blend curve is set by multiplying the reference blend curve by the pixel size ratio. A program for a processor to execute the control method according to any one of claims 8 to 12. A storage medium for storing a program for a processor to execute the control method according to any one of claims 8 to 12.

Images