JP5354252B2 - 3D display manufacturing system, 3D display system, and 3D display system manufacturing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a three-dimensional display manufacturing system that presents an appropriate three-dimensional image without requiring strict alignment between a light controller array and a space light modulator, and to provide a three-dimensional display system and a method of manufacturing the three-dimensional display system. <P>SOLUTION: When the three-dimensional display system 10 is manufactured, the displacement of a lens array 22 relative to the space light modulator 21 is detected by a measurement operation part 102 based on an image photographed by a photographing device 101, and parameters concerning the displacement are stored in a storage section 6. When the stereoscopic display system 10 is operated (used), the space light modulator 21 is controlled by a display control section 5 based on the parameters stored in the storage section 6. This makes it possible to present an appropriate three-dimensional image even if the lens array 22 is displaced relative to the space light modulator 21. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a stereoscopic display manufacturing system including a stereoscopic display for presenting a stereoscopic image, a stereoscopic display system, and a manufacturing method of the stereoscopic display system.

  There is a stereoscopic display as a stereoscopic display for presenting a stereoscopic image. The stereoscopic display projects an object included in the visual volume (a cone having a viewpoint as a vertex and an image presentation surface in a cross section) onto the image presentation surface, and presents an image that can be seen through the image presentation surface to both eyes. Thus, the image is stereoscopically viewed.

In particular, integral photography (IP) is one of the technologies that enables stereoscopic images to be observed with the naked eye. In integral photography, a lens array consisting of a plurality of convex lenses as a light controller is placed on a spatial light modulator such as an LCD (liquid crystal display), and elements for reproducing a stereoscopic image at the focal position under each convex lens. Present an image. The light emitted from each pixel of each element image is emitted so as to be directed only in a specific direction by the effect of the convex lens. Since the lens array disposed on the spatial light modulator functions organically to create a discrete light beam space, a light beam state corresponding to the viewing volume from an arbitrary viewpoint is reproduced.
JP 2006-287592 A

  In the above integral photography, in order for the light emitted from each pixel of each element image to be directed in a specific direction, the position of each pixel needs to correspond exactly to the position of the light controller. There is. When a positional deviation occurs between each pixel of the spatial light modulator and the light beam controller, light emitted from each pixel of the spatial light modulator is directed in a direction different from the assumed direction. In that case, an unintended stereoscopic image is presented.

  When an LCD is used as the spatial light modulator, the length of one side of each pixel is, for example, several hundred μm. That is, it is necessary to align the spatial light modulator and the light beam controller with an accuracy of several hundred μm or less (for example, several tens of μm). This alignment cannot be performed manually, and a dedicated device or the like is required. Thereby, the manufacturing cost increases.

  Patent Document 1 discloses a method for generating a stereoscopic image at high speed based on the positional relationship between the lenticular lens array and the LCD. In this method, it is assumed that detailed specifications (LCD pixel pitch, lens pitch, etc.) at the time of manufacturing the lenticular lens array and the LCD are known. Further, regarding the positional relationship between the lenticular lens array and the LCD, it is necessary for the observer to make fine adjustments by trial and error, which requires a great deal of time and labor.

  An object of the present invention is to provide a stereoscopic display manufacturing system, a stereoscopic display system, and a stereoscopic display system capable of presenting an appropriate stereoscopic image without requiring exact alignment between the light controller array and the spatial light modulator. It is to provide a manufacturing method.

( 1 ) A three-dimensional display manufacturing system according to a first aspect of the present invention is a three-dimensional display constituted by an element display surface, and a correction image control means for controlling the element display surface so as to display a correction image when manufacturing the three-dimensional display , A photographing unit that photographs the element display surface in a state in which the correction image is displayed, a correction data calculating unit that calculates correction data based on a captured image obtained by the photographing unit, and a correction data calculating unit. Storage means for storing the corrected data, and stereoscopic image control means for controlling the element display surface so as to present a stereoscopic image based on the stereoscopic image data for presenting the stereoscopic image. A spatial light modulator composed of a plurality of pixels that generate light and a plurality of light controllers, and controls the direction of light generated from each pixel of the spatial light modulator And a correction image control unit that sequentially turns on one or a plurality of pixel columns along the first direction among the plurality of pixels of the spatial light modulator and is perpendicular to the first direction. A plurality of types of correction images are sequentially displayed on the element display surface by sequentially lighting one or a plurality of pixel rows along the second direction, and the photographing unit displays each correction image on the element display surface. Each of the element display surfaces is photographed, and the correction data calculation means has the most luminous intensity corresponding to each pixel of the spatial light modulator on the positional relationship between the photographing means and the spatial light modulator and on a plurality of types of captured images by the photographing means. Based on the high position, the actual position of each light controller in the light controller array is calculated and the preset position of the light controller array relative to the spatial light modulator and the actual position of the light controller array relative to the spatial light modulator. The position of The stereoscopic image control means creates stereoscopic image data using the correction data stored in the storage means during the stereoscopic image presentation operation, and the stereoscopic image data is calculated based on the created stereoscopic image data. The spatial light modulator is controlled so as to present
In this stereoscopic display manufacturing system, at the time of manufacturing the stereoscopic display, the element display surface is controlled by the correction image control means, and one or a plurality of pixel rows along the first direction among the plurality of pixels of the spatial light modulator. Are lit in order and one or more pixel columns along a second direction perpendicular to the first direction are lit in order. Thereby, a plurality of types of correction images are sequentially displayed on the element display surface. Every time one or more pixel columns along the first direction are lit, the element display surface is photographed by the photographing means. In addition, the element display surface is photographed by the photographing unit every time one or a plurality of pixel columns along the second direction are turned on. Thereby, a plurality of types of captured images are obtained. In these captured images, a distribution of light emitted from the pixels of the spatial light modulator that is turned on appears.
Based on the positional relationship between the imaging means and the spatial light modulator and the position with the highest luminous intensity corresponding to each pixel of the spatial light modulator on a plurality of types of captured images by the imaging means, each light controller of the light controller array Is calculated. By using a plurality of types of captured images obtained as described above, the actual position of each light beam controller in the first direction and the second direction can be efficiently identified. Based on the calculated actual position of each light controller, the amount of deviation between the preset position of the light controller array relative to the spatial light modulator and the actual position of the light controller array relative to the spatial light modulator is corrected. It is calculated as correction data by the data calculation means. The calculated correction data is stored by the storage means.
At the time of a stereoscopic image presentation operation on the stereoscopic display, stereoscopic image data is created by the stereoscopic image control means using the correction data stored in the storage means. Then, the spatial light modulator is controlled by the stereoscopic image control means so as to present a stereoscopic image based on the generated stereoscopic image data.
In this way, stereoscopic image data for presenting a stereoscopic image is created in accordance with the amount of deviation between the preset position of the light beam controller array and the actual position. Therefore, it is not necessary to perform precise alignment between the light controller array and the spatial light modulator, and an appropriate stereoscopic image can be presented even when the position of the light controller array is deviated from a preset position. Can do.

( 2 ) A coordinate system in which the X axis, the Y axis, and the Z axis are determined based on a preset position of the light controller array is defined as a first coordinate system, and the actual position of the light controller array is used as a reference. The coordinate system in which the axes, the Y-axis, and the Z-axis are defined is the second coordinate system, and the first coordinates in the X-axis direction, the Y-axis direction, the Z-axis direction, and the rotation direction around the Z-axis of the first coordinate system When the variables indicating the amounts of deviation between the system and the second coordinate system are the first, second, third, and fourth parameters, the positions of the respective light controllers in the light controller array are the first and second. The correction data calculation means is based on the positional relationship between the imaging means and the spatial light modulator, and on a plurality of types of captured images obtained by the imaging means, using the third and fourth parameters. The light based on the position with the highest luminous intensity corresponding to each pixel of the spatial light modulator The coordinates in the first coordinate system of each light controller in the controller array are calculated as actual coordinates, and the first, second, third, and fourth coordinates are calculated based on the relationship between the actual coordinates and the theoretical coordinates of each light controller. The parameter value may be calculated as correction data.

In this case, the values of the first, second, third, and fourth parameters indicating the amount of deviation between the first coordinate system and the second coordinate system are calculated based on the relationship between the theoretical coordinates and the actually measured coordinates. Is done. In the first coordinate system, the X axis, the Y axis, and the Z axis are determined based on a preset position of the light beam controller array. The second coordinate system, X-axis the actual position of the optical plate array standards, Y-axis and Z-axis is defined.

  Thus, by creating stereoscopic image data using the calculated values of the first, second, third, and fourth parameters as correction data, it is possible to reliably present an appropriate stereoscopic image.

( 3 ) In the method for manufacturing a stereoscopic display system according to the second invention, the stereoscopic display system includes a stereoscopic display configured by an element display surface, storage means for storing correction data, and a stereoscopic display for presenting a stereoscopic image. Stereoscopic image control means for controlling the element display surface to present a stereoscopic image based on the image data, the element display surface comprising a spatial light modulator composed of a plurality of pixels that generate light, and a plurality of And a light beam controller array for controlling the direction of light generated from each pixel of the spatial light modulator. The stereoscopic image control means is stored in the storage means during the stereoscopic image presentation operation. Stereo image data is created using the corrected data, and the spatial light modulator is controlled to present a stereo image based on the created stereo image data. Sequentially lighting one or more pixel columns along the first direction among the plurality of pixels of the container, and sequentially lighting one or more pixel columns along the second direction perpendicular to the first direction. A step of sequentially displaying a plurality of types of correction images on the element display surface, and each time the correction images are displayed on the element display surface, a plurality of types of captured images are obtained by photographing the element display surface with the photographing means. Based on the process, the positional relationship between the imaging means and the spatial light modulator, and the position with the highest luminous intensity corresponding to each pixel of the spatial light modulator on a plurality of types of captured images by the imaging means, The actual position of the light controller is calculated, and the deviation amount between the preset position of the light controller array with respect to the spatial light modulator and the actual position of the light controller array with respect to the spatial light modulator is calculated as correction data. And a step of storing the correction data in the storage means.
In this manufacturing method, one or more pixel columns along the first direction among the plurality of pixels of the spatial light modulator are sequentially turned on and 1 or 2 along the second direction perpendicular to the first direction. A plurality of types of correction images are sequentially displayed on the element display surface by sequentially lighting the plurality of pixel columns. Every time one or more pixel columns along the first direction are lit, the element display surface is photographed by the photographing means. In addition, the element display surface is photographed by the photographing unit every time one or a plurality of pixel columns along the second direction are turned on. Thereby, a plurality of types of captured images are obtained. In these captured images, a distribution of light emitted from the pixels of the spatial light modulator that is turned on appears.
Based on the positional relationship between the imaging means and the spatial light modulator and the position with the highest luminous intensity corresponding to each pixel of the spatial light modulator on a plurality of types of captured images by the imaging means, each light controller of the light controller array Is calculated. By using a plurality of types of captured images obtained as described above, the actual position of each light beam controller in the first direction and the second direction can be efficiently identified. Based on the calculated actual position of each light controller, the amount of deviation between the preset position of the light controller array relative to the spatial light modulator and the actual position of the light controller array relative to the spatial light modulator is corrected. Calculated as data. The calculated correction is stored by the storage means.
In this case, at the time of the stereoscopic image presentation operation by the stereoscopic display, the stereoscopic image control means creates the stereoscopic image data using the correction data stored in the storage means, and presents the stereoscopic image based on the created stereoscopic image data. It is possible to control the spatial light modulator.
Thereby, it is possible to create stereoscopic image data for presenting a stereoscopic image in accordance with the amount of deviation between the preset position of the light beam controller array and the actual position. Therefore, it is not necessary to perform precise alignment between the light controller array and the spatial light modulator, and even when the position of the light controller array is deviated from a preset position, an appropriate three-dimensional image can be displayed on the three-dimensional display. It becomes possible to present.

  According to the present invention, it is not necessary to perform precise alignment between the light controller array and the spatial light modulator, and even when the position of the light controller array is deviated from a preset position, an appropriate stereoscopic image can be obtained. Can be presented.

(1) Configuration of 3D Display FIG. 1 is a schematic external view showing a 3D display in a 3D display system according to an embodiment of the present invention.

  As shown in FIG. 1, the stereoscopic display 1 is configured in a box shape by combining a plurality of rectangular element display surfaces 2. In the present embodiment, the three-dimensional display 1 is configured in a cube by six square element display surfaces 2. A stereoscopic image 3 is presented in a spherical area (hereinafter referred to as a virtual sphere) S of a virtual space surrounded by the element display surface 2.

  FIG. 2 is a schematic plan view showing the element display surface 2 constituting the stereoscopic display 1. FIG. 3 is a schematic cross-sectional view showing the element display surface 2 constituting the stereoscopic display 1.

  As shown in FIGS. 2 and 3, the element display surface 2 is configured by a laminated structure of a planar spatial light modulator 21 and a planar lens array 22.

  The spatial light modulator 21 includes a matrix display element that can present colors in a matrix. The spatial light modulator 21 has a plurality of pixels arranged in a matrix. As the spatial light modulator 21, for example, a liquid crystal display (LCD), a plasma display, an organic electroluminescence (EL) display, an inorganic EL display, a light emitting diode (LED) array, a field emission display (FED), or a photograph can be used. .

  The lens array 22 includes a plurality of convex lenses (hereinafter simply referred to as “lenses”) 220 capable of controlling the direction of light, and has a function of reproducing the state of light from the spatial light modulator 21 in various directions. The plurality of lenses 220 of the lens array 22 are arranged in a matrix.

  The direction of the light beam from the plurality of pixels of the spatial light modulator 21 is controlled by each lens 220 of the lens array 22. Each lens 220 is assigned a pixel group. Each lens 220 is arranged so that the direction of light rays from only the assigned pixel group can be controlled.

  The number of rays that can be controlled by the lens array 22 and the direction of the rays depends on the optical design of the lens 220, such as the number of pixels assigned to each lens 220, the distance between the lens 220 and the pixel, and the focal length of the lens 220. It depends on etc.

  As shown in FIG. 3, the lenses 220 of the lens array 22 control the light traveling in various directions from the pixels Pm1, Pm2, Pm3, and Pm4 of the spatial light modulator 21 in directions indicated by dotted lines.

  As the light beam controller, a pinhole array in which a plurality of pinholes are regularly formed, a diffraction grating array, or the like may be used instead of the lens array 22. In addition, a lenticular lens, a parallax barrier, or the like can be used if the application is limited.

  Here, common points of the light control function by the lens 220 and the light control function by the pinhole will be described with reference to FIG. 4A shows a light control function by the lens 220, and FIG. 4B shows a light control function by a pinhole.

  As shown in FIG. 4A, the distance between the lens 220 and the spatial light modulator 21 is adjusted to be equal to the focal length f of the lens 220. In this case, a plurality of lights emitted from the pixel Pm1 are controlled in a direction parallel to a straight line connecting the pixel Pm1 and the principal point (optical center) 220a of the lens 220 by passing through the lens 220. A plurality of lights emitted from the pixel Pm 2 are controlled in a direction parallel to a straight line connecting the pixel Pm 2 and the principal point 220 a of the lens 220 by passing through the lens 220.

  As shown in FIG. 4B, in the case of using a pinhole array in which a pinhole PH is formed instead of the lens array 22, only light passing through the pinhole PH among a plurality of lights emitted from the pixel Pm1. Is injected outside. In addition, among the plurality of lights emitted from the pixel Pm2, only the light passing through the pinhole PH is emitted to the outside. Therefore, if the position of the pinhole PH is the same as the position of the principal point 220a of the lens 220 in FIG. 4A, the light emitted from each pixel goes in the same direction as when the lens 220 is used.

  However, the amount of light that passes through the pinhole PH and goes to the outside is smaller than the amount of light that passes through the lens 220 and goes to the outside. Therefore, it is preferable to use the lens array 22 in order to present a brighter stereoscopic image.

  In FIG. 4, the light control function by the lens 220 and the pinhole PH has been conceptually described. Note that actual light control is slightly different from the ideal light control as described above.

(2) Method for Creating Stereo Image Data Next, a method for creating stereo image data for presenting the stereo image 3 on the stereo display 1 will be described. FIG. 5 is a diagram for explaining a method of creating stereoscopic image data for presenting the stereoscopic image 3.

  A solid shape 30 to be visualized is defined in a virtual sphere inside the element display surface 2. This three-dimensional shape 30 is represented by three-dimensional shape data. A light ray traveling in an arbitrary direction at an arbitrary point on the surface of the three-dimensional shape 30 is considered. The positional relationship between the element display surface 2 and the three-dimensional shape 30 is uniquely determined by data definition.

  The position (coordinate) of the intersection when each light ray intersects the element display surface 2, the angle between the light ray and the element display surface 2, and the color are obtained. Further, it is determined at which position on which element display surface 2 each light ray intersects. Finally, stereoscopic image data for displaying the color of the intersection point on each pixel of the spatial light modulator 21 is created so that the light beam traveling in the direction of the angle is reproduced by the lens array 22.

  Ideally, the position, angle, and color of the above intersection are obtained for light rays traveling in all directions from the entire surface of the three-dimensional shape 30, and three-dimensional image data representing an image to be displayed on each element display surface 2 is created. Actually, the virtual sphere in the space surrounded by the element display surface 1 is discretized into a plurality of voxels, and the direction of light rays emitted from each voxel is discretized. In the present embodiment, the direction of light emitted from each voxel is controlled by the lens array 22 so as to be discretized.

  If one voxel of the phantom sphere inside the element display surface 2 is a part of the three-dimensional shape 30, a light ray directed from the voxel in a discretized direction is obtained. In this way, the intersections between the light beams traveling in a plurality of directions discretized from each of the plurality of voxels on the surface of the three-dimensional shape 30 and the element display surface 2 are obtained, and the angle between the light beam and the element display surface 2; Then, voxel colors are obtained, and stereoscopic image data is created based on the intersections, angles, and colors.

  For example, as shown in FIG. 5, a light beam emitted from one voxel b <b> 1 on the surface of the three-dimensional shape 30 is controlled by a lens 220 of the lens array 22 so as to be directed in the direction of a dotted arrow. Stereoscopic image data is created so that the color of the voxel b1 is displayed on the pixel at the intersection of these rays and the element display surface 2.

  In addition, light rays emitted from other voxels b2 on the surface of the three-dimensional shape 30 are controlled by the lenses 220 of the lens array 22 so as to be directed in the direction of solid arrows. Stereoscopic image data is created so that the color of the voxel b2 is displayed at the pixel at the intersection of these rays and the element display surface 2.

  The above description has been made for ease of understanding, but in practice, the reproducible rays are limited by the relationship between each lens 220 of the lens array 22 and the number of pixels of the spatial light modulator 21. The reverse algorithm to that described above is used. That is, the light ray reproducible by each lens 220 of the lens array 22 is traced back through the pixels of the spatial light modulator 21 to obtain the color of the voxel at the intersection with the three-dimensional shape 30 to be presented. Determine the color to be displayed.

  When one light beam reproducible by the lens 220 of the lens array 22 intersects a plurality of voxels of the three-dimensional shape 30, the color of the voxel closer to the element display surface 2 is determined as the color to be displayed on the pixel. . This is because a point located in the back as viewed from the observer is hidden by a point located in front. For example, as shown in FIG. 5, when one light beam reproducible by the lens 220 of the lens array 22 intersects with a plurality of voxels b2 and b3 of the three-dimensional shape 30, the voxel b3 closer to the element display surface 2 is obtained. Is determined as the color to be displayed on the pixel.

  In this way, stereoscopic image data for displaying the color of each voxel on the surface of the stereoscopic shape 30 on a plurality of pixels on the element display surface 1 is created. By displaying an image on a plurality of pixels of the spatial light modulator 21 based on the stereoscopic image data, as a result, light rays from the stereoscopic shape 30 are reproduced.

  FIG. 6 is a diagram for explaining an image displayed on the spatial light modulator 21 of the stereoscopic display 1 based on the stereoscopic image data.

  For example, an image when the stereoscopic shape is viewed from the observation point V1 is displayed on the pixel “A” of the spatial light modulator 21. In addition, an image when the stereoscopic shape is viewed from the observation point V2 is displayed on the pixel “B” of the spatial light modulator 21. Thereby, the observer can see the three-dimensional shape from different angles when the eye is moved from the observation point V1 to the observation point V2.

(3) Stereoscopic display system and stereoscopic display manufacturing system The configurations of the stereoscopic display system and the stereoscopic display manufacturing system will be described.

  FIG. 7 is a block diagram illustrating a configuration of a stereoscopic display system and a stereoscopic display manufacturing system. As shown in FIG. 7, the stereoscopic display system 10 includes a stereoscopic display 1, a display control unit 5, and a storage unit 6. The stereoscopic display manufacturing system includes a stereoscopic display system 10, an imaging device 101, and a measurement calculation unit 102. The imaging device 101 and the measurement calculation unit 102 are used when the stereoscopic display system 10 is manufactured, and are not necessary when the stereoscopic display system 10 is used.

  The photographing apparatus 101 includes a CMOS (complementary metal oxide semiconductor) digital camera, a CCD camera, a film camera, a surface luminance meter, a sensor, or the like. The photographing apparatus 101 photographs the element display surface 2 of the stereoscopic display 1. Hereinafter, an image photographed by the photographing apparatus 101 is referred to as a photographed image.

  The storage unit 6 includes a data storage medium such as a hard disk or a memory card, and stores stereoscopic shape data representing the shape of a stereoscopic image to be presented in the virtual sphere, basic attributes of the stereoscopic display 1, basic attributes of the photographing apparatus 101, and the like. To do. Here, the basic attributes of the stereoscopic display 1 are the size, interval and arrangement of the pixels of the spatial light modulator 21, the diameter, interval, arrangement, field angle and focal length of the lens 220 of the lens array 22, and the lens array 22. The ideal positional relationship with the spatial light modulator 21 is included. Further, the basic attributes of the photographing apparatus 101 include the resolution, field angle, focal length, position, posture, and the like of the photographing apparatus 101.

  The measurement calculation unit 102 and the display control unit 5 each include a CPU (Central Processing Unit) and a storage device such as a semiconductor memory. The measurement calculation unit 102 performs a calculation process described later based on the captured image of the imaging apparatus 101 and various information stored in the storage unit 6. The display control unit 5 controls the spatial light modulator 21 based on the calculation result of the measurement calculation unit 102 and various information stored in the storage unit 6. Details of operations of the measurement calculation unit 102 and the display control unit 5 will be described later.

  The display control unit 5 and the storage unit 6 of the stereoscopic display system 10 may be provided in the stereoscopic display 1 or may be provided outside the stereoscopic display 1. The measurement calculation unit 102 can be configured by a personal computer, for example. Moreover, the display control part 5 and the memory | storage part 6 can be comprised by a personal computer, for example.

  In the present embodiment, when the stereoscopic display system 10 is manufactured, the positional deviation of the lens array 22 with respect to the spatial light modulator 21 is detected based on the photographed image of the photographing apparatus 101, and parameters described later regarding the positional deviation are stored in the storage unit 6. Remembered. Then, the spatial light modulator 21 is controlled by the display control unit 5 based on the parameters stored in the storage unit 6 when the stereoscopic display system 10 is operated (used). Thereby, even when the lens array 22 is displaced with respect to the spatial light modulator 21, an appropriate stereoscopic image can be presented. The details will be described below.

(4) Imaging of element display surface by imaging device (4-1) Relationship between observation point and luminous intensity The relationship between the observation point and luminous intensity will be described. FIG. 8 is a schematic diagram for explaining the relationship between the observation point and the luminous intensity. Here, the luminous intensity is the intensity of light obtained at each observation point.

  FIG. 8A shows the positional relationship between the pixels Pm1 to Pm4, the lens 220, and the observation points V11 to V17 of the spatial light modulator 21. FIG. 8B shows the luminous intensity distribution obtained from the light emitted from the pixels Pm1 and Pm3. In FIG. 8B, the luminous intensity obtained from the light emitted from the pixel Pm1 is indicated by a dotted line, and the luminous intensity obtained from the light emitted from the pixel Pm3 is indicated by a one-dot chain line.

  As shown in FIGS. 8A and 8B, the luminous intensity obtained from the light emitted from the pixel Pm1 is highest at the observation point V16 on the straight line connecting the pixel Pm1 and the principal point 220a of the lens 220. Become. Similarly, the luminous intensity obtained from the light emitted from the pixel Pm3 is highest at the observation point V12 on the straight line connecting the pixel Pm3 and the principal point 220a of the lens 220.

  Thus, the luminous intensity obtained from the light emitted from each pixel of the spatial light modulator 21 varies depending on the observation point, and becomes the largest on a straight line connecting each pixel and the main point 220a of the lens 220.

(4-2) Positional Relationship Between Imaging Device and Lens FIG. 9 is a schematic diagram showing a positional relationship among the imaging device 101, the lens array 22, and the spatial light modulator 21. In FIG. 9, the field of view of the photographing apparatus 101 is indicated by a virtual plane VP. The virtual plane VP corresponds to a photographed image of the photographing apparatus 101. Light traveling through the virtual plane VP toward the photographing center point Pc of the photographing apparatus 101 appears on the photographed image of the photographing apparatus 101.

  The photographing center point Pc of the photographing apparatus 101 is a central point that receives light from the outside. For example, when a camera is used as the photographing apparatus 101, the optical center of the camera lens becomes the photographing center point Pc.

  In FIG. 9, it is assumed that when only one pixel Pm11 of the spatial light modulator 21 is turned on, a light spot having a high luminous intensity appears in the captured image of the imaging apparatus 101. In this case, it is considered that the photographing center point Pc of the photographing apparatus 101, the main point 220a of the lens 220 corresponding to the pixel Pm11, and the pixel Pm11 of the spatial light modulator 21 are on a common straight line. The light spot on the photographed image of the photographing apparatus 101 corresponds to the intersection Pi between the straight line connecting the pixel Pm11 and the photographing center point Pc and the virtual plane VP. Hereinafter, a point on the virtual plane VP corresponding to a light spot on the photographed image of the photographing apparatus 101 is referred to as a virtual light spot.

  Thereby, the position of the photographing center point Pc of the photographing apparatus 101 and at least one of the position of the light spot (virtual light spot Pi on the virtual plane VP) and the position of the pixel Pm11 on the photographed image of the photographing apparatus 101 are obtained. For example, the actual position of the lens 220 corresponding to the pixel Pm11 can be obtained.

(4-3) Luminous Intensity of Light Spot on Photographed Image FIG. 10 is a diagram for explaining the distribution of luminous intensity on the photographed image of the photographing apparatus 101. FIG. 10A shows the positional relationship between the imaging device 101, the lens 220, and the spatial light modulator 21. FIG. 10B is a diagram illustrating a light intensity distribution that appears on a captured image of the image capturing apparatus 101 when each pixel of the spatial light modulator 21 is turned on. In FIG. 10B, the vertical axis indicates the luminous intensity, and the horizontal axis indicates the coordinates on the captured image.

  In FIG. 10A, the pixels Pm11 and Pm21 are on a straight line connecting the photographing center point Pc of the photographing apparatus 101 and the principal point 220a of the lens 220, and the pixels Pm9, Pm10, Pm12, Pm13, Pm19, Pm20, and Pm22. , Pm23 are located at positions deviated from a straight line connecting the photographing center point Pc of the photographing apparatus 101 and the principal point 220a of the lens 220.

  In this case, as shown in FIG. 10B, the light intensity obtained when the pixel Pm11 is turned on is higher than the light intensity obtained when the pixels Pm9, Pm10, Pm12, and Pm13 are turned on. Accordingly, one pixel Pm11 that can obtain the highest luminous intensity among the plurality of pixels corresponding to one lens 220 is specified.

  Similarly, the light intensity obtained when the pixel Pm21 is turned on is higher than the light intensity obtained when the pixels Pm19, Pm20, Pm22, and Pm23 are turned on. Thereby, one pixel Pm21 from which the highest luminous intensity is obtained among the plurality of pixels corresponding to one lens 220 is specified.

  Furthermore, by lighting all the pixels one by one in order and measuring the light intensity, it is possible to specify one pixel at a time for obtaining the highest light intensity for all the lenses 220. Thereby, the actual positions of all the lenses 220 can be obtained.

(5) Coordinate system (5-1) Definition of each coordinate system Here, an imaging device coordinate system, a captured image coordinate system, an LCD coordinate system, a lens coordinate system, and a world coordinate system are defined. FIG. 11 is a schematic diagram for explaining the relationship among the imaging device coordinate system, the captured image coordinate system, the LCD coordinate system, the lens coordinate system, and the world coordinate system.

  The photographing apparatus coordinate system Cc is determined based on the photographing apparatus 101. In FIG. 11, the origin Oc of the photographing apparatus coordinate system Cc is determined at the photographing center point Pc of the photographing apparatus 101. Further, the Zc axis is determined in the direction in which the photographing apparatus 101 is directed, and the Xc axis and the Yc axis are determined in directions orthogonal to each other on a plane perpendicular to the Zc axis. The position and orientation of the photographing apparatus 101 can be obtained by Tsai's algorithm or the like.

  The captured image coordinate system Ci is determined based on the virtual plane VP. In FIG. 11, the origin Oi of the captured image coordinate system Ci is defined at one vertex of the virtual plane VP, and the Xi axis and the Yi axis are defined in directions along one side and the other side of the virtual plane VP that are orthogonal to each other. The photographed image coordinate system Ci can be uniquely obtained from the photographing apparatus coordinate system Cc using basic attributes (focal length, angle of view, resolution, etc.) of the photographing apparatus 101. Further, the position of the virtual light spot appearing on the virtual plane VP can be easily expressed by coordinates in the captured image coordinate system Ci.

  The LCD coordinate system Cl is determined based on the spatial light modulator 21. In FIG. 11, the origin Ol of the LCD coordinate system Cl is determined at one vertex of the spatial light modulator 21. In addition, the Xl axis and the Yl axis are determined in a direction along one side and the other side orthogonal to each other of the spatial light modulator 21, and the Zl axis is determined in a direction perpendicular to the surface.

  In the present embodiment, each pixel of the spatial light modulator 21 is arranged along the Xl axis and the Yl axis. The position of each pixel of the spatial light modulator 21 can be easily expressed by coordinates in the LCD coordinate system Cl.

  The lens coordinate system Ce is determined based on the lens array 22. In FIG. 11, the origin Oe of the lens coordinate system Ce is determined on one surface of the lens array 22. Further, an Xe axis and a Ye axis are defined in a direction orthogonal to each other on one surface of the lens array 22, and a Ze axis is defined in a direction perpendicular to the surface.

  In the present embodiment, the lenses 220 of the lens array 22 are arranged along the Xe axis and the Ye axis. The position of each lens 220 of the lens array 22 can be easily expressed in the lens coordinate system Ce.

  The world coordinate system Cw is set as a reference for all coordinate systems. In the present embodiment, the world coordinate system Cw is defined so as to coincide with the lens coordinate system Ce in a state where the spatial light modulator 21 and the lens array 22 are in a predetermined ideal positional relationship. When the lens array 22 is in an ideal positional relationship with respect to the spatial light modulator 21, the origin Oe, Xe axis, Ye axis, and Ze axis of the lens coordinate system Ce are the origin Ow, Xw axis of the world coordinate system Cw, It corresponds to the Yw axis and the Zw axis. Here, the ideal positional relationship between the spatial light modulator 21 and the lens array 22 is, for example, the positional relationship between the spatial light modulator 21 and the lens array 22 in design.

  Actually, since the lens array 22 is often not strictly positioned at an ideal position, a deviation occurs between the lens coordinate system Ce and the world coordinate system Cw.

(5-2) Conversion to World Coordinate System (5-2-1) Lens Coordinate System FIG. 12 is a schematic diagram for explaining the relationship between the lens coordinate system Ce and the world coordinate system Cw. As shown in FIG. 12, the deviation amount of the lens coordinate system Ce with respect to the world coordinate system Cw, the parameters _{e x,} e _y, represented by _e z, e _theta. The parameters e _x , e _y , and e _z represent the amount of deviation in the Xw axis direction, the amount of deviation in the Yw axis direction, and the amount of deviation in the Zw axis direction of the origin Oe of the lens coordinate system Ce with respect to the origin Ow of the world coordinate system Cw. Also, the parameters e _theta, represents the angle of inclination of the Xe axis and Ye axis of the lens coordinate system Ce respect Xw axis and Yw axis of the world coordinate system Cw. Here, since it is relatively easy to maintain the parallel relationship between the spatial light modulator 21 and the lens array 22, it is assumed that the Zw axis of the world coordinate system Cw and the Ze axis of the lens coordinate system Ce are parallel. be able to.

In this case, the relationship between the lens coordinate system Ce and the world coordinate system Cw is expressed by the following equation (1). Here, F is a function having e _x , e _y , e _z , and e _θ as variables.

As represented by the following equation (2), the coordinates of the world coordinate system Cw represented by (x _w , y _w , z _w ) are expressed by parameters F _x , e _y , e _z , e _θ by the function F. Can be converted into the coordinates of the lens coordinate system Ce represented by (x _e , y _e , z _e ).

Furthermore, the coordinates of the lens coordinate system Ce can be converted to the coordinates of the world coordinate system Cw by the inverse function F ⁻¹ calculated from the equation (2). Thereby, the position of each lens 220 of the lens array 22 can be converted from the coordinates of the lens coordinate system Ce to the coordinates of the world coordinate system Cw using the parameters e _x , e _y , e _z , and e _θ .

Incidentally, the deviation amount represented by the parameter e _z, as described above, the parameter e _x, actually less likely to occur as compared with the amount of deviation represented by e _y, e _theta. Therefore, in the following, e _z = 0 is set for simplification of description.

(5-2-2) Other Coordinate Systems As described above, the world coordinate system Cw is determined based on an ideal position of the lens array 22 that is predetermined with respect to the spatial light modulator 21. Therefore, the LCD coordinate system Cl and the world coordinate system Cw have a unique relationship, and the LCD coordinate system Cl can be uniquely converted to the world coordinate system Cw. Therefore, the position of each pixel of the spatial light modulator 21 can be uniquely converted from the coordinates of the captured image coordinate system Ci to the coordinates of the world coordinate system Cw.

  Further, as described above, the position and orientation of the photographing apparatus 101 are obtained by the Tsai algorithm or the like. Since the positional relationship between the imaging device 101 and the spatial light modulator 21 is constant, the imaging device coordinate system Cc and the LCD coordinate system Cl have a unique relationship. Thereby, the imaging device coordinate system Cc can be uniquely converted to the world coordinate system Cw.

  The captured image coordinate system Ci can be uniquely obtained from the imaging apparatus coordinate system Cc as described above. Thereby, the captured image coordinate system Ci can be uniquely converted into the world coordinate system Cw. Therefore, the position of the virtual light spot on the virtual plane VP can be uniquely converted from the coordinates of the captured image coordinate system Ci to the coordinates of the world coordinate system Cw.

(6) Operations of Measurement Calculation Unit and Display Control Unit Operations of the measurement calculation unit 102 and the display control unit 5 will be described. 13 to 15 are flowcharts illustrating the operation of the measurement calculation unit 102, and FIG. 16 is a flowchart illustrating the operation of the display control unit 5.

  First, the operation of the measurement calculation unit 102 will be described. The operation of the measurement calculation unit 102 is performed when the stereoscopic display system 10 is manufactured.

As shown in FIG. 13, the measurement calculation unit 102 first performs lens coordinate calculation processing based on the captured image of the imaging apparatus 101 (step S1). Next, the measurement calculation unit 102 performs a parameter calculation process (step S2). As a result, the values of the parameters e _x , e _y , e _z , and e _θ are calculated. The calculated parameters e _x , e _y , e _z , and e _θ are stored in the storage unit 6. The values of these parameters e _x , e _y , e _z , e _θ are used as correction data.

  Details of the lens coordinate calculation processing in step S1 will be described. As illustrated in FIG. 14, the measurement calculation unit 102 reads the basic attributes of the stereoscopic display 1 and the basic attributes of the imaging device 101 from the storage unit 6 (step S11). Next, the measurement calculation unit 102 provisionally sets the position of each lens 220 on the captured image of the imaging apparatus 101 based on the diameter, interval, and arrangement of the lenses 220 of the lens array 22 and the position and orientation of the imaging apparatus 101. (Step S12).

  Next, the measurement calculation part 102 gives a pixel lighting signal to the display control part 5 (step S13). In response to the pixel lighting signal, the display control unit 5 lights one pixel of the spatial light modulator 21. In this case, the pixel to be lit is designated by the pixel lighting signal.

  Next, the measurement calculation unit 102 gives a photographing signal to the photographing apparatus 101 (step S14). In response to the photographing signal, the photographing apparatus 101 photographs the element display surface 2 of the stereoscopic display 1. In this case, a light intensity distribution due to light emitted from the pixels lit in step S <b> 13 appears on the photographed image of the photographing apparatus 101.

  Next, the measurement calculation unit 102 acquires the luminous intensity distribution appearing on the captured image of the imaging apparatus 101 as luminous intensity distribution data (step S15). Next, the measurement calculation unit 102 determines whether or not the light intensity distribution data has been acquired for all the pixels (step S16). When the light intensity distribution data is not acquired for all the pixels, the measurement calculation unit 102 returns to the process of step S13 and turns on one pixel among the pixels that are not turned on.

  By repeating the processes of steps S13 to S16, the pixels of the spatial light modulator 21 are sequentially turned on one by one, and the element display surface 2 is photographed by the photographing apparatus 101 every time the pixels are turned on. Thereby, light intensity distribution data for all the pixels is acquired. Note that the order of the pixels to be lit may be set in advance, for example, along the arrangement of the pixels, or may be random.

  When the light intensity distribution data is acquired for all the pixels, the measurement calculation unit 102 obtains the highest light intensity among the plurality of pixels corresponding to one lens 220 of the lens array 22 based on the acquired light intensity distribution data. A pixel is specified (step S17).

  Note that, as a method of specifying the position with the highest luminous intensity on the photographed image of the photographing apparatus 101, a method of obtaining the center of gravity of the integrated distribution by superimposing the luminous intensity distributions of the respective photographed images, and a peak when the sum of the luminous intensity is obtained is used. For example, there is a method, or a method of performing weighting based on a probability that a light spot appears at a position to be calculated, and the like.

  Next, the measurement calculation unit 102 determines the coordinates of the principal point 220a of one lens 220 (hereinafter referred to as lens coordinates) based on the position of the pixel and the position of the photographing center point Pc of the photographing apparatus 101 in the world coordinate system. Calculate with Cw (step S18).

  Specifically, in step S <b> 11, the position of each pixel of the spatial light modulator 21 and the position of the photographing center point Pc of the photographing apparatus 101 are read from the storage unit 6 as basic attributes of the stereoscopic display 1 and the photographing apparatus 101. The position of each pixel and the position of the photographing center point Pc of the photographing apparatus 101 can be represented by the coordinates of the world coordinate system Cw.

The principal point 220a of the lens 220 is on a straight line connecting the pixel specified in step S17 and the photographing center point Pc of the photographing apparatus 101. In the present embodiment, since e _z = 0, the lens coordinates in the Zw axis direction are specified. Thereby, in the world coordinate system Cw, the lens coordinates can be obtained from the coordinates of the pixel specified in step S17 and the coordinates of the photographing center point Pc of the photographing apparatus 101.

  The lens coordinates can also be obtained as follows based on the position of the photographing center point Pc of the photographing apparatus 101 and the position of the light spot appearing on the photographed image. First, the position of the virtual light spot Pi on the virtual plane VP can be obtained from the position of the light spot appearing on the captured image. As described above, the position of the virtual light spot Pi can be represented by the coordinates of the world coordinate system Cw.

  The principal point 220a of the lens 220 is on a straight line connecting the virtual light spot Pi corresponding to the pixel specified in step S17 and the photographing center point Pc of the photographing apparatus 101. Thereby, in the world coordinate system Cw, the lens coordinates can be obtained from the coordinates of the virtual light spot Pi corresponding to the pixel specified in step S17 and the coordinates of the photographing center point Pc of the photographing apparatus 101.

In the following description, the lens coordinates of the world coordinate system Cw calculated in step S18 are represented by P ^e _w . Further, numbers 1 to n (n is the number of all the lenses 220) are assigned to the respective lenses 220, and the lens coordinates of the i-th (1 ≦ i ≦ n) lens 220 calculated in step S18 are indicated. Expressed as iP ^e _w .

Next, measurement computation unit 102 determines whether the calculated lens coordinate ^P _{e w} for all lens 220 (step S19). When the lens coordinates P ^e _w have not been calculated for all the lenses 220, the measurement calculation unit 102 returns to the process of step S17.

By the process of step S17~S19 are repeated, the lens coordinates ^P _{e w} of all the lenses 220 of the spatial light modulator 21 is calculated. The order of the lenses 220 for calculating the lens coordinates may be set in advance along the arrangement of the lenses 220, or may be random, for example.

When the lens coordinates P ^e _w are calculated for all the lenses 220, the measurement calculation unit 102 ends the lens coordinate calculation process.

  Next, the parameter calculation process in step S2 of FIG. 13 will be described. As shown in FIG. 15, the measurement calculation unit 102 acquires the lens coordinates of each lens 220 in the lens coordinate system Ce based on the arrangement and interval of the lenses 220 (step S31). The arrangement and interval of the lenses 220 are included in the basic attributes of the stereoscopic display 1 read from the storage unit 6 in step S11 of FIG.

In the following description, the lens coordinates of the lens coordinate system Ce acquired in step S31 are represented by P ^l _e . Further, the lens coordinates of the i-th lens 220 calculated in step S31 are represented by iP ^l _e .

Next, the measurement calculation unit 102 initializes the shift amount of the lens array 22 stored in the storage unit 6 (step S32). Specifically, the values of the parameters e _x , e _y , e _z , e _θ calculated in the previous parameter calculation process are returned to predetermined initial values.

  Next, the measurement calculation unit 102 sets an allowable range of the deviation amount of the lens array 22 (step S33). When the position of the lens array 22 is manually adjusted when the stereoscopic display 1 is manufactured, the amount of deviation of the lens array 22 that actually occurs is limited to a certain range. In step S33, an allowable range for the amount of deviation that occurs when the lens array 22 is manually aligned is set. For example, the allowable range of the deviation amount is set to be equal to or smaller than the lens pitch of the lens array 22. Here, the lens pitch refers to the distance between the principal points 220a of the adjacent lenses 220.

Next, the measurement calculation unit 102 converts the lens coordinates P ^l _e in the lens coordinate system Ce into the lens coordinates P ^l _w in the world coordinate system Cw using the parameters e _x , e _y , e _z , and e _θ. (Refer to said Formula (2). Step S34). Here, the lens coordinates P ^l _e are represented by (x _e , y _e , z _e ), and the lens coordinates P ^l _w are represented by (x _w , y _w , z _w ).

Subsequently, the measurement calculation unit 102 obtains the values of the parameters e _x , e _y , e _z , and e _θ based on the lens coordinates P ^l _e and the lens coordinates P ^l _w of each lens 220. For example, an appropriate value of the parameters e _x , e _y , e _z , and e _θ representing the deviation amount is obtained using the least square method. In addition, the values of the parameters e _x , e _y , e _z , and e _θ may be obtained by other methods. For example, with respect to all the lenses 200, the sum of the absolute values of the differences between the lens coordinates P ^l _e and the lens coordinates P ^l _w is obtained, and the parameters e _x , e _y , e _z , e _θ are obtained by iterative calculation such as Newton's method. May be obtained. Hereinafter, in this example, a method for obtaining the values of the parameters e _x , e _y , e _z , and e _θ using the least square method will be described.

For each lens 220, the measurement calculation unit 102 calculates a distance ε between the lens coordinates P ^l _w converted in step S34 and the lens coordinates P ^e _w calculated in step S18 of FIG. 14 (step S35). The distance εi for the i-th lens 220 is represented by iP ^e _w −iP ^l _w .

  Next, the measurement calculation unit 102 calculates the square sum S of distances ε (see the following formula (3)) for the lens 220 (step S36).

Next, the measurement calculation unit 102 calculates the values of the parameters e _x , e _y , e _z , and e _θ that minimize the square sum S (step S37). Mathematically, as shown in the following formula (4), the square sum S parameter e _x, e y, as a function obtained by partially differentiating _{e z,} e _theta is 0 respectively, the parameter e _x as the correction data , E _y , e _z and e _θ are obtained.

However, it is relatively difficult to analytically solve the above equation (4) including trigonometric functions or higher order terms. In this case, a solution can be obtained numerically by using the Newton method or the like. For example, the parameter within the allowable range of the deviation amount set in step S33 e _x, e y, by performing a calculation by applying specific values to _{e z,} e _theta, square sum S becomes minimum parameter e _x , E _y , e _z and e _θ are specified.

In consideration of errors due to the resolution of the photographing apparatus 101 and the manufacturing accuracy of the lens array 22, parameters are calculated using a least square method weighted according to the manufacturing accuracy of the lens array 22 instead of the above equation (3). The values of e _x , e _y , e _z , and e _θ may be obtained. In this case, examples of the weight include dispersion, deviation, symmetry, intensity, or summation of the luminous intensity distribution.

Thus, after calculating the values of the parameters e _x , e _y , e _z , and e _θ , the measurement calculation unit 102 ends the parameter calculation process.

  Next, the operation of the display control unit 5 will be described. The operation of the display control unit 5 is executed when the stereoscopic display system 10 is manufactured and when the stereoscopic display system 10 is used. Specifically, the processes of steps S41 to S43 are performed when the stereoscopic display system 10 is manufactured, and the processes of steps S41, S42, and S44 to S49 are performed when the stereoscopic display system 10 is used.

  As illustrated in FIG. 16, the display control unit 5 reads out the three-dimensional shape data and the basic attributes of the three-dimensional display 1 from the storage unit 6 (step S41).

  Next, the display control unit 5 determines whether or not a pixel lighting signal is given from the measurement calculation unit 102 (step S42). When the pixel lighting signal is given, the display control unit 5 controls the spatial light modulator 21 to light the pixel specified by the pixel lighting signal (step S43). Thereafter, the display control unit 5 returns to the process of step S42. When the processes of steps S42 and S43 are repeated, the lens coordinate calculation process of FIG. 14 and the parameter calculation process of FIG. 15 by the measurement calculation unit 102 are executed.

When the pixel lighting signal is not given, the display control unit 5 reads the values of the parameters e _x , e _y , e _z , and e _θ stored in the storage unit 6 as correction data (step S44).

Next, the display control unit 5 corrects an ideal coordinate to an actual coordinate for one lens 220 (for example, the i-th lens 220) (step S45). Specifically, ideal lens coordinates are calculated based on the basic attributes of the stereoscopic display 1 read from the storage unit 6 in step S41. The lens coordinates are corrected to the actual lens coordinates using the values of the parameters e _x , e _y , e _z and e _θ read from the storage unit 6 in step S44.

  Next, the display control unit 5 displays on the pixel group assigned to one lens 220 (for example, the i-th lens 220) based on the corrected lens coordinates, the solid shape data, the angle of view of the lens 220, and the like. Rendering of a power image (creation of stereoscopic image data) is performed (step S46). Note that the angle of view of the lens 220 is included in the basic attributes of the stereoscopic display 1 read from the storage unit 6 in step S41.

  Next, the display control unit 5 determines whether or not rendering has been performed for all the lenses 220 (step S47). If rendering has not been performed for all the lenses 220, the display control unit 5 returns to the process of step S45.

  When rendering has been performed for all the lenses 220, the display control unit 5 integrates the images rendered for each lens 220 into one image (step S48). In this way, stereoscopic image data is created using the correction data.

  Then, the display control unit 5 displays the image on the spatial light modulator 21 (step S49). Through steps S44 to S49, a stereoscopic image based on the generated stereoscopic image data is presented on the stereoscopic display 1.

(7) Effects of this Embodiment As described above, in this embodiment, the parameter e regarding the positional deviation of the lens array 22 is measured by the measurement calculation unit 102 based on the photographed image of the photographing apparatus 101 when the stereoscopic display system 10 is manufactured. The values of _x , e _y , e _z and e _θ are obtained.

During the operation (use) of the stereoscopic display system 10, the coordinates of the lenses 220 are corrected from the designed coordinates to the actual coordinates by using the values of the parameters e _x , e _y , e _z , and e _θ . Then, stereoscopic image data for presenting a stereoscopic image is created according to the corrected coordinates of each lens 220. Thereby, even when the position of the lens array 22 with respect to the spatial light modulator 21 is deviated from the designed position, an appropriate stereoscopic image can be presented.

(8) Other Embodiments In the above embodiment, in the lens coordinate calculation process by the measurement calculation unit 102, the pixels of the spatial light modulator 21 are turned on one by one in order, and all the states in which each pixel is turned on are photographed. Photographed by the apparatus 101. In this example, instead of lighting the pixels of the spatial light modulator 21 one by one in order, a correction image is displayed on the spatial light modulator 21.

(8-1) Correction Image FIG. 17 is a plan view illustrating an example of a correction image. As shown in FIG. 17, in the correction images ID1 and ID2, the pixels of the spatial light modulator 21 are lit in stripes.

  The lighting pixel line LI1 in the correction image ID1 in FIGS. 17A to 17D is configured by a plurality of columns of pixels arranged in parallel and spaced apart from the Yl axis of the LCD coordinate system Cl. In addition, the lighting pixel line LI2 in the correction image ID2 in FIGS. 17E to 17H is configured by a plurality of rows of pixels arranged parallel to and spaced from the Xl axis of the LCD coordinate system Cl. .

  The interval (cycle) between the lighting pixel lines LI <b> 1 and LI <b> 2 is set to be larger than the lens pitch of the lens array 22. Further, the numbers of the correction images ID1 and ID2 are determined according to the intervals between the lighting pixel lines LI1 and LI2.

  In the example of FIG. 17, the lit pixel lines LI1 and LI2 are arranged in a cycle of four pixels, and unlit pixels for three columns or three rows are arranged between the adjacent lit pixel lines LI1 and LI2. That is, in this example, the lens pitch is smaller than the length of 4 pixels.

  In this case, as shown in FIGS. 17A to 17D, four types of correction images ID1 in which the positions of the lighting pixel lines LI1 are shifted from each other by one pixel are used. Also, as shown in FIGS. 17E to 17H, four types of correction images ID2 are used in which the positions of the lit pixel line LI2 are shifted from each other by one pixel. In this way, the number of correction images ID1 and ID2 corresponding to the value obtained by dividing the range pitch by the pixel pitch is used.

  FIG. 18 is a schematic side view illustrating a direction in which light emitted from the lighting pixel line LI1 travels when the correction image ID1 is displayed on the spatial light modulator 21. FIG. 18 schematically shows a plane (X1-Zl plane in the LCD coordinate system Cl) perpendicular to the lighting pixel line LI1.

  As shown in FIG. 18, since the interval between the lit pixel lines LI1 is larger than the lens pitch, light emitted from two or more lit pixel lines LI1 passes through the principal point 220a of the common lens 220 to the photographing apparatus 101. There is no heading. Therefore, the light that appears in the captured image of the image capturing apparatus 101 through one lens 220 is limited to the light emitted from one lighting pixel line LI1.

  In FIG. 18, the lights F3 and F4 appear in the photographed image of the photographing apparatus 101, but the lights F1, F2, and F5 do not appear in the photographed image of the photographing apparatus 101.

  When the correction image ID2 is displayed on the spatial light modulator 21, the direction of light emitted from the lighting pixel line LI2 is in a plane perpendicular to the lighting pixel line LI2 (Yl-Zl plane in the LCD coordinate system Cl). FIG. 18 is the same as the state shown in FIG.

(8-2) Lens Array Positioning In this embodiment, the lens array 22 is positioned with respect to the spatial light modulator 21 by a manual operation or a manufacturing robot with a certain level of accuracy. Specifically, all the lenses 220 of the lens array 22 are aligned so as not to deviate by more than one lens pitch from the ideal position.

  As a method for aligning the lens array 22 with a certain accuracy or more, there is a method in which pixels in a predetermined area of the spatial light modulator 21 are lit and the lens array 22 is aligned with reference to the lit pixels.

  For example, pixels in an area substantially equal to one lens 220 are turned on at the center and four corners of the spatial light modulator 21. Then, the lens 220 arranged at the center and four corners of one surface of the lens array 22 is aligned with the position of the pixel to be lit. In this case, it is possible to easily align the lens array 22 so that all the lenses 220 do not deviate from the ideal position by one lens pitch or more.

(8-2) Lens Coordinate Calculation Processing FIG. 19 is a flowchart of lens coordinate calculation processing using the correction images ID1 and ID2. The lens coordinate calculation process of FIG. 19 will be described while referring to differences from the lens coordinate calculation process of FIG.

  As shown in FIG. 19, the measurement calculation part 102 gives a full lighting signal to the display control part 5 after performing the process of step S11 (step S51). In response to the full lighting signal, the display control unit 5 lights all the pixels of the spatial light modulator 21.

  Next, the measurement calculation unit 102 gives a photographing signal to the photographing apparatus 101 (step S52). In response to the photographing signal, the photographing apparatus 101 photographs the element display surface 2 of the stereoscopic display 1. On the photographed image of the photographing apparatus 101, a luminous intensity distribution in a state where all the pixels are turned on appears.

  Next, the measurement calculation unit 102 provisionally determines the actual position of each lens 220 of the lens array 22 from the distribution of luminous intensity appearing on the photographed image of the photographing apparatus 101 (step S53). Specifically, the barycentric point of light intensity (the position with the highest light intensity) is specified from the captured image of the image capturing apparatus 101. It is considered that the principal point 220a of each lens 220 is on a straight line connecting the position on the virtual plane VP corresponding to the center of gravity point and the photographing center point Pc of the photographing apparatus 101.

  Here, when the lens array 22 is shifted by one lens pitch or more with respect to the spatial light modulator, it is determined which lens 220 corresponds to the center of gravity of the luminous intensity appearing on the photographed image of the photographing apparatus 101. I can't.

  Therefore, in the present embodiment, the lens array 22 is aligned so that all the lenses 220 do not deviate from the ideal position by one lens pitch or more. Thereby, the actual position of each lens 220 can be tentatively obtained from the barycentric point of the luminous intensity appearing on the photographed image of the photographing apparatus 101.

  Next, the measurement calculation unit 102 specifies a pixel corresponding to each lens 220 based on the position of each lens 220 provisionally determined in step S53 (step S54).

  Next, the measurement calculation unit 102 gives a correction image display signal to the display control unit 5 (step S55). In response to the correction image display signal, the display control unit 5 causes the spatial light modulator 21 to display one of the correction images ID1 and ID2. The type of the correction image to be displayed is specified by the correction image display signal.

  Next, the measurement calculation unit 102 gives a photographing signal to the photographing apparatus 101 (step S56). In response to the photographing signal, the photographing apparatus 101 photographs the element display surface 2 of the stereoscopic display 1.

  Next, the measurement calculation unit 102 determines whether or not shooting has been performed for all the correction images ID1 and ID2 (step S57). As described above, the number of correction images ID <b> 1 and ID <b> 2 is determined according to the relationship between the lens pitch of the lens array 22 and the pixel pitch of the spatial light modulator 21.

If all the correction images ID1 and ID2 are not photographed, the measurement calculation unit 102 returns to the process of step S55. When performing photographing for all the correction image ID1, ID2, measurement computation unit 102, based on the captured image of the imaging apparatus 101 calculates a lens coordinate ^P _{e w} in the world coordinate system Cw of each lens 220 (step S58).

The method of calculating the lens coordinates ^P _{e w} in step S58 will be described in detail. In this embodiment, by displaying a plurality of correction images ID1 to the spatial light modulator 21, the coordinate x ^e _w main point 220a of the lens 220 in the Xw axis direction in the world coordinate system Cw is obtained. In addition, by displaying a plurality of correction images ID2 on the spatial light modulator 21, the coordinates y ^e _w of the principal point 220a of the lens 220 in the Yw axis direction of the world coordinate system Cw are obtained.

  FIG. 20 is a schematic side view along the Xw-Zw plane of the world coordinate system Cw. A method of calculating the coordinates of the lens 220 in the Xw axis direction of the world coordinate system Cw will be described with reference to FIG.

  In FIG. 20, the Xw-axis direction, the Yw-axis direction, and the Zw-axis direction of the world coordinate system Cw are parallel to the Xl-axis direction, the Yl-axis direction, and the Zl-axis direction of the LCD coordinate system Cl, respectively, and the world coordinate system Cw The Xw-Zw plane is parallel to the Xl-Zl plane of the LCD coordinate system Cl.

  In FIG. 20, the ideal position of the principal point 220a of the i-th lens 220 is indicated by iP1, and the actual position of the principal point 220a of the i-th lens 220 is indicated by iP2. Further, a row PA of pixels along the Xw axis of the world coordinate system Cw is shown. The principal point 220a of the i-th lens 220 and the pixel row PA are on a common Xw-Zw plane.

  In step S54 described above, the pixel corresponding to each lens 220 is specified. In FIG. 20, the pixels in the region RE in the pixel row PA are associated with the i-th lens 220.

  In step S55 described above, the plurality of correction images ID1 are sequentially displayed on the spatial light modulator 21, whereby each pixel included in the pixel row PA is sequentially turned on as the lighting pixel line LI1. At that time, it is assumed that the highest luminous intensity is obtained when the pixel Pm21 is turned on among the pixels in the region RE.

  In this case, the principal point 220a of the i-th lens 220 is considered to be located on a straight line connecting the pixel Pm21 and the photographing center point Pc of the photographing apparatus 101 on the Xw-Zw plane.

  Here, the distance dx between the pixel Pm21 and the principal point 220a of the lens 220 in the Xw-axis direction is expressed by the following equation (5).

In Expression (5), x ^m _l is the coordinates of the pixel Pm21 in the Xl axis direction of the LCD coordinate system Cl. H is a function for converting coordinates in the LCD coordinate system Cl into coordinates in the world coordinate system Cw.

Coordinate x ^m _l of pixel Pm21 is easily determined based on the type of basic attributes, and the correction image using read from the storage unit 6 in the process of step S11 (FIG. 19). Further, as described above, the LCD coordinate system Cl and the world coordinate system Cw have a unique relationship, and the coordinates in the LCD coordinate system Cl can be converted into the coordinates in the world coordinate system Cw.

  Further, the distance between the lens array 22 and the spatial light modulator 21 is Lf, and the angle formed by the straight line connecting the pixel Pm21 and the photographing center point Pc of the photographing apparatus 101 with respect to the spatial light modulator 21 and the lens array 22 is αx. Then, the following formula (6) is established.

In Expression (6), x ^c _w is a coordinate of the photographing center point Pc of the photographing apparatus 101 in the Xw axis direction of the world coordinate system Cw, and z ^c _w is a photographing apparatus 101 in the Zw axis direction of the world coordinate system Cw. The coordinates of the photographing center point Pc.

The distance Lf corresponds to the focal length of the lens 220, and is read as a basic attribute of the stereoscopic display 1 from the storage unit 6 in the process of step S11 (FIG. 19). Further, as described above, the coordinates x ^c _w and z ^c _w of the photographing center point Pc of the photographing apparatus 101 are uniquely specified.

From equations (5) and (6) it is possible to find the coordinates ^x _{e w} main point 220a of the lens 220.

In the case _{where e z} is not 0, the above equation (6), the Lf and Lf + _{e ^z,} a ^z _{c w} and _{^z c} w -e ^z.

In Figure 20, by displaying a plurality of correction images ID1 to the spatial light modulator 21 has been described how to obtain the coordinates x ^e _w main point 220a of the lens 220 in the Xw axis direction in the world coordinate system Cw , in the same manner, a plurality of correction images ID2 by displaying the spatial light modulator 21, it is possible to determine the coordinate y ^e _w principal point 220a of the lens 220 in the Yw axis of the world coordinate system Cw. In that case, the same principle as in FIG. 20 is used in the Yw-Zw plane of the world coordinate system Cw.

When ez = 0, the coordinate z ^e _w of the principal point 220a of the lens 220 in the Zw axis direction of the world coordinate system Cw is specified (z ^e _w = 0). Thereby, the lens coordinates iP ^e _w of the i-th lens 220 are obtained.

In this way, it is possible to obtain the lens coordinates P ^e _w of each lens 220 in the world coordinate system Cw.

(8-3) Effect In the present embodiment, the lens array 22 is aligned so that all the lenses 220 do not deviate from the ideal position by one lens pitch or more, and the correction images ID1 and ID2 are used for each. The lens coordinates of the lens 220 are calculated. In this case, as compared with the case where each pixel is turned on one by one and the lens coordinates of each lens 220 are calculated as described above, the number of times of photographing by the photographing apparatus 101 can be greatly reduced. Thereby, the time required for manufacturing the stereoscopic display system 10 can be significantly shortened.

(9) Still another embodiment In general, one dot of a display device such as an LCD is composed of three subpixels of RGB (red, green, blue). The three subpixels are arranged in a stripe shape or a triangle shape. When three subpixels are arranged in a stripe shape, the pixel pitch in one of the Xl-axis direction and the Yl-axis direction is about one third of the pixel pitch in the other. Therefore, when the correction images ID1 and ID2 are used, one of the correction images ID1 and ID2 is required to be three times as large as the other.

  In the above-described embodiment, a model of the lens array 22 in which light is controlled as in the case of using a pinhole array has been considered. However, as the pixel position moves away from a position immediately below the principal point 220a of the lens 220, the focal point increases. The distance changes, the aberration increases, and the error increases. Therefore, the quality of the stereoscopic image can be further improved by calculating the direction in which the light travels based on a model that takes into account the strict optical characteristics of the lens 220.

(10) Correspondence between each component of claim and each part of embodiment The following describes an example of the correspondence between each component of the claim and each part of the embodiment. It is not limited.

In the above embodiment, the display control unit 5 is an example of a correction image control unit and a stereoscopic image control unit, the measurement calculation unit 102 is an example of a correction data calculation unit, and the imaging device 101 is an example of an imaging unit. The storage unit 6 is an example of a storage unit, the lens 220 or the pinhole PH is an example of a light controller, and the lens array 22 or the pinhole array is an example of a light controller array. The world coordinate system Cw is an example of the first coordinate system, and the lens coordinate system Ce is an example of the second coordinate system.

  As each constituent element in the claims, various other elements having configurations or functions described in the claims can be used.

  INDUSTRIAL APPLICABILITY The present invention can be effectively used when manufacturing and operating a stereoscopic display system that presents various stereoscopic shapes.

1 is a schematic external view showing a stereoscopic display in a stereoscopic display system according to an embodiment of the present invention. It is a typical top view which shows the element display surface which comprises a three-dimensional display. It is typical sectional drawing which shows the element display surface which comprises a three-dimensional display. It is a figure for demonstrating the control function of the light by a lens, and the control function of the light by a pinhole. It is a figure for demonstrating the production method of the stereo image data for presenting a stereo image. It is a figure for demonstrating the image displayed on the spatial light modulator of a stereo display based on stereo image data. It is a block diagram which shows the structure of a stereoscopic display system and a stereoscopic display manufacturing system. It is a schematic diagram for demonstrating the relationship between an observation point and a luminous intensity. It is a schematic diagram which shows the positional relationship of an imaging device, a lens array, and a spatial light modulator. It is a figure for demonstrating the luminous intensity of the light spot on the picked-up image of an imaging device. It is a schematic diagram for demonstrating the relationship between an imaging device coordinate system, a picked-up image coordinate system, an LCD coordinate system, a lens coordinate system, and a world coordinate system. It is a schematic diagram for demonstrating the relationship between a lens coordinate system and a world coordinate system. It is a flowchart which shows operation | movement of a measurement calculating part. It is a flowchart which shows operation | movement of a measurement calculating part. It is a flowchart which shows operation | movement of a measurement calculating part. It is a flowchart which shows operation | movement of a display control part. It is a top view which shows an example of the image for a correction | amendment. It is a typical side view which shows the direction where the light emitted from a lighting pixel line heads when a correction image is displayed on a spatial light modulator. It is a flowchart of a lens coordinate calculation process using the image for correction. It is a typical side view along the Xw-Zw plane of the world coordinate system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 3D display 2 Element display surface 5 Display control part 6 Memory | storage part 10 3D display system 21 Spatial light modulator 22 Lens array 101 Image pick-up device 102 Measurement calculating part 220 Lens

Claims (3)

A three-dimensional display composed of element display surfaces;
Correction image control means for controlling the element display surface so as to display a correction image at the time of manufacturing the stereoscopic display;
Photographing means for photographing the element display surface in a state in which the correction image is displayed;
Correction data calculation means for calculating correction data based on a photographed image obtained by the photographing means;
Storage means for storing the correction data calculated by the correction data calculation means;
Stereoscopic image control means for controlling the element display surface so as to present a stereoscopic image based on stereoscopic image data for presenting a stereoscopic image;
The element display surface is
A spatial light modulator composed of a plurality of pixels that generate light;
A light controller array comprising a plurality of light controller, and controlling a direction of light generated from each pixel of the spatial light modulator,
The correction image control means sequentially turns on one or a plurality of pixel columns along a first direction among the plurality of pixels of the spatial light modulator and a second direction perpendicular to the first direction. By sequentially lighting one or a plurality of pixel rows along the line, a plurality of types of the correction images are sequentially displayed on the element display surface,
The photographing means photographs the element display surface each time each correction image is displayed on the element display surface,
The correction data calculating means is based on a positional relationship between the imaging means and the spatial light modulator and a position having the highest luminous intensity corresponding to each pixel of the spatial light modulator on a plurality of types of captured images obtained by the imaging means. Calculating the actual position of each light controller of the light controller array, the preset position of the light controller array relative to the spatial light modulator, and the position of the light controller array relative to the spatial light modulator. Calculate the deviation from the actual position as the correction data,
The stereoscopic image control means creates the stereoscopic image data using the correction data stored in the storage means during the stereoscopic image presentation operation, and presents the stereoscopic image based on the created stereoscopic image data. A stereoscopic display manufacturing system for controlling the spatial light modulator. A coordinate system in which the X axis, the Y axis, and the Z axis are determined based on the preset position of the light controller array is defined as a first coordinate system, and the actual position of the light controller array is used as a reference. A coordinate system in which the axis, the Y-axis, and the Z-axis are determined is a second coordinate system,
Variables indicating the amount of deviation between the first coordinate system and the second coordinate system in the X-axis direction, the Y-axis direction, the Z-axis direction, and the rotation direction around the Z-axis of the first coordinate system, respectively. When the second, third, and fourth parameters are used, the position of each light controller of the light controller array is set to the first coordinate system using the first, second, third, and fourth parameters. Expressed in theoretical coordinates at
The correction data calculating means is based on a positional relationship between the imaging means and the spatial light modulator and a position having the highest luminous intensity corresponding to each pixel of the spatial light modulator on a plurality of types of captured images obtained by the imaging means. Then, the coordinates in the first coordinate system of each light controller in the light controller array are calculated as actual coordinates, and the first coordinates are calculated based on the relationship between the actual coordinates and the theoretical coordinates of each light controller. , second, stereoscopic display manufacturing system according to claim 1, wherein the calculating the value of the third and fourth parameter as correction data. A method of manufacturing a stereoscopic display system,
The stereoscopic display system includes a stereoscopic display configured by an element display surface, storage means for storing correction data, and the element display surface so as to present a stereoscopic image based on stereoscopic image data for presenting a stereoscopic image. 3D image control means for controlling
The element display surface includes a spatial light modulator composed of a plurality of pixels that generate light, and a plurality of light beam controllers, and controls the direction of light generated from each pixel of the spatial light modulator. A child array,
The stereoscopic image control means creates the stereoscopic image data using the correction data stored in the storage means during the stereoscopic image presentation operation, and presents the stereoscopic image based on the created stereoscopic image data. Controlling the spatial light modulator;
The manufacturing method includes:
One or more pixels along the second direction perpendicular to the first direction and at least one pixel row along the first direction among the plurality of pixels of the spatial light modulator are turned on in order. A step of sequentially displaying a plurality of types of correction images on the element display surface by sequentially lighting the columns;
A step of obtaining a plurality of types of captured images by photographing the element display surface by photographing means each time each correction image is displayed on the element display surface;
Based on the positional relationship between the imaging unit and the spatial light modulator and the position with the highest luminous intensity corresponding to each pixel of the spatial light modulator on a plurality of types of captured images by the imaging unit, the light controller array The actual position of each light controller is calculated, and the amount of deviation between the preset position of the light controller array with respect to the spatial light modulator and the actual position of the light controller array with respect to the spatial light modulator Calculating as the correction data,
And a step of storing the correction data in the storage means.

Images