JP4853149B2 - Image processing apparatus, image display apparatus, image processing method, program, and recording medium - Google Patents

Description

  The present invention relates to an image processing device, an image display device, an image processing method, a program, and a recording medium for performing image processing for simultaneously displaying a bird's-eye image and a captured image taken at a shooting point specified on the bird's-eye image. About. More specifically, the present invention provides an image in which a geographical relationship between a plurality of captured images can be easily grasped by obtaining a display image by arranging a captured image captured at this shooting point at a shooting point of a bird's-eye view image. This relates to a processing apparatus or the like.

  Conventionally, when displaying a panoramic image as a wide-angle image, only a partial area is displayed so as to be scrollable, or the entire image is displayed in an elongated manner in the horizontal direction. In addition, in a computer that is one of devices that display a panoramic image, a plurality of panoramic images can be displayed simultaneously by opening a plurality of files related to the panoramic image.

For example, Patent Document 1 proposes a method of simultaneously displaying a global viewpoint image and a local viewpoint image. In other words, Patent Document 1 proposes a method for projecting an image on a dome-shaped screen to ensure an immersive feeling and for grasping the current position by displaying a bird's-eye view image (bird's-eye view image).
JP 2004-129003 A

  As described above, although a plurality of panoramic images can be displayed simultaneously, it is difficult to grasp the geographical relationship between the images. Moreover, even if a plurality of panoramic images are displayed as they are as local viewpoint images by the method disclosed in Patent Document 1, it is difficult to intuitively grasp the geographical relationship between the plurality of panoramic images.

  An object of the present invention is to make it possible to easily grasp the geographical relationship between a plurality of captured images.

The concept of this invention is
An image processing device that performs image processing for simultaneously displaying a bird's-eye image and a captured image captured at a capturing point specified on the bird's-eye image,
An image placement unit that obtains a display image by placing a shot image taken at the shooting point at the shooting point of the bird's-eye view image ,
The image placement section, an image, characterized in that arranged so as to face the captured image to the viewpoint, the further bird's-eye image in which the photographed image is arranged to convert the image seen from the viewpoint Ru obtain the display image In the processing unit.

In this invention, in order to simultaneously display a bird's-eye image and a photographed image photographed at a photographing point specified on the bird's-eye image, the photographed image photographed at this photographing point is arranged at the photographing point of the bird's-eye image. Thus, a display image is obtained. As described above, since the display image is obtained by arranging the captured image at the capturing point of the bird's-eye view image, the geographical relationship between the plurality of captured images can be easily grasped.

For example, a photographed image arranged at the photographing point of the bird's-eye view image is arranged so as to face the viewpoint, and further, the bird's-eye view image on which the photographed image is arranged is converted into an image viewed from the viewpoint to obtain a display image. In this case, the bird's-eye view image is displayed in a state viewed from the viewpoint, and the captured image is displayed in a state of facing the viewpoint, so that an image that is consistent with the viewing direction can be displayed. It is easier to grasp the geographical relationship between a plurality of captured images.

For example, the photographed image is a wide-angle image (panoramic image), and a partial region corresponding to the viewpoint is cut out from the wide-angle image and disposed at the photographing point of the bird's-eye view image. In this case, the content of the photographed image displayed on the bird's-eye view image corresponds to the viewpoint, and it is possible to fully understand the photographing point by the photographed image.

For example, a user operation unit is provided for the user to change the position of the viewpoint. By changing the viewpoint position, the direction in which the bird's-eye view image is viewed is changed, and the content of the captured image arranged in the bird's-eye view image is changed. This makes it easier to grasp the geographical relationship between a plurality of captured images.

  For example, a user operation unit for changing a bird's-eye view image and a captured image for a user to obtain a display image is provided. By changing the image, the display contents of the bird's-eye view image and the captured image are changed, and so-called scene conversion is performed.

  According to the present invention, a captured image taken at a shooting point is arranged at a shooting point of a bird's-eye image to obtain a display image, and the geographical relationship between a plurality of shot images can be easily grasped.

  (1) Audiovisual system

  FIG. 1 shows the configuration of an audiovisual system. This audio-visual system is arranged in a space between an audio-visual room (room) 1 composed of an inner wall 2 and an outer wall 3 in which walls on one surface are arranged at a predetermined interval, and an inner wall 2 and an outer wall 3. Are equipped with equipment such as a projector 10 for at least performing a video display function or an audio output function.

  The audio-visual room 1 is an audio-visual room formed in a general home, and an inner wall 2 is provided on one of four walls forming one room of a normal general home, and the wall on the one side is This is a double structure composed of an inner wall 2 and an outer wall 3. The audio-visual room 1 is formed by arranging the remaining three walls and the outer wall 3 on the wall surface of a building structure, for example, in the same manner as a wall in a normal house. Moreover, even if it exists in a ceiling and a floor, it forms the same as the ceiling and floor in a normal house.

  As shown in FIG. 2, the inner wall 2 has a vertical frame 4 installed at a fixed pitch between the ceiling and the floor, and then a horizontal frame 5 is horizontally disposed between the vertical frames 4 and 4. A skeleton is formed, and the plate-like units 6A to 6D are arranged on the skeleton as shown in FIG. Here, plate unit 6A-6D comprises the plate-shaped member.

  The details of the frame configuration of the inner wall 2 will be further described with reference to FIG. FIG. 3 shows only the frame portion 24 corresponding to the pair of vertical frames 4 and 4. The upper side of FIG. 3 shows a view when the frame part 24 is viewed from the front in the figure, and the lower side of FIG. 3 shows the view when the frame part 24 is viewed from the upper side in the figure. In other words, the frame portion 24 has a U-shaped cross section on the surface to which the plate unit is fixed.

  As shown on the upper side of FIG. 3, the frame portion 24 includes six frames 51 </ b> A to 51 </ b> F. Here, each of the frames 51E and 51F constitutes the vertical frame 4 shown in FIG. 2, and each of the frames 51A to 51D constitutes a horizontal frame 5 shown in FIG. Each of the frames 51A to 51F is formed of a material such as metal, for example.

  Each of the frames 51A to 51F is provided with a long hole (or a round hole) at a predetermined position, and these long holes are respectively provided at predetermined positions for equally dividing the frames 51A to 51F. It has been. For example, when a part of the frame 51 (frames 51A to 51F) is enlarged, the frame 51 has long holes 53-1 to 53-4 that are long in the short direction, as shown in the example of FIG. Each of them is drilled at a predetermined interval.

  That is, in the frame portion 24, as shown in FIG. 3, among the frames 51 </ b> A to 51 </ b> F, each of the frames 51 </ b> A to 51 </ b> D constituting the horizontal frame 5 is provided with a vertically long slot. Each of the frames 51E and 51F to be provided is provided with a horizontally long slot.

  The frames 51E and 51F are formed as L-shaped angle members, and are fixed to each of the straight frames 51A to 51D by welding, for example. For example, in the frame portion 24, as shown in FIG. 5, a straight frame (horizontal frame) 51A and an L-shaped frame (vertical frame) 51E are fixed by welding, so that As shown, the cross section has a U-shaped shape.

  That is, although the details will be described later, since the frame portion 24 has a U-shaped cross section, each of the frames 51E and 51F is not only from the front side in the figure but also from the left and right side surfaces. It is possible to use a long hole provided in.

  Next, details of attachment of the plate unit to the frame portion 24 described above will be described. In the following description, the frames 51A to 51F are simply referred to as the frame 51 when it is not necessary to distinguish them individually.

  With reference to FIG. 6 and FIG. 7, when attaching a plate-shaped unit to the frame part 24, the load direction support 25 supported in a load direction is demonstrated. 6 shows a perspective view when the load-direction support 25 is fixed to the frame 51, and FIG. 7 shows a view when FIG. 6 is viewed from the upper side.

  The load direction support 25 is formed from materials, such as a metal, for example. As shown in FIG. 6, the load-direction support 25 is made of an L-shaped angle material, and the L-shaped angle material has a bolt 61 </ b> A and a bolt 61 </ b> B that can be fitted in a long hole provided in the frame 51. Each is provided. Here, the frame 51 is a horizontal frame, and corresponds to the frames 51B to 51D.

  When the weight direction support 25 is fixed to the frame 51, as shown in FIG. 7, the bolts 61A and 61B of the load direction support 25 are fitted into predetermined long holes of the frame 51, and the bolts 61A and 61 The nut 62A and the nut 62B are screwed into each of the bolts 61B.

  With reference to FIG. 8, FIG. 9, the front-back direction support 26 supported in the front-back direction when attaching a plate-shaped unit to the flame | frame part 24 is demonstrated. The front-rear direction support 26 is made of a material such as metal, for example. As shown in FIG. 8, the longitudinal support 26 (26A, 26B) is formed in a U-shaped cross section and is detachably attached to the frame portion 24. As shown in FIG. 9, the front / rear direction support 26 uses the U-shaped shape to sandwich the plate unit 36 between the front / rear direction supports 26A and 26B. Support with.

  As shown in FIG. 9, for example, a predetermined number of bolts (that can be fitted into long holes provided in a frame (vertical frame) 51 of the frame portion 24) are provided on each of the front and rear support 26 </ b> A and 26 </ b> B. These bolts are fitted into the elongated holes of the frame 51 and are fixed by a number of nuts (the nut 73A and the nut 73B in the example of FIG. 9) corresponding to the number of bolts. .

  Further, when the plate-like unit 36 is sandwiched between the front and rear support 26A and 26B, the plate-like unit 36 and the plate-like unit 36 are further sandwiched so as to further sandwich the plate-like unit 36 (in FIG. 9, the cushioning material 71A). The buffer material 71B, the buffer material 72A, and the buffer material 72B) are also sandwiched together.

  Each of the buffer material 71A, the buffer material 71B, the buffer material 72A, and the buffer material 72B is formed of a material such as urethane (sponge) or rubber, for example. Each of the buffer material 71A and the buffer material 71B is provided in front of the plate-like unit 36, and each of the buffer material 72A and the buffer material 72B is provided behind the plate-like unit 36 and is applied to the plate-like unit 36. Absorbs and protects the impact.

  Further, when removing the plate unit 36 from the frame portion 24, the nut 73A screwed to the bolt provided on the front / rear support 26A is loosened, and the front / rear support 26A is moved to the right along the elongated hole of the frame 51. Let Similarly, the nut 73B screwed to the bolt provided on the front-rear support 26B is loosened, and the front-rear support 26B is moved to the left along the elongated hole of the frame 51. Thereby, the plate-like unit 36 can be removed.

  With the nuts 73A, 73B completely removed from the bolts, the bolts provided on the front / rear direction supports 26A, 26B can be pulled out from the long holes of the frame 51 and the front / rear direction supports 26A, 26B can be removed from the frame portion 24. The plate unit 36 can be removed.

  In the frame configuration of the inner wall 2 shown in FIG. 2, a predetermined number of the frame portions 24 described above are arranged in the horizontal direction. With reference to FIG. 10, the case where the frame part 24 is connected in the horizontal direction will be described. FIG. 10 shows a view of the connected frame portions 24-1 and 14-2 as viewed from above.

  In the frame portions 24-1 and 24-2, each of the frames 51E and 51F, which are vertical frames, is formed as an L-shaped angle member as described with reference to FIGS. 3 and 5, and is not only from the front side. The elongated holes provided in each of the frames 51E and 51F can also be used from the left and right side surfaces. That is, since each of the frame portions 24-1 and 24-2 has a U-shaped cross section of a surface to which the plate units 36A and 36B are fixed, the U-shaped shape is used. Then, the frame portions 24-1 and 14-2 are connected in the horizontal direction.

  The joint portion 81 fixes, for example, a left side surface in the drawing of the frame portion 24-1 and a right side surface in the drawing of the frame portion 24-2 with bolts 81A and nuts 81B. That is, the joint portion 81 has an L-shaped shape when the L-shaped frame 51F of the frame portion 24-1 and the L-shaped frame 51E of the frame portion 24-2 are arranged in the horizontal direction. The long holes provided in each of the facing surfaces are fixed by a predetermined number of bolts 81A and nuts 81B.

  As described above, the inner wall 2 is composed of a plurality of plate-like units (plate-like members) held so as to be replaceable. Although FIG. 1 shows that the entire inner wall 2 is made of a plate-like unit, only a part of the inner wall 2 may be made of a plurality of plate-like units held in a replaceable manner. Thereby, an indoor atmosphere can be variously changed by replacing | exchanging to the plate-shaped unit from which makeup | decoration differs as needed. Further, it is possible to easily change the size of the screen for rear projection, or change the position of the diaphragm to which the vibrator is attached.

  Returning to FIG. 1, the plate-like units 6 </ b> A to 6 </ b> D are various functional units related to viewing of video content, and are arranged in a framework according to functions, designs, and the like required for the audiovisual room 1. In the example shown in FIG. 1, three plate-like units 6A each serving as a speaker are arranged between the vertical frames 4 at both ends of the inner wall 2. Hereinafter, the plate-like unit 6A responsible for the function of the speaker is appropriately referred to as a speaker unit.

  Here, the speaker unit 6 </ b> A is configured such that vibrators 8 that vibrate when driven by audio signals are arranged at a constant pitch on a diaphragm 9 that vibrates due to vibrations of the vibrator 8. Thus, the vibrator 8 is driven based on the audio signal, so that the diaphragm 9 is vibrated to function as a speaker. Note that the speaker unit 6A is configured to hold a diaphragm 9 in a frame body (not shown) so as to freely vibrate, thereby ensuring sufficient strength and being arranged on the vertical frame 4. A decorative member made of saran or the like is provided on the front side surface of the diaphragm 9.

  In the audiovisual room 1, between the vertical frames 4 and 4 inside the speaker unit 6A, a plate unit 6B responsible for a simple blindfold function and a plate unit 6C responsible for a screen function are arranged. In the following, the plate-like unit responsible for the blind function is referred to as a blank unit, and the plate-like unit 6C responsible for the function as a screen is referred to as a screen unit. Here, the blank unit 6 </ b> B is formed of a plate material such as a gypsum board with makeup on the front side surface by assuming a simple blindfold function.

  On the other hand, the screen unit 6C functions as a screen of the rear projection projector, and is formed by reinforcing a plate material applied to the screen of the rear projection projector with a frame. Specifically, it is formed by holding a frosted glass, an opaque acrylic plate or the like with a frame. In the audiovisual room 1, the projector 10 is disposed on the outer wall 3 located on the back surface of the screen unit 6C corresponding to the screen unit 6C. As a result, the audiovisual room 1 is created so that the video of the various video contents can be viewed by the screen unit 6C.

  In the audiovisual room 1, a blank unit 6B and a plate-like unit 6D that functions as an operation unit of the video equipment are arranged between the vertical frames 4 and 4 following the screen unit 6C. In the following, the plate-like unit that functions as the operation unit is referred to as an operation unit.

  Here, the operation unit 6D is provided with a bay 11 in which video equipment and the like are arranged. For example, in the bay 11, among the video devices provided in the audiovisual room 1, a device that needs to directly accept a user operation, for example, a DVD (Digital Versatile Disc) player that needs to replace a recording medium, a video A tape recorder or the like is provided. For example, the bay 11 is provided with a receiving unit that receives a remote control signal from a remote commander necessary for remote control of each video device.

  In a device other than the device related to the recording medium described above, the device is disposed in a space between the inner wall 2 and the outer wall 3. The distance D between the inner wall 2 and the outer wall 3 is set to a distance sufficient for humans to enter the space between the inner wall 2 and the outer wall 3 to install and maintain various video devices. One of the blank units 6 </ b> B is installed so that it can be removed from the front so that an entry port into which a human can enter can be formed in the space between 2 and the outer wall 3.

  In the audiovisual system shown in FIG. 1, a wall of one of the four walls of the audiovisual room 1 is formed by an inner wall 2 and an outer wall 3 arranged at a predetermined interval, and a screen unit 6C and a speaker are formed on the inner wall 2. Unit 6A is arranged. The projector 10 and the speaker unit 6 </ b> A are related to video content obtained from video equipment (including equipment related to a recording medium placed in the bay 11) placed in the space between the inner wall 2 and the outer wall 3. Driven by signal and audio signal. As a result, an image is projected from the projector 10 onto the screen unit 6C to display an image, and sound is output from the speaker unit 6A.

  According to the audiovisual room system shown in FIG. 1, one of four walls is formed by an inner wall 2 and an outer wall 3 having a space between them, and a screen unit 6C and a speaker unit 6A are arranged on the inner wall 2. In the space between the inner wall 2 and the outer wall 3, the projector 10 and other equipment (equipment) related to video content are arranged, and the equipment related to video content is arranged in the audiovisual room 1. Instead, it is possible to satisfactorily view content without disturbing the indoor atmosphere.

  In addition, according to the audiovisual system shown in FIG. 1, the inner wall 2 and the outer wall 3 are arranged at a predetermined interval that allows a person to enter, so that various video devices can be placed in the space between the inner wall 2 and the outer wall 3. When arranging the various video devices arranged in the space, a service person or the like can enter and work on the space, and workability can be improved.

  Further, according to the audiovisual system shown in FIG. 1, the inner wall 2 is formed of a plurality of plate material units, and a predetermined number of plate-like units constitute the speaker unit 6A, thereby incorporating the function of the speaker in the wall surface itself. It is. Therefore, in this audiovisual system, the audio of the video content can be enjoyed without separately arranging a speaker in the audiovisual room 1, thereby further preventing the room atmosphere from being disturbed. . In this case, it is also possible to provide high-quality sound by dividing the inner wall of the speaker into a plurality of parts.

  In addition, according to the audiovisual system shown in FIG. 1, the inner wall 2 is formed by a plurality of plate-like units held so as to be replaceable. The atmosphere can be changed in various ways, and further, the size of the screen or the position of the speaker can be easily changed. Moreover, this board | plate material unit can be removed and it can be set as the entrance into space.

  FIG. 11 shows another configuration example of the audiovisual system. This audiovisual system is arranged in a space between an audiovisual room (room) 21 composed of an inner wall 2 and an outer wall 3 in which two corresponding walls are arranged at a predetermined interval, and the inner wall 2 and the outer wall 3. 2 is equipped with equipment such as the projector 10 for causing at least a video display function or an audio output function. In FIG. 11, portions corresponding to those in FIG.

  That is, in this audiovisual system, two opposing faces of the four walls are formed by the inner wall 2 and the outer wall 3 having a space therebetween. Of the two opposing surfaces, one inner wall 2 is formed by the speaker unit 6A and the blank unit 6B, and the speaker unit 6A is driven in the space between the inner wall 2 and the outer wall 3 on one side. An amplifier 23 is disposed.

  The other inner wall 2 is formed by a screen unit 6C, a blank unit 6B, and a speaker unit 6A, and the video equipment 22 is arranged in a space between the inner wall 2 and the outer wall 3 on the side where the screen unit 6C is provided. Also, audio data is transmitted from the video device 22 on the side where the screen unit 6C is provided to the amplifier 23 by wireless communication such as Bluetooth.

  The audiovisual system shown in FIG. 11 is configured in the same manner as the audiovisual system shown in FIG. 1 except for the configuration described above. Even if two opposing surfaces of the four walls are formed by the inner wall 2 and the outer wall 3 as in the audiovisual system shown in FIG. 11, the same effect as the audiovisual system shown in FIG. 1 can be obtained. . Further, in this case, a sound field with a sense of reality can be obtained by providing a speaker on the back side of the screen.

  FIG. 12 shows another configuration example of the audiovisual system. In this audiovisual system, a large screen screen unit 6C is provided on the inner wall 2 of the audiovisual room 31, and a plurality of projectors 10 are arranged on the back of the large screen screen unit 6C. High resolution video is displayed on a large screen by multi-screen display.

  Here, as shown in FIG. 13, each projector 10 is fixed to the outer wall 3 such that the short side direction is the wall thickness direction. Thereby, even if the projector 10 is fixed to the outer wall 3, the space between the inner wall 2 and the outer wall 3 is not so narrow.

  Further, as shown in FIG. 13, each projector 10 emits image light (emitted light used for displaying an image) in a direction taken on the wall surface. This image light is reflected by the mirror 32 arranged on the optical path, and the image light is irradiated onto the screen unit 6C. In this case, since the optical path of the image light is bent by the mirror 32, the distance between the inner wall 2 and the outer wall 3 can be shortened as shown in FIG.

  Thereby, the mounting direction of the projector 10 is not limited to the upward direction, and may be provided in the downward direction, the horizontal direction, or the like. The dimensions shown in FIGS. 13 and 14 are examples showing the distance between the inner wall 2 and the outer wall 3 necessary for displaying a screen with a certain size.

  The audiovisual system shown in FIG. 12 is configured in the same manner as the audiovisual system shown in FIG. 1 except for the configuration described above. According to the audiovisual system shown in FIG. 12, the same effect as that of the audiovisual system shown in FIG. 1 can be obtained. In this case, as described above, the image light is emitted from the projector 10 in the direction along the outer wall 3, and then reflected by the mirror 32 toward the screen unit 6 </ b> C, and the inner wall 2 and the outer wall 3 are The distance between them can be shortened and displayed on a large screen. In addition, the projector 10 can be stably attached to the strong outer wall 3, and the space between the outer wall 3 and the inner wall 2 is not so narrow, so that the maintainability and the like can be improved.

  15A and 15B show another example of the configuration of the audiovisual system. FIG. 15A shows a perspective view, and FIG. 15B shows a side view. In this audiovisual system, all four walls of the audiovisual room 41 are formed by the inner wall 2 and the outer wall 3, and a space is formed so that humans can enter. The audiovisual room 41 has a double ceiling structure in which the ceiling 2A is arranged with a predetermined interval with respect to the ceiling back 3A, and the floor 2B is arranged with a predetermined interval with respect to the under floor 3B. have.

  A rectangular parallelepiped structure including the inner wall 2, the ceiling 2A, and the floor 2B is supported with a plurality of legs 42 with respect to the lower floor 3B. That is, the floor 2B is supported by the plurality of legs 42 with respect to the floor 3B, the four inner walls 2 are held by the floor 2B, and the ceiling 2A is held by the four inner walls 2 as described above. 2, a rectangular parallelepiped structure composed of the ceiling 2A and the floor 2B is supported by a plurality of legs 42 with respect to the floor 3B.

  In the audiovisual room 41, for example, the screen unit 6C and the speaker unit 6A are provided on the inner wall 2 of one surface, and the speaker unit 6A is provided on the remaining inner wall surface, floor surface, and ceiling surface. Created to allow viewing of video content.

  The audiovisual system shown in FIG. 15 is configured in the same manner as the audiovisual system shown in FIG. 1 except for the configuration described above, and can obtain the same operational effects. Moreover, since it has a double ceiling structure or a double floor structure, it is possible to arrange equipment such as a projector and an amplifier behind the ceiling or under the floor. In addition, a rectangular parallelepiped structure composed of the inner wall 2, the ceiling 2A, and the floor 2B is supported by a plurality of legs 42 with respect to the underfloor 3B, and a space can be easily and reliably provided on all of the walls, ceiling, and floor. In addition, the structure composed of the inner wall, the ceiling and the floor can be completely independent from the outer wall.

  16A and 16B show another configuration example of the audiovisual system. FIG. 16A shows a perspective view, and FIG. 16B shows a side view. In this audiovisual system, all four walls of the audiovisual room 51 are formed by the inner wall 2 and the outer wall 3, and a space is formed so that humans can enter. The audiovisual room 51 has a double ceiling structure in which the ceiling 2A is arranged at a predetermined interval with respect to the ceiling back 3A, and the floor 2B is arranged at a predetermined interval with respect to the under floor 3B. have. The ceiling 2A and the floor 2B are fixed to the outer wall 3, and the inner wall 2 is fixed to the ceiling 2A and the floor 2B.

  In the audiovisual room 51, for example, the screen unit 6C and the speaker unit 6A are provided on the inner wall 2 of one surface, and the speaker unit 6A is provided on the remaining inner wall surface, floor surface, and ceiling surface. Created to allow viewing of video content.

  The audiovisual system shown in FIG. 16 is configured in the same manner as the audiovisual system shown in FIG. 1 except for the configuration described above, and the same operational effects can be obtained. Moreover, since it has a double ceiling structure or a double floor structure, it is possible to arrange equipment such as a projector and an amplifier behind the ceiling or under the floor. Further, the ceiling 2A and the floor 2B are fixed to the outer wall 3, and the inner wall 2 is fixed to the ceiling 2A and the floor 2B. A space can be easily and reliably formed in all of the wall, ceiling, and floor, and the inner wall 2 Can be stably fixed to the outer wall 3 via the ceiling 2A and the floor 2B.

  FIG. 17 shows another configuration example of the audiovisual system. The audiovisual room 61 in this audiovisual system has a double structure on the ceiling, floor, and wall, like the audiovisual rooms 41 and 51 of the audiovisual system shown in FIGS. As shown in FIG. 18, the audiovisual room 61 has a room entrance 62 for entering the inside through the outer wall 3 and the inner wall 2 from the outside. In this case, the user enters the inside of the audiovisual room 61 through a door 63 provided on the inner wall 2 through a tunnel-shaped passage constituting the room entrance 62. A space entrance (door) 64 for entering the space between the inner wall 2 and the outer wall 3 is provided on the side surface of the room entrance 62, that is, the side surface of the tunnel-like passage.

  In addition, as shown in FIG. 17, the audiovisual room 61 is provided with a sound absorbing material by the curtain 65 in the space between the inner wall 2 and the outer wall 3, whereby sound waves generated by the speaker unit 6 </ b> C are reflected by the outer wall 3. Thus, the sound reproduction by the speaker unit 6C is not adversely affected. In such a sound absorbing material, mat-shaped glass wool may be affixed to the inside of the outer wall 3, and various members and arrangement methods can be widely applied.

  The audiovisual system shown in FIG. 17 is configured in the same manner as the audiovisual system shown in FIG. 1 except for the configuration described above, and can obtain the same operational effects. In addition, the space entrance 64 is not provided in the inner wall 2 or the outer wall 3, but the space entrance 64 can be easily provided, and a serviceman or the like can connect the space entrance 64 between the inner wall 2 and the outer wall 3. It can easily enter the space.

  In the above-described configuration example of the audiovisual system, the case where one, two, and four of the four walls are created by the inner wall 2 and the outer wall 3 is described. However, the present invention is not limited to this. The wall, ceiling, and floor constituting the audiovisual room (room) can be made to have a double structure as necessary, and the same effects as in the above configuration example can be obtained.

  (2) Image display device

  FIG. 19 shows a configuration example of the image display system 101 for displaying an image on the inner wall 2 constituting the audiovisual room in the audiovisual system described above. The image display system 101 includes a data processing device 102, an image display device 111, and an illumination 120. The image display device 111 includes a media database 112, a sound field processing unit 113C, an image processing unit 113B, a display information selection unit 113A, an illumination information selection unit 113E, an illumination information generation unit 113D, a background image display unit 115, and a bird's eye image display. A remote commander 116 including a unit 117 and a touch panel 118, an illumination controller 119, and a speaker 122. Here, the sound field processing unit 113C, the image processing unit 113B, the display information selection unit 113A, the illumination information selection unit 113E, and the illumination information generation unit 113D constitute a control unit 113, respectively.

  The data processing device 102 supplies the omnidirectional image data D3 to the image display device 111. The data processing apparatus 102 generates omnidirectional image data D3 from the image data D1 based on the imaging result acquired in advance at the shooting location.

  That is, in this image display system 101, an imaging result is acquired in advance at a predetermined shooting point, for example, by panning an imaging device (camera) by a certain angle, thereby capturing a wide range that is difficult to display on one screen. An imaging result by a wide-angle image is acquired. In this embodiment, by capturing the imaging result while panning the imaging device and making the imaging device make one round, the imaging result of this wide-angle image has an image of the entire periphery that can be overlooked at this imaging point. Applied.

  Here, in this shooting, the rotation axis and the like related to the pan of the imaging device are precisely adjusted so that when the imaging device rotates once, the imaging device returns correctly to the initial position. This also compares the imaging result captured at the starting position in this way with the imaging result captured by rotating once and returning to the initial position, and if the comparison result does not return correctly to the original position. In this case, the photographing is repeated by adjusting the rotation axis again. In this shooting, the exposure is set to be optimal for each imaging result by automatic aperture adjustment.

  In this image display system 101, at the time of acquiring the imaging result of the still image, the sound at the shooting point is recorded by the surround method for a certain time. As the surround system, for example, a 5.1 channel surround system is applied. At this time, this sound is recorded with a fixed direction by a plurality of fixed microphones.

  In the image display system 101, these photographing and sound recording are executed at a plurality of points set in advance, and image data D 1 and sound data D 2 obtained by the photographing and recording are provided to the data processing apparatus 102.

  The data processing device 102 is formed by, for example, a computer capable of image processing, and processes the image based on the image data D1 so that the image display device 111 can easily process the acquired image data D1. All-around image data D3 is generated. Here, as shown in FIG. 20A, the all-around image data D3 is image data relating to a plurality of images (hereinafter referred to as “partial images”) obtained by dividing the periphery of each photographing point by a predetermined angle range. . In the all-around image data D3, this partial image is formed by a set of image files by JPEG (Joint Photographic Coding Experts Group) or the like. Here, FIG. 20B shows an image based on the original image data D1, and in FIG. 20A, a frame indicated by reference numeral FL indicates the size of one screen displayed by the image display system 101.

  As a result, the data processing apparatus 102 joins a plurality of imaging results captured at each imaging point to generate an all-around image, and then cuts out a partial image, thereby generating all-around image data D3. Further, the azimuth information of each partial image is set so as to detect the azimuth of each partial image at the photographing point, and the all-around image data D3 is generated.

  By the way, the image obtained by the imaging device in this way is obtained by imaging light from a subject incident through an aperture represented by a pinhole of a pinhole camera on a flat imaging surface formed by an imaging device. As a result, the figure distortion increases in the peripheral portion of the image. As a result, it is difficult to naturally connect images without having a sense of incongruity in images that are sequentially panned and photographed, and, for example, the roofs of buildings are bent or displaced at the connected portions.

  For this reason, the data processing apparatus 102 removes this image distortion from the image based on the image data D1, and then generates an all-around image and cuts out a partial image. For this reason, as shown in FIG. 21, the data processing device 102 projects an image of the imaging result on the imaging screen 104 by a plane corresponding to the imaging surface, and views the projection image of the imaging screen 104 from the imaging device 105. As described above, a process of re-projecting the image projected on the imaging screen 104 to the virtual imaging screen 106 capable of removing the image distortion is executed by arithmetic processing, thereby removing this kind of image distortion. Here, in order to simplify the processing, the data processing apparatus 102 executes the arithmetic processing by defining the virtual imaging screen 106 with a surface that is curved only in the longitudinal direction (horizontal direction) of the image. Thereby, the virtual imaging screen 106 is formed in a shape in which a cylinder is partially cut by a plane parallel to the axis.

  As a result, the data processing apparatus 102 assumes that the imaging screen 104 and the virtual imaging screen 106 are in contact with each other at the center, for example, as shown in FIG. Define The coordinates of the viewpoint are (0, 0, -a). Here, when 1/2 of the horizontal width of the image on the imaging screen 104 is set to the value 1, when ½ of the angle of view is θ, it is defined by tan θ = 1 / a, and the virtual imaging screen A radius of 106.

  As shown in FIG. 22, the data processing apparatus 102 maps each pixel according to the imaging result on the imaging screen 104, and a straight line connecting each pixel and the point of the viewpoint (0, 0, -a) is a virtual imaging screen. The XY coordinates of the intersection crossing 106 are detected. The data processing device 102 corrects the position of each pixel by replacing the XY coordinate value of each pixel on the imaging screen 104 with the XY coordinate by this intersection, thereby executing the image distortion removal processing.

  In this case, as shown in FIG. 23 in comparison with FIG. 22, this series of processing may be executed by processing using a polar coordinate system instead of processing using a three-dimensional coordinate system.

  The data processing apparatus 102 corrects the image distortion in this way to generate an all-around image (hereinafter referred to as an “all-around image”), cuts out the partial image, and sets the image data of each partial image as data. A plurality of image data files are generated by compression. Further, the image data files of the partial images are collected for each shooting point to generate the all-around image data D3. At this time, the audio data D2 acquired at each photographing point is added to generate the all-around image data D3. In this case, the audio data D2 may be a separate file in association with the all-around image data D3.

  Further, the data processing device 102 detects a vanishing point for each omnidirectional image data D3, and adds the Y coordinate value of this vanishing point to the omnidirectional image data D3. Here, as shown in FIG. 24, vanishing points are straight lines L1, L2,... On the screen plane corresponding to the parallel lines when the parallel lines in the three-dimensional space are projected on the image plane by perspective transformation. , Ln,... Converge. In other words, the vanishing point is an “infinitely far point” on a planar image on which a space having depth is actually projected, and is a line parallel to the depth direction (for example, a ridgeline in the case of a room image). ) And the extension of the surface extending in the depth direction (for example, in the case of a room image, each plane corresponding to the floor, wall, and ceiling extending in the depth direction) converges to infinity. It is recognized. The straight line drawn in the horizontal direction in the image from the vanishing point is the horizon.

  The data processing apparatus 102 performs image processing on each image acquired at one shooting point to detect parallel lines in the depth direction, detects a Y coordinate value of an intersection of the parallel lines, and sets it as a vanishing point coordinate value. . The image processing related to the detection of the parallel lines is executed by detecting feature points and statistically processing them. Further, the data processing apparatus 102 detects the Y coordinate value of the vanishing point for each combination of parallel lines and statistically processes it, and detects one Y coordinate value for one image. Further, by further statistically processing the Y coordinate values detected in a plurality of images used for creating the all-around image, a Y-coordinate value relating to the vanishing point is detected for each all-around image, and each Y coordinate value is It is added to the entire surrounding image data D3. Accordingly, the Y coordinate value of the contact point between the imaging screen 104 and the virtual imaging screen 106 described above with reference to FIG. 21 is set to the Y coordinate value of the vanishing point detected in this way. The statistical processing related to the series of processing is, for example, averaging processing.

  In addition, when the data processing apparatus 102 generates the omnidirectional image data D3 from the partial images in this way, the data processing device 102 corrects the gradation in each image so as to correct the difference in aperture between the plurality of images, thereby A partial image based on the all-around image data D3 is formed from an image photographed under a constant aperture condition. Specifically, the data processing apparatus 102 detects the maximum value and the minimum value of the pixel values in all the images that are used for generating the all-around image data, and the maximum value and the minimum value of these pixel values do not exceed the dynamic range. In this way, the gradation of each image is corrected so that the luminance level does not change abruptly at the joints between successive images, thereby generating all-around image data D3 in the case of shooting under a constant aperture condition.

  In addition, when a sufficient dynamic range is ensured in the imaging result by the imaging device, instead of ex-post tone correction of a plurality of images taken by such automatic aperture control, The imaging result used for generation may be imaged with a constant aperture.

  The image display system 101 supplies the omnidirectional image data D3 generated in this way to the image display device 111 together with the corresponding bird's-eye image data D4, and records it in the media database 112.

  Here, the bird's-eye view image data D4 is image data of a bird's-eye view image. The bird's-eye view image is an image of a shooting point (location) related to acquisition of the all-around image data D3 as seen from above, and an image obtained by actual shooting, an image by illustration, an image by computer graphics, or the like is applied. In addition, the bird's-eye view image is created so as to be able to confirm a shooting point related to acquisition of the all-around image data D3 by using a mark or the like and to be able to search the corresponding all-around image data D3. Specifically, for example, the bird's-eye view image is created by placing a mark at a shooting point related to acquisition of the all-around image data D3, and creating link information specifying the corresponding all-around image data D3 for each mark. Is done. Instead of this, similar link information is set on the omnidirectional image data D3 side so that the shooting point can be marked on the bird's-eye view image. The all-around image data D3 and the bird's-eye image data D4 are read from various recording media such as an optical disk, a magnetic disk, and a memory card and supplied to the image display apparatus 111, or downloaded by way of a network such as the Internet. 111.

  The media database 112 is formed by, for example, a large-capacity hard disk device, and records and holds these all-around image data D3 and bird's-eye image data D4 under the control of the control unit 113, and holds these all-around image data D3, The bird's-eye image data D4 is reproduced and output. Further, the illumination control information D5 related to the illumination control output from the control unit 113 is recorded and held, and the held illumination control information D5 is reproduced and output to the control unit 113.

  The background image display unit 115 is a display device with a large screen, and displays an image based on the video data DV1 output from the control unit 113. The background screen display unit 115 includes, for example, the inner wall 2 (screen unit 6C) and the projector 10 that constitute the audiovisual system described above. Here, the video data DV1 is video data generated from the omnidirectional image data D3, whereby the image display device 111 causes the background image display unit 115 to display a desired shooting point in a desired direction. Display a landscape image.

  The remote commander 116 displays a bird's-eye view image and notifies the control unit 113 of various user operations related to the bird's-eye view image and the like by wireless communication or the like. That is, the bird's-eye image display unit 117 is formed by a liquid crystal display panel arranged in the remote commander 116 and displays a bird's-eye image related to a shooting point of an image displayed on the background image display unit 115. The touch panel 118 notifies the control unit 113 of various user operations by the bird's-eye view image display unit 117. The remote commander 116 is formed on the surface of a table disposed in the audiovisual room, for example, as will be described later.

  The illumination controller 119 is a control unit that controls the illumination 120 of the room (audio-visual room) in which the image display device 111 is disposed, and varies the brightness and color temperature of the illumination 120 in accordance with instructions from the control unit 113.

  The control unit 113 is a computer that executes a series of processes related to the image display device 111, and controls the operation of each unit by executing a program recorded in a memory (not shown). The control unit 113 configures various functional blocks by executing this program. In this embodiment, the program related to the processing of the control unit 113 is provided by being preinstalled in the image display device 111. However, instead of such preinstallation, an optical disk, a magnetic disk, You may make it provide by recording on various recording media, such as a memory card, and you may make it provide by downloading via networks, such as the internet.

  Here, the display information selection unit 113A selects an image to be displayed on the bird's-eye image display unit 117 and the background image display unit 115 according to the notification from the remote commander 116, and the image data D3 and D4 are displayed in the media database 112 according to the selection result. And instructing the output of the audio data D2. The image processing unit 113B processes the image data D3 and D4 output from the media database 112 to generate video data DV2 and DV1 to be used for driving the bird's-eye image display unit 117 and the background image display unit 115, and the bird's-eye image. The data is output to the display unit 117 and the background image display unit 115.

  The sound field processing unit 113C processes the audio data D2 output from the media database 112 and drives the speaker 122 that is used to generate a surround sound field. The speaker 122 is configured by the inner wall 2 (speaker unit 6A) in the above-described audiovisual system.

  At this time, the sound field processing unit 113C changes the localization position of the sound image based on the audio data D2 according to the image displayed on the background image display unit 115, and thereby the orientation of the image displayed on the background image display unit 115. The speaker 122 is driven by the localization of the sound image corresponding to. Note that such a change in the localization position of the sound image is obtained by, for example, changing the localization position largely by switching the channel of each audio signal that drives the speaker 122, and then performing addition / subtraction processing between these channels to finely determine the localization position. It is executed by changing.

  The illumination information generation unit 113D processes the image data D3 output from the media database 112, generates illumination control information D5 necessary for controlling the illumination 120, and records it in the media database 112. The illumination information selection unit 113E generates illumination corresponding to the image displayed on the background image display unit 115 from the illumination control information D5 generated by the illumination information generation unit 113D and recorded in the media database 112 in accordance with the notification from the remote commander 116. The control information D5 is selectively acquired, and the operation of the illumination controller 119 is controlled by the acquired illumination control information D5.

  FIG. 25 is a flowchart illustrating a processing procedure related to basic processing of the control unit 113. When the power of the image display device 111 is turned on, the control unit 113 starts this processing procedure and proceeds from step SP1 to step SP2, where a menu screen for selecting a bird's-eye view image from recording in the media database 112 is displayed. To do. Furthermore, the selection of a bird's-eye image to be displayed is accepted by the user's selection on this menu screen, and the bird's-eye image data D4 of this bird's-eye image is acquired from the media database 112.

  As shown in FIG. 26 (A1), the control unit 113 displays the bird's-eye image 131 on the bird's-eye image display unit 117 based on the acquired bird's-eye image data D4. At this time, in this bird's-eye view image 131, the photographing points of the all-around image data D3 are indicated by marks M1 to M5. In the bird's-eye view image shown in FIG. 26, the marks M1 to M5 are represented by circles, and the characters “A” and “B” indicate the buildings that actually exist at the locations where the respective characters are displayed. To do.

  Subsequently, the control unit 113 proceeds to step SP3, where the operation of the touch panel 118 by the user is detected, and any one of the shooting points represented by these marks M1 to M5 is selected by the user's selection of the marks M1 to M5. Whether to display an image taken at the shooting point on the background image display unit 115 is accepted. Further, by accepting the display direction, it is accepted whether the image obtained by viewing which direction is displayed on the background image display unit 115 at the selected photographing point. As shown in FIG. 26A1, when the mark is selected by the user, the control unit 113 displays a mark MK indicating the direction and the selection on the selected mark, and the touch panel by the user. The direction of the mark MK is changed by the operation of 118, thereby accepting the input of the photographing point and direction. As shown in FIG. 26 (A1), in this embodiment, the design of the camera is applied to this orientation and the mark MK indicating that it has been selected.

  When the input of the shooting point and the orientation is received in this way, the control unit 113 proceeds to step SP4 and acquires the corresponding all-around image data D3 from the media database 112 by the link information set for the selected mark M1. .

  Further, as shown in FIG. 27, in the subsequent step SP5, a partial image corresponding to the orientation set in step SP3 is selected from the plurality of partial images based on the acquired all-around image data D3 (FIG. 27A). Then, the images are connected (FIG. 27 (B1)), and an image for one screen is cut out from the connected images according to the direction set in step SP3 and displayed on the background image display unit 115 (FIG. 27 (C1)). ). Note that the distortion correction processing described above with reference to FIGS. 21 to 23 may be executed together with the processing in step SP5 instead of the creation of the omnidirectional image data D3. Similarly, the above-described processing for correcting the gradation as if it was photographed under a constant aperture condition may be executed together with the processing in this step SP5 instead of when creating the all-around image data D3. .

  In this way, when the image for one screen is cut out and displayed on the background image display unit 115, the control unit 113, as shown in FIG. Compare the height of the eyes when the user views the image in the set background image display unit 115 with the Y coordinate value of the vanishing point set in the all-around image data D3, as shown in FIG. The display position is corrected and displayed so that the position of the vanishing point of the image displayed on the background image display unit 115 is the height of the eyes of the user. As a result, the control unit 113 displays the image in this way so that the user can appreciate the scenery and the like without feeling uncomfortable.

  When the image based on the omnidirectional image data D3 is displayed in this way, the control unit 113 proceeds to step SP6, starts reproducing the audio data related to the omnidirectional image data D3, and drives the speaker 122. At this time, the control unit 113 changes the localization position of the sound image so that the sound image is localized at a position corresponding to the image displayed by the background image display unit 115 by the processing by the functional block of the sound field processing unit 113C described above. Set the sound field.

  Thus, the control unit 113 displays the scenery that the user sees on the background image display unit 115 when the user reaches the location indicated by the user in the bird's-eye view image in the direction indicated by the user. Various sounds heard by the user are reproduced by the speaker 122, and in the example shown in FIG. 26 (A1), an image as shown in FIG. 26 (A2) is displayed on the background image display unit 115.

  When the image is displayed and the reproduction of the audio data is started in this manner, the control unit 113 proceeds to step SP7 and determines whether or not the user has instructed to change the direction by operating the touch panel 118. If a negative result is obtained here, the control unit 113 proceeds to step SP8 and determines whether or not the user has selected another mark M1 to M4 by operating the touch panel 118, and the user has instructed movement. If a negative result is obtained here, the control unit 113 returns to step SP7.

  As a result, the control unit 113 waits for a direction change and movement instruction by the user, and when the direction change is instructed by the user, an affirmative result is obtained in step SP7, whereby the direction is changed from step SP7 to step SP10. After the detection, the process proceeds to step SP11 (FIG. 29). Here, as indicated by reference numeral B in FIG. 27A, the control unit 113 gradually scrolls the image displayed on the background image display unit 115 in the direction instructed to be changed by the user. In this case, as shown in FIGS. 27 (B2) and (C2) by comparison with FIGS. 27 (B1) and (C1), the regions to be cut out by joining the partial images as needed are gradually changed. As a result, the scrolling process is executed on the all-around image based on the all-around image data D3.

  In the subsequent step SP12, the processing by the functional block of the sound field processing unit 113C is switched, and the position of the sound image formed by the speaker 122 is gradually moved so as to be interlocked with the scrolling in the background image display unit 115. The process returns to step SP7. Further, at this time, as shown in FIG. 26 (B1), the mark MK indicating the direction is changed so as to be interlocked with the scrolling in the background image display unit 115 as compared with FIG. 26 (A1). Thus, in the example of FIG. 26 (B1), the background image display unit 115 displays a distant view of the building with the symbol B shown in FIG. 26 (B2).

  On the other hand, when movement is instructed by the user, an affirmative result is obtained in step SP8, whereby the control unit 113 proceeds from step SP8 to step SP13 (FIG. 30). Here, the control unit 113 detects the shooting point (position) of the movement destination using the marks M1 to M5 instructed by the user, and in the subsequent step SP14, this movement destination is the same as described above for the steps SP11 and SP12. The display of the background image display unit 115 and the sound field setting are changed in the direction of the shooting point.

  In subsequent step SP15, the entire surrounding image data D3 is acquired from the media database 112 for this destination in the same manner as described above for step SP4. In the subsequent step SP16, in the same manner as described above for step SP5, an image related to display at the destination is generated from the all-around image data D3 acquired from the media database 112.

  In subsequent step SP17, the control unit 113 gradually zooms in on the image being displayed on the background image display unit 115, and switches the display so far to the image generated in step SP15 by crossfade. It should be noted that transitions by various effects such as mosaics can be applied as necessary in place of such transitions by zoom-in and cross-fade. Further, in the following step SP18, in conjunction with such display switching, the sound from the speaker 122 is added to the sound field by the audio data D2 corresponding to the destination by cross-fading or fading in once after fading out. The field is switched and the process returns to step SP7.

  Thereby, as shown in FIG. 26 (B1), the control unit 113 is arranged in the northwest in a state where the user designates a shooting point by the mark M1 and the background image display unit 115 displays the landscape on the north side. When the movement to the mark M2 is instructed, as shown in FIG. 26 (A1), after changing the display and sound field of the background image display unit 115 so as to change the direction to the mark M2, FIG. 26 (C1) As shown by (C2), the display and sound field of the background image display unit 115 are changed so as to move toward the mark M2, and the image and sound field according to the position of the mark M2 are provided to the user. In addition, the control unit 113 changes the position and orientation of the mark MK in the bird's-eye view image display so as to be interlocked with the switching of these displays.

  Thus, the image display device 111 provides images and sounds so that a walk-through can be experienced using an image in a live-action image.

  In addition, when moving a place in the real world, for example, a target may be captured and moved in a direction other than the front. Thus, when a movement is instructed by the user, the process of turning to the destination may be omitted in the series of processes for switching the display of the background image display unit 115 so as to move in the direction of the destination. Good. In this case, the image provided after the movement may be an image according to the direction immediately before the movement, or may be an image according to the direction of the target instructed by the user. In such a case, the scrolling process in the moving direction may be performed together with the zooming process when the display is switched. Further, regarding the transition of the display at the time of switching the display, the display may be switched once to the bird's-eye view image.

  FIG. 31 is a schematic diagram for explaining the display on the background image display unit 115 when the user instructs transition on the background image. When the user instructs a transition on the background image by operating the touch panel 118, the control unit 113 displays the marks N1 to N4 corresponding to the marks M1 to M5 displayed as the bird's-eye image for a certain period of time on the background image display unit 115. Display on the image. The marks N1 to N4 related to the display of the background image display unit 115 add the display position information of the marks N1 to N4 to the omnidirectional image data D3 by the data processing device 102 when the omnidirectional image data D3 is generated. It is displayed based on this position information. In this case, based on the marks M1 to M5 set in the corresponding bird's-eye view image, the control unit 113 calculates and displays the display positions of the marks N1 to N4 on the image based on the all-around image data D3. May be.

  When the control unit 113 displays the marks N1 to N4 in this way, the bird's-eye view image display unit 117 displays a menu for switching the focus of these marks N1 to N4, a determination menu, and a menu for instructing scrolling to the left and right. When the user selects any of the marks N1 to N4 by operating these menus, the entire surrounding image data D3 corresponding to the marks N1 to N4 is obtained in the same manner as described above with reference to FIGS. 25, 29, and 30. The display is acquired and the display of the background image display unit 115 is switched.

  In the switching of the display, the display up to that point is zoomed in while scrolling the display so that the display position of the mark selected by the user is in the center of the screen, and the mark selected by the user by the crossfade is related. Switch the display to an image. In addition, in the image after the display switching at this time, the image is displayed according to the direction of viewing the shooting point related to the mark selected by the user from the shooting point related to the original image.

  On the other hand, when the user selects the scroll menu, the control unit 113 is instructed by the user to display the area to be cut out from the entire surrounding image so as to indicate the change of the area to be displayed by the arrows C1 and C2. By gradually changing the direction, the display on the background image display unit 115 is scrolled. In these processes, the control unit 113 also changes the sound field so as to correspond to the display on the background image display unit 115, thereby ensuring a sense of reality.

  As a result, the control unit 113 can select the destination on the displayed image without having to grasp the current position from the bird's eye view, and can change the direction to experience the walkthrough. In addition, the immersive feeling can be further improved.

  In the above-described processing of FIGS. 25, 29, and 30, it has been described that only the bird's-eye image is displayed on the bird's-eye image display unit 117, but the state where the bird's-eye image is displayed (hereinafter, “bird's-eye mode”). ”), When a shooting location to be displayed is designated by a user operation, the same image as the image displayed on the background image display unit 115 is displayed (hereinafter referred to as“ immersion mode ”). The display state of the bird's-eye image display unit 117 can be freely changed to the above-described bird's-eye mode and the immersive mode so that when the user performs a return operation in this immersive mode, the mode is shifted again to the bird's-eye mode. It may be switchable.

  FIG. 32A shows a display example of the bird's-eye image display unit 117 in the bird's-eye mode, and FIG. 32B shows a display example of the bird's-eye image display unit 117 in the immersive mode. In the bird's-eye view mode shown in FIG. 32A, a bird's-eye view image is displayed on the bird's-eye view image display unit 117. Rectangular display areas D1 to D5 are provided corresponding to the shooting points M1 to M5 of the all-around image in the bird's-eye view image. In FIG. 32A, these display area display areas D1 to D5 are blank, but in actuality, images in the current direction at the respective points are displayed in the respective display areas.

  In this bird's-eye view mode, the bird's-eye view image display unit 117 displays direction selection buttons BL and BR superimposed on the bird's-eye view image. The user can arbitrarily select the direction by operating the selection buttons BL and BR on the touch panel 118. In accordance with the selection of the orientation, the orientation of the bird's eye image displayed on the bird's eye image display unit 117 is also changed, and the images displayed in the display areas D1 to D5 are also changed correspondingly.

  Here, the change of the bird's-eye view image can be achieved by making the bird's-eye view image data D4 read from the media database 112 according to the direction as described above. In this case, the media database 112 stores bird's-eye image data D4 corresponding to each direction. Note that the change of the direction of the bird's-eye view image can also be achieved by coordinate conversion processing. Moreover, the change of the display image of each display area can be achieved by changing the part cut out from the omnidirectional image of each point according to direction as mentioned above.

  In this bird's-eye view mode, the user can select a shooting point by touching any one of the shooting points M1 to M5 with the touch panel 118. Thereby, the all-around image data D3 of the selected point is read from the media database 112, and the image of the selected shooting point is displayed on the background image display unit 115 in the direction selected in the bird's eye mode. The At this time, as shown in FIG. 32B, the display state of the bird's-eye image display unit 117 is in the immersive mode, and the same image as the image displayed on the background image display unit 115 is displayed.

  In this immersion mode, the bird's-eye view image display unit 117 displays direction selection buttons BL and BR superimposed on the image. The user can arbitrarily select the direction by operating the selection buttons BL and BR on the touch panel 118. In accordance with the selection of the orientation, the image displayed on the background image display unit 115 is changed, and the image displayed on the bird's eye image display unit 117 is also changed correspondingly. This change of the image can be achieved by changing the portion cut out from the entire surrounding image of the selected point as described above according to the direction.

  In this immersive mode, the bird's-eye view image display unit 117 displays marks N2 and N3 indicating other shooting points that are superimposed on the image and present in the image. The user can directly select another shooting point by touching the mark of the shooting point on the touch panel 118. As described above, when another shooting point is selected in the immersion mode, the entire surrounding image data D3 of the shooting point is read from the media database 112, and the background image display unit 115 displays the image of the shooting point. Is displayed in the orientation at the time of selection. In this case, the image displayed on the bird's-eye view image display unit 117 is also changed to that at the shooting point.

  In this immersion mode, the bird's eye image display unit 117 displays a return button RB superimposed on the image. By operating this return button RB on the touch panel 118, the user shifts from the immersive mode to the bird's-eye view mode, and returns to the state where the bird's-eye image is displayed on the bird's-eye image display unit 117 as shown in FIG. 32A.

  FIG. 33 shows a transition diagram of the display state of the above-described bird's-eye image display unit 117. In the bird's-eye view mode, the direction can be changed without changing the mode. And if there is point selection in this bird's-eye view mode, it will change to immersion mode. In the immersive mode, the orientation can be changed without changing the mode. In this immersion mode, it is possible to perform point selection without changing the mode. Then, when the return operation is performed in the immersion mode, the mode is changed to the bird's-eye view mode.

  By the way, when displaying a famous sightseeing spot, a theme park, etc. with a bird's-eye view image, it is predicted that marks will be densely arranged, and depending on the bird's-eye view image, an image of a desired place may not be displayed easily. . In this case, even when the background image display unit 115 is moved by the mark on the display screen, the mark is displayed densely, and in this case as well, there is a risk that the image of the desired place cannot be easily displayed. There is.

  Therefore, when the user instructs parallel display of all-around images by operating the touch panel 118, the control unit 113 causes the background image display unit 115 to display all of the two shooting points instructed by the user as shown in FIG. All the surrounding images D3A are displayed side by side. In this case, it is only necessary to display the entire surrounding images related to the two shooting points side by side, and display only a part of the entire surrounding image within the range that can be displayed by the background image display unit 115. May be. Two images may be displayed separately on the background image display unit 115 and the bird's-eye image display unit 117, or may be displayed together on the bird's-eye image display unit 117. Furthermore, a third display device may be provided for display. The control unit 113 scrolls this display according to an instruction from the user and accepts an input of a movement destination.

  That is, FIG. 35 is a flowchart showing a processing procedure of the control unit 113 related to the parallel display of the all-around image. When starting this processing procedure, the control unit 113 moves from step SP21 to step SP22, displays the bird's-eye image by the bird's-eye image display unit 117, and selects the marks M1 to M5 displayed in the bird's-eye image in the subsequent step SP23. The photographing point detected by the touch panel 118 and thereby instructed by the user is received. At this time, the control unit 113 accepts the selection of two marks M1 to M5, and thereby accepts the selection of the shooting point for the two images used for parallel display.

  Subsequently, the control unit 113 proceeds to step SP24, reads out the two omnidirectional image data D3 relating to the shooting point acquired in step SP23 from the media database 112, and the omnidirectional image by the two read omnidirectional image data D3. D3A is displayed side by side as shown in FIG.

  Subsequently, the control unit 113 proceeds to step SP25 to determine whether or not the user has instructed to scroll the entire surrounding image displayed on the upper side by operating the touch panel 118. If a positive result is obtained here, step SP26 is obtained. Then, as indicated by an arrow D in FIG. 34, the all-around image D3A displayed on the upper side is scrolled, and the process returns to step SP25. On the other hand, if a negative result is obtained in step SP25, the control unit 113 proceeds from step SP25 to step SP27, and has the user instructed to scroll the entire surrounding image D3A displayed on the lower side by operating the touch panel 118? Judge whether or not. If a positive result is obtained here, the control unit 113 proceeds from step SP27 to step SP28, scrolls the entire surrounding image displayed on the lower side as indicated by an arrow E in FIG. 34, and returns to step SP25.

  On the other hand, when a negative result is obtained in step SP27, the control unit 113 proceeds to step SP29 and determines whether or not the user has operated the determination menu by operating the touch panel 118. If a negative result is obtained here, the control unit 113 returns from step SP29 to step SP25. Thus, the control unit 113 scrolls the all-around image D3A displayed up and down in response to a user operation.

  On the other hand, if a positive result is obtained in step SP29, the control unit 113 proceeds to step SP30. Here, the control unit 113 detects which direction is the center position in the horizontal direction of the omnidirectional image D3A displayed above and below among a plurality of shooting points. Further, as shown on the bird's-eye view image 131 by FIG. 36, based on this detection result, the intersection P of the straight lines L1 and L2 extending in the direction detected from each imaging point is detected, and the nearest imaging point ( In this case, the photographing point by the mark M3) is detected. As a result, the control unit 113 detects the destination desired by the user, and in the subsequent step SP31, acquires all-around image data D3 related to the destination and displays it on the background image display unit 115, and then proceeds to step SP32. This processing procedure is completed.

  As a result, the control unit 113 scrolls the display of the two omnidirectional images so that the movement destination is in the center of the screen in the full-screen display of the omnidirectional images, and accepts the input of the movement destination by these two omnidirectional images. The display is switched to improve usability.

  In addition, instead of setting the destination direction by such scrolling, the setting of the destination direction may be accepted by displaying and moving the cursor on the entire surrounding image. In such parallel display of all-around images, when the bird's-eye view image display unit 117 displays the images, the setting of the movement destination direction may be received by operating the touch panel 118.

  In parallel with the processing related to the series of image display in the background image display unit 115, the control unit 13 controls the illumination 120 according to the image displayed on the background image display unit 115 under the control of the illumination controller 119. Various changes.

  For this reason, in the series of processes described above, the control unit 113 first acquires the omnidirectional image data D3, generates illumination control information D5 for the omnidirectional image data D3, and records it in the media database 112.

  That is, the control unit 113 calculates an average value of pixel values constituting the partial image for each partial image constituting the omnidirectional image data D3, thereby detecting the average luminance level of each partial image. Here, as shown in FIG. 37, the partial image is obtained by sequentially panning the image pickup device at one shooting point, so that an image pickup result with a high luminance level is obtained in the shooting with the sun facing behind. On the other hand, when shooting in the direction of the sun, imaging results with a low luminance level can be obtained due to so-called backlight.

  As a result, as shown in FIG. 38, the average luminance level detected in this way changes in a sinusoidal shape according to the position of the partial image with respect to the sun direction. The control unit 113 generates the illumination control information D5 so as to change the illumination 120 according to the average luminance level of each part of the omnidirectional image detected in this way.

  Specifically, in this embodiment, the illumination control information D5 is generated for each partial image so that the brightness of the illumination 120 is proportional to the average luminance level detected in each partial image. In addition, illumination control information D5 is generated for each partial image so that the color temperature of the illumination is proportional to the average luminance level detected in each partial image. Accordingly, the brighter the image displayed on the background image display unit 115, the brighter the illumination 120 is set, and the apparent contrast of the image displayed on the background image display unit 115 is increased. That is, in this case, a portion with a high luminance level appears brighter, thereby increasing the apparent contrast. In addition, the color temperature of the portion with a high luminance level is increased, so that an image with a higher luminance level appears to be an image of sunlight, and this also increases the apparent contrast.

  As shown in FIG. 39, the control unit 113 records and holds the illumination control information D5 generated in this way in the media database 112 together with an image ID that is an identification code for identifying the corresponding partial image. Further, the illumination control information D5 recorded in the media database 112 is searched according to the part of the entire surrounding image displayed on the background image display unit 115, and the corresponding illumination control information D5 is detected, and the detected illumination control information D5 is detected. The brightness and color temperature of the illumination 120 are controlled by the illumination controller 119.

  FIG. 40 is a flowchart showing a processing procedure of the control unit 113 related to the control of the illumination 120. When starting this processing procedure, the control unit 113 proceeds from step SP41 to step SP42 and acquires an all-around image. In the subsequent step SP43, the illumination control information D5 is generated from the acquired all-around image as described above, and recorded and held in the media database 112.

  In subsequent step SP44, when an image to be displayed on the background image display unit 115 is selected, in subsequent step SP45, the illumination control information D5 corresponding to the image to be displayed is obtained from the media database 112, and subsequent step SP46. The background image display unit 115 displays the image selected in step SP44. In the following step SP47, the illumination 120 is controlled by the illumination control information D5 acquired in step SP45. Further, when the user gives an instruction to change the image displayed on the background image display unit 115 in subsequent step SP48, the process returns to step SP44.

  In addition, when sufficient processing capability can be ensured in the control unit 113, the illumination control information may be generated in real time instead of the generation of the advance illumination control information. On the contrary, it may be created when the omnidirectional image data D3 is generated and provided together with the omnidirectional image data D3.

  According to the above configuration, the user can operate the bird's-eye view image to switch the wide-angle image display at different shooting points, and by scrolling the image display, the display of the image by live-action is higher than the conventional one. A feeling can be secured.

  In addition, the audio data corresponding to the wide-angle image is used to switch the audio data to be provided to the user in conjunction with the switching of the wide-angle image display and the scrolling, and the position of the sound image is changed to further enhance the presence of the sound. It can be improved, and an even higher immersion feeling can be obtained.

  In addition, the user-friendliness can be improved by switching the wide-angle image display and scrolling the image display even by the user input on the wide-angle display image.

  Further, user convenience can also be improved by displaying the wide-angle images side by side and switching the display to an image related to the shooting location in the direction according to the instruction from the user.

  In addition, by changing the lighting in the room in conjunction with the switching of the display on the background image display unit and the scrolling of the wide-angle image, the atmosphere can be changed according to the image, and the contrast can be further increased to further increase the immersion. A feeling can be improved.

  Specifically, the illumination can be varied by changing either or both of brightness and color temperature, and by increasing the brightness and increasing the color temperature according to the luminance level, Immersion feeling can be improved.

  In contrast to the above-described illumination control, the illumination control information D5 is generated for each partial image so that the brightness and color temperature of the illumination are inversely proportional to the average luminance level detected in each partial image. You may do it.

  That is, the illumination 120 is set to be darker as the image displayed on the background image display unit 115 becomes brighter, and the apparent contrast of the image displayed on the background image display unit 115 is increased. That is, in this case, the portion with a low luminance level appears darker, thereby increasing the apparent contrast. In addition, the color temperature of the portion with a low luminance level is increased, so that an image with a lower luminance level appears to be an image of sunlight, thereby increasing the apparent contrast.

  Contrast can also be increased by changing the brightness and color temperature related to the illumination by reducing the brightness and lowering the color temperature according to the brightness level of the image displayed on the background image display unit. , The feeling of immersion can be further improved.

  Also, according to the image to be displayed, proportional control that makes the brightness and color temperature proportional to the average luminance level detected in each partial image, and conversely, the average luminance level detected in each partial image You may make it switch to the inverse proportional control which makes the brightness of illumination and color temperature inversely proportional. Thereby, the illumination control is adaptively switched according to the image to be displayed. Specifically, for example, lighting is controlled by proportional control when displaying a seaside landscape, and lighting is controlled by inverse proportional control when displaying an image taken indoors. Thus, even if the control related to the illumination is switched according to the image to be displayed, the feeling of immersion can be further improved.

  Further, at the time of capturing an image for generation of all-around image data, the sun direction may be recorded and held, and information specifying this sun direction may be added to the all-around image data D3. The image display device 111 varies the brightness and color temperature of the illumination according to the direction of the sun of the image displayed on the background image display unit 115 based on the information specifying the direction of the sun. Thus, the immersion feeling can also be improved by varying the brightness and color temperature of the illumination displayed in the background image display unit 115 in accordance with the direction of the sun at the time of shooting.

  Further, time information at the time of photographing that specifies the photographing time may be added to the all-around image data D3, and illumination may be controlled based on the time information at the time of photographing. Specifically, when the shooting time is evening, the brightness of the illumination is reduced and the color temperature is lowered. When the shooting time is night, the brightness of the illumination is further reduced and the color temperature is lowered. When the shooting time is daytime, the illumination brightness is increased and the color temperature is increased. Thus, even if the illumination is varied according to the photographing time of the image displayed on the background image display unit 115, the immersive feeling can be improved.

  FIG. 41 is a block diagram illustrating a configuration of another image display system 141. In this image display system 141, all-around image data and bird's-eye image data are held in media databases 144A to 144C which are servers on the network. As a result, in the image display system 141, the image display device 142 accesses the media databases 144A to 144C via the broadband network 143 to acquire the all-around image data and the bird's-eye image data. The configuration is the same as that of the display device 111 (see FIG. 19). As described above, even when the recording / reproducing unit related to the image data is arranged on the network, the same effect as the above-described embodiment can be obtained.

  In the above-described embodiment, the case where the partial image is generated from the entire surrounding image and used as the entire image data has been described. The entire image data may be configured, or the entire surrounding image data may be configured by one entire surrounding image.

  In the above-described embodiment, the case where an image based on a still image is displayed has been described. However, the present invention is not limited to this and can be widely applied to the case where an image based on a moving image is displayed. In this case, it is conceivable to connect to a server that supplies all-around image data through a network as described above with reference to FIG. Further, the server provides a plurality of imaging results for generation of the above-described omnidirectional image data by moving images, and the image display device side provides various images by combining the imaging results of the plurality of moving images. It is possible to display a moving image on the background image display unit by changing the direction.

  In addition, a dedicated imaging mechanism that captures a moving image from the entire surroundings is used to obtain and provide a moving image, and the orientation of the moving image is changed in various ways by image processing of the moving image. Video can also be displayed. Note that such a dedicated imaging mechanism for capturing a moving image from the entire periphery includes, for example, a method using a curved mirror, a case using a pyramid mirror, and the like, in this case, the image display apparatus transmits in accordance with the imaging mechanism. It is necessary to convert the obtained imaging result into an image.

  Moreover, in the above-described embodiment, the case where the omnidirectional image is applied as a wide-angle image that is difficult to display on one screen has been described. The present invention can be widely applied to display of images having various spreads as required.

  In the above-described embodiment, the case where the present invention is applied to the display of a wide-angle image having a spread in the horizontal direction has been described. However, the present invention is not limited to this, and instead of or in addition to this. The present invention can also be widely applied to display of wide-angle images having a spread in the vertical direction.

  In the above-described embodiment, the case of controlling the brightness and color temperature of illumination has been described. However, the present invention is not limited to this, and the distribution of illumination may be controlled. That is, for example, when displaying an image by backlighting, by making the side displaying the image brighter and darkening the opposite side, the direction of the sun can be sensed by the distribution of illumination, making it more realistic A feeling can be improved. On the other hand, when displaying images taken with the sun on the back, it is possible to sense the direction of the sun due to the distribution of illumination, even by making the background side brighter, further improving the sense of reality. Can do.

  In the above-described embodiment, the case where the immersive feeling is increased by sound and illumination has been described. However, the present invention is not limited to this, and the immersive feeling is further increased by controlling the wind, temperature, and the like. Also good.

  (3) Arrangement example of projector

  As described above, a projector is disposed in an audiovisual room (room) having a space between the ceiling and the back of the ceiling, a space between the floor and the floor, or a space between the inner wall and the outer wall. Various examples will be described.

  FIG. 42 shows the inside of an audiovisual room (room) 401 surrounded by the inner wall 2, the ceiling 2A, and the floor 2B. In the audiovisual room 401, projection windows 404 as openings are provided at a plurality of locations on the ceiling 2A and the floor 2B, respectively, at two locations in the illustrated example. Here, the projection window 404 is formed on a screen or the like provided in the room, a space between the ceiling 2A and the ceiling back 3A (see FIGS. 15 and 16), a floor 2B and an underfloor 3B (see FIGS. 15 and 16). ) So that image light can be emitted from a projector 402 (not shown in FIG. 42) disposed in a space between the transparent plate material and a transparent plate material such as glass. . The projection window 404 is provided with a lid (not shown) when it is simply formed by an opening, and is covered with this lid when not in use.

  The projector 402 arranged between the ceiling 2A and the ceiling back 3A selectively moves to any position of the projection window 404 formed on the ceiling 2A along the surface of the ceiling 2A, and the projection The image light is irradiated to a predetermined position in the audiovisual room 401 through the window 404, and an image is displayed at the predetermined position. Similarly, the projector 4021 arranged between the floor 2B and the underfloor 3B selectively moves to any position of the projection window 404 formed on the floor 2B along the surface of the floor 2B. Video light is irradiated to a predetermined position in the audiovisual room 401 through the projection window 404, and an image is displayed at the predetermined position. Thus, by adopting a configuration in which image light is irradiated from the projector 402 to a predetermined position in the room through the projection window 404, the projector 402 is not taken out into the room even when the projector 402 is used. Does not disturb the indoor atmosphere.

  For example, a guide member is disposed between the two projection windows 404 so that the projector 402 can selectively move to the positions of the two projection windows 404. In this case, the guide member guides the projector 402 and is also used as a power supply line that supplies power to the projector 402. As described above, the projector 402 is selectively moved to the positions of the two projection windows 404 while being guided by the guide member, but the position detection mechanism is installed so as to correctly move to the position corresponding to the projection window 404. Have. This position detection mechanism detects a position detection member provided at the position of each projection window 404, thereby detecting whether or not the position is present.

  For example, when the position detection mechanism is an optical method, a reflection sheet or the like as a position detection member is attached to the back surface of the ceiling 2A or a guide member. For example, when the position detection mechanism uses an IC tag, the IC tag is attached to the back surface of the ceiling 2A or a guide member. In this way, various position detection members are arranged on the back surface of the ceiling 2A and the like in accordance with the position detection method employed by the position detection mechanism.

  FIG. 43 shows an example of the moving mechanism of the projector 402. Both ends of the bottom surface of the projector 402 are supported by two bases 451 with screws, and each base 451 is engaged with two rails 452 with screws. As the two rails 452 are rotated in synchronization by the motor 453, the projector 402 moves back and forth (X direction).

  Further, both ends of each of the two rails 452 have a structure with a screw, and each rail 452 is installed by being engaged with two rails 454 with a screw. As the two rails rotate in synchronization with the motor 455, the projector 402 moves left and right (Y direction).

  In the case of the moving mechanism of the projector 402 shown in FIG. 43, as described above, the guide member for guiding the projector 402 to the positions of the two projection windows 404 is constituted by, for example, the rail 452 or the rail 454.

  The projector 402 includes a mirror that reflects the image light emitted from the projection mechanism in a predetermined direction, and a mirror movable mechanism that changes the position and angle of the mirror. FIG. 44 shows an example of the mirror movable mechanism. A cylinder 461 is attached to the upper surface of the projector 402, and a mirror 462 is attached to the tip of the cylinder 461. With this cylinder 461, the distance of the mirror 462 from the projector 402 can be adjusted.

  A biaxial motor 463 is provided at the mounting position of the mirror 462 of the cylinder 461. The direction of the mirror 462 can be adjusted by the two-axis motor 463. In this way, the position and angle of the mirror 462 can be changed by the cylinder 461 and the biaxial motor 463, so that the direction in which the image light emitted from the projection mechanism of the projector 402 is directed, that is, the indoor position where the image is projected. It can be adjusted and the degree of freedom in selecting the video display position can be increased.

  Returning to FIG. 42, in the audiovisual room 401, screens 405a and 405b are installed in advance in the locations corresponding to the projection windows 404 on the ceiling 2A and the floor 2B described above or moved by the user. Is arranged.

  FIG. 45 shows the configuration of the projector 402 described above. The projector 402 includes a controller 402a that controls the overall operation, the moving mechanism 402b (see FIG. 43), the mirror movable mechanism 402d (see FIG. 44), a projection mechanism 402c for projecting image light, a remote A remote control receiving unit 402e that receives a control signal wirelessly transmitted from the commander 471, a video receiving unit 402f that receives a video signal wirelessly transmitted from a video device 472 such as a DVD player, a personal computer, a TV tuner, and the like. A correction processing unit 402g that performs correction processing for correcting distortion of an image projected on a predetermined position in the room by the projection mechanism 402c, for example, trapezoidal distortion, with respect to the video signal received by the unit 402f, and a position detection mechanism 402h Have. The video equipment 472 is accommodated in a space between the inner wall 2 and the outer wall 3, for example.

  The remote control reception unit 402e receives a control signal wirelessly transmitted from the remote commander 471, and the video reception unit 402f receives a video signal wirelessly transmitted from the video equipment 472. The projector 402 and the video equipment 472 And the projector 402 and the remote commander 471 are wirelessly connected, so that the cable does not get in the way when the projector 402 moves in the space.

  The projection mechanism 402c generates and emits video light based on the video signal received by the video receiving unit 402f. That is, the projection mechanism 402c emits the image light, a light source, an optical spatial modulation element such as a liquid crystal panel that generates image light by spatially modulating the light emitted from the light source with an image signal, as is well known in the art. It is formed by an optical system or the like.

  The remote control receiving unit 402e receives a control signal from the remote commander 471 and notifies the controller 402a. The position detection mechanism 402h detects the movement of the projector 402 to the position where the projection window 404 is provided by the detection of the position detection member described above, and notifies the controller 402 of it. Under the control of the controller 402a, the moving mechanism 402b rotates the screw by the driving force of the motor (see FIG. 43), thereby moving the projector 402 along a rail provided in a space such as a ceiling or a floor.

  The operation of the projector 402 shown in FIG. 45 will be briefly described. A case where the user operates the remote commander 471 to output a control signal instructing to irradiate a predetermined position in the room through a predetermined projection window 404 from the projector 402 disposed in a predetermined space. State.

  In this case, a control signal from the remote commander 471 is received by the remote control receiving unit 402e, and this control signal is supplied to the controller 402a. The controller 402a receives this control signal, controls the moving mechanism 402b, and starts moving the projector 402 to a position corresponding to the predetermined projection window 404 described above. When the movement of the projector 402 to the position of the predetermined projection window 404 is detected by the position detection mechanism 402h, the controller 402a controls the movement mechanism 402b to stop the movement of the projector 402.

  In this case, as shown in FIG. 46, for example, the projector 402 arranged in the space between the ceiling 2A and the ceiling back 3A is a projection window that has been positioned so far along the ceiling surface according to an instruction from the user. It moves from the position of 404 to the position of the next projection window (predetermined projection window 404). In this case, as shown in FIG. 47, for example, the projector 402 disposed in the space between the floor 2B and the underfloor 3B is a projection window that has been positioned so far along the floor surface according to an instruction from the user. It moves from the position of 404 to the position of the next projection window (predetermined projection window 404).

  When the projector 402 moves to the position of the predetermined projection window 404 as described above, the controller 402a controls the mirror moving mechanism 402d, and the mirror 463 passes the predetermined light in the room through the predetermined projection window 404. The position and angle of the mirror 402d are adjusted so that the position is irradiated.

  In addition, when the projector 402 moves to the position of the predetermined projection window 404 as described above, the controller 402a activates the operations of the projection mechanism 402b, the correction processing unit 402g, and the video processing unit 402f. The video receiving unit 402f receives a video signal from the video device 472 and supplies the video signal to the projection mechanism 402c via the correction processing unit 402g.

  Thereby, image light based on the image signal is emitted from the projection mechanism 402c, and the image light is reflected by the mirror 402d and then irradiated to a predetermined position in the room. Then, at a predetermined position in the room, an image based on an image signal supplied from the image device 472 to the image receiving unit 402f is displayed.

  Note that the controller 402a sets the focus and zoom magnification of the projection mechanism 402c in response to a user operation such as focus and zoom in response to a notification from the remote control receiving unit with the video displayed at a predetermined position in the room. adjust.

  Here, as shown in FIG. 48, in the projector 402 arranged in the space between the ceiling 2A and the ceiling back 3A, a mirror 462 is arranged on the optical path of the image light emitted from the projection mechanism 402c. The optical path of the image light emitted toward the direction along the ceiling surface is bent by 462 and irradiated to the screen 405 in the room through the projection window 404 provided on the ceiling 2A. In this case, the irradiation position can be adjusted by changing the tilt of the mirror 462.

  In addition, as shown in FIG. 49, the projector 402 arranged in the space between the floor 2B and the underfloor 3B similarly has a mirror 462 arranged on the optical path of the image light emitted from the projection mechanism 402c. The optical path of the image light emitted in the direction along the floor surface by the mirror 462 is bent and projected onto the screen 405 through the projection window 404 provided on the floor 2B.

  In the configurations shown in FIGS. 48 and 49, trapezoidal distortion or the like is generated by irradiating the screen 405 with image light obliquely. As a result, the correction processing unit 402g of the projector 402 performs processing for preventing the trapezoidal distortion and the like on the video signal.

  As described above, the correction processing unit 402g performs correction processing on the video signal received by the video receiving unit 402f so that the distortion of the video projected to a predetermined position in the room is corrected by the projection mechanism 402c. And supplied to the projection mechanism 402c. Thereby, a good image without distortion is displayed at a predetermined position in the room.

  When the projection window 404 includes a lid, the remote commander 471 controls the opening / closing of the projection window 404. That is, as described above, when the control signal instructing to irradiate the image light to the predetermined position in the room through the predetermined projection window 404 is transmitted from the projector 402 to the remote control receiving unit 402e of the projector 402, the predetermined projection is performed. A control signal is supplied to instruct the window 404 to open the lid.

  Further, as described above, when the predetermined position in the room where the image light is irradiated is the screen 405 and is always wound up, the remote commander 471 controls the opening and closing of the screen 405. That is, as described above, when the control signal instructing to irradiate the image light to the predetermined position in the room through the predetermined projection window 404 is transmitted from the projector 402 to the remote control receiving unit 402e of the projector 402, the screen 405 A control signal is provided to instruct opening.

  As described above, the projector 402 is disposed in the space between the ceiling 2A and the ceiling back 3A, the space between the floor 2B and the floor 3B, or the space between the inner wall 2 and the outer wall 3. It is not arranged in the audiovisual room 401 and does not disturb the indoor atmosphere. In addition, the projector 402 can selectively move to a plurality of positions in the space according to a user operation and irradiate the image light in the room, and the user can move at a plurality of positions in the room without preparing a plurality of projectors. You can watch the video.

  FIG. 50 shows the inside of the audiovisual room (room) 401 in comparison with FIG. 42 described above. In FIG. 42, an image is projected on the screen 405, but in FIG. 50, an image is projected on the ceiling 2A and the floor 2B. That is, the image light from the projector 402 disposed in the space between the ceiling 2A and the ceiling back 3A is irradiated to the floor 2B through the projection window 404 provided on the ceiling 2A. Further, the image light from the projector 402 arranged in the space between the floor 2B and the floor 3B is irradiated to the ceiling 2A through the projection window 404 provided on the floor 2B. In FIG. 50, the projector 402, the ceiling 3A, and the floor 3B are not shown.

  FIG. 51 shows the inside of the audiovisual room (room) 401 in comparison with FIG. 42 described above. In FIG. 42, an image is projected on the screen 405, but in FIG. 51, an image is projected on the inner wall 2 and the window 430. That is, the image light from the projector 402 arranged in the space between the ceiling 2A and the ceiling back 3A is irradiated on the inner wall 2 through the projection window 404 provided on the ceiling 2A. Further, image light from the projector 402 disposed in the space between the floor 2B and the floor 3B is irradiated to the window 430 formed on the inner wall 2 through the projection window 404 provided on the floor 2B. In FIG. 51, the projector 402, the ceiling back 3A, and the under floor 3B are not shown.

  FIG. 52 shows the inside of the audiovisual room (room) 401 in comparison with FIG. 42 described above. In FIG. 42, an image is projected on the screen 405, but in FIG. 52, an image is projected on the table 420. In other words, the image light from the projector 402 arranged in the space between the floor 2B and the floor 3B is irradiated on the table 420 through the projection window 404 provided on the floor 2B. In FIG. 52, the projector 402 and the underfloor 3B are not shown.

  Here, as shown in FIG. 52, the table 420 has a table surface 423 having a substantially square shape when viewed from above, and has a structure in which the four corners of the table surface 423 are supported by the legs 422, respectively. As shown in FIG. 53, the table surface 423 is formed of a transparent plate material such as tempered glass. In addition, a rear projection screen 425 is disposed almost on the entire table surface 423. As a result, the table 420 receives the image light from the projection window 404 provided on the floor 2B by the screen 425, and displays the image by the projector 402 disposed in the space between the floor 2B and the underfloor 3B.

  FIG. 54 shows the inside of the audiovisual room (room) 401 in comparison with FIG. 42 described above. In FIG. 42, an image is projected on the screen 405, but in FIG. 54, an image is projected on the table 420. That is, the image light from the projector 402 arranged in the space between the ceiling 2A and the ceiling back 3A is irradiated to the table 420 through the projection window 404 provided on the ceiling 2A. In FIG. 54, the projector 402 and the ceiling back 3A are not shown.

  55 and 56 show a partial configuration of the audiovisual room system in comparison with FIGS. 48 and 49. In this audiovisual system, video is displayed by one projector 402 in two audiovisual rooms (rooms) 401U and 401D that are continuous in the vertical direction (vertical direction). FIG. 55 shows an example in which an image is displayed on the screen 405 in the audiovisual room 401D on the lower level and an image is displayed on the screen 405 in the audiovisual room 401U on the lower level. On the other hand, FIG. 56 shows an example in which an image is displayed on the screen 405 in the audiovisual room 401D on the lower level and an image is displayed on the table surface of the table 420 in the audiovisual room 401U on the lower level.

  That is, the projector 402 is disposed in a space between the ceiling 2A of the audiovisual room 401D below the floor and the floor 2B of the audiovisual room 401U above the floor. The projector 402 is selectively placed at the position of the projection window 404 provided on the ceiling 2A of the audiovisual room 401D below the floor or the position of the projection window 404 provided on the floor 2B of the audiovisual room 401U on the floor. It can be moved. In this case, although not shown, a guide member (see rails in FIG. 43) for guiding the projector 402 to the position of the projection window 404 is provided in the space.

  The projector 402 moves to the position of the projection window 404 provided on the ceiling 2A of the audiovisual room 401D by the operation of the remote commander 471 in the audiovisual room 401D below the floor. Further, the operation of the remote commander 471 in the audiovisual room 401U on the floor moves to the position of the projection window 404 provided on the floor 2B of the audiovisual room 401U.

  When the projector 402 moves to the position of the projection window 404 provided on the ceiling 2A of the audiovisual room 401D on the lower floor, the image of the mirror 462 is displayed through the projection window 404 so that an image is displayed at a predetermined position in the audiovisual room 401D. The direction is switched. Further, when moving to the position of the projection window 404 provided on the ceiling 2A of the audiovisual room 401D on the lower floor, the direction of the mirror 462 is displayed so that an image is displayed at a predetermined position of the audiovisual room 401D through the projection window 404. Is switched.

  55 and 56, the two audiovisual rooms 401U and 401D can be used as one projector 402, and the configuration can be simplified accordingly.

  FIG. 57 shows a partial configuration of the audiovisual room system in comparison with FIGS. 55 and 56. In this audiovisual system, video is displayed by one projector 402 in two audiovisual rooms (rooms) 401L and 401R that are continuous in the left-right direction (horizontal direction).

  That is, the projector 402 is disposed in a space between the ceiling 2A and the ceiling back 3A of the audiovisual rooms 401L and 401R. The projector 402 is selectively moved to the position of the projection window 404 provided on the ceiling 2A of the left audiovisual room 401L or the position of the projection window 404 provided on the ceiling 2A of the right audiovisual room 401R. It is possible. In this case, although not shown, a guide member (see rails in FIG. 43) for guiding the projector 402 to the position of the projection window 404 is provided in the space.

  The projector 402 moves to the position of the projection window 4 provided on the ceiling 2A of the audiovisual room 401L by the operation of the remote commander 471 in the audiovisual room 401L on the left side. Further, the operation of the remote commander 471 in the right audiovisual room 401R moves the position of the projection window 404 provided on the ceiling 2A of the audiovisual room 401R.

  When the projector 402 moves to the position of the projection window 404 provided on the ceiling 2A of the left audio-visual room 401L, the mirror 462 is displayed so that an image is displayed at a predetermined position in the audio-visual room 401L via the projection window 404. The direction is switched. Further, when moving to the position of the projection window 404 provided on the ceiling 2A of the audiovisual room 401R on the right side, the direction of the mirror 462 is set so that an image is displayed at a predetermined position of the audiovisual room 401R through the projection window 404. Is switched.

  With the configuration shown in FIG. 57, the two audiovisual rooms 401L and 401R can be used as one projector 402, and the configuration can be simplified accordingly. It goes without saying that one projector can be used in the same manner in more audiovisual rooms arranged in the horizontal direction. In FIG. 57, the projector 402 is arranged in the space between the ceiling 2A and the ceiling back 3A. However, the projector 402 arranged in the space between the floor and the floor can be similarly configured.

  FIG. 58 shows the inside of the audiovisual room (room) 401 in comparison with FIG. 50 described above. FIG. 50 shows a case in which two projection windows 404 are provided on the ceiling 2A. In FIG. 58, one strip-shaped projection window 404 is provided on the ceiling 2A. The projection window 404 is created by being covered with, for example, transparent glass. In this case, the projection position of the image can be changed variously by moving the projector 402 to each part of the strip-shaped projection window 404.

  FIG. 59 shows the inside of the audiovisual room (room) 401 in comparison with FIG. 58 described above. 58 shows a case in which one strip-shaped projection window 404 is provided on the ceiling 2A, but in FIG. 59, one strip-shaped projection window 404 is provided on the inner wall 2. In FIG. The projection window 404 is created by being covered with, for example, transparent glass. In this case, the projector 402 arranged in the space between the inner wall 2 and the outer wall 3 can be moved to each part of the belt-like projection window 404 to change the image projection position on the opposite inner wall 2 in various ways.

  In the above-described embodiment, the guide member of the projector 402 is a rail. However, the present invention is not limited to this, and various methods can be widely applied.

  Further, in the above-described embodiment, the case where an image is projected on the screen 405, the table 420, the floor 2B, the ceiling 2A, the inner wall 2 and the like has been described. In the projection location, various settings can be made.

  In the above-described embodiment, the case where the ceiling, floor, and wall are formed in a double structure has been described. However, the present invention is not limited to this, and the main point is that the projector can be moved along the ceiling surface, floor surface, and wall surface. In addition, it may be arranged on the back side of the ceiling, the back side of the floor, or the back side of the wall, and can be widely applied to audiovisual rooms having various structures.

  (4) Remote commander configuration example

  A remote commander that can be used as the remote commander 116 of the image display system 101 described above or the remote commander 471 that controls the projector 402 described above will be described.

  FIG. 60 is a schematic diagram illustrating the configuration of the remote commander 201. The remote commander 201 receives a user operation from the table 202. Here, as shown in FIG. 61, the table 202 has a table surface 203 having a substantially square shape when viewed from above, and has four corners of the table surface 203 supported by legs 204. The table surface 203 is formed of a transparent plate material such as tempered glass. A screen 206 and a transparent touch panel 207 of a rear projection type projector are laminated in this order on almost the entire table surface 203. The touch panel 207 constitutes a sensing unit that detects an instruction by a user operation. Here, the entire table surface 203 may be a transparent portion, but at least a portion corresponding to the screen 206 may be a transparent portion.

  As shown in FIG. 61, the table 202 includes a table base 208 on the lower side of the table surface 203 and inside the legs 204. That is, as shown in FIG. 62A and FIG. 62B, respectively, a side view and a bottom view, the table base 208 is formed in a rectangular tube shape that substantially covers the entire lower surface of the table surface 203. As will be described later, the projector 210 disposed below the table 202 is covered with the table base 208. Here, the entire table surface 203 may be supported by the table base 208, and the legs 204 may be omitted.

  Here, the projector 210 is a rear projection type projector, and the video light by the video signal SV that is wirelessly transmitted from a video device (not shown) such as a DVD player, a video tape recorder, and a TV tuner and received by the transmission / reception unit 212. Is emitted upward. Thereby, as schematically shown in FIG. 63, an image based on the video signal SV is displayed on the screen 206 provided on the table surface 203.

  The signal processing unit 211 detects an instruction by the operation of the touch panel 207 by the user, and notifies the transmission / reception unit 212 of this instruction. The transmission / reception unit 212 wirelessly transmits the instruction by the user operation to the video equipment. In addition, the transmission / reception unit 212 wirelessly communicates with the video device, receives the video signal SV from the video device, and supplies this to the projector 210.

  The operation of the remote commander 201 shown in FIG. 60 will be described. For example, when the user performs a touch operation outside the effective screen of the touch panel 207, the signal processing unit 211 detects a menu screen request instruction. The request instruction for the menu screen is transmitted to the video equipment through the transmission / reception unit 212.

  Then, the video signal for the menu screen transmitted from the video device in response to the instruction instruction for the menu screen is received by the transmission / reception unit 212, and the video signal SV is supplied to the projector 210. The projector 210 emits image light based on the image signal SV and irradiates the screen 206 disposed on the table surface 203. As a result, a menu screen is displayed on the screen 206.

  In addition, when the user operates the touch panel 207 while the menu screen is displayed on the screen 206 in this way, the signal processing unit 211 detects an instruction to select a predetermined menu. This menu selection instruction is transmitted to the video equipment through the transmission / reception unit 212. Thereby, various operations of the video equipment can be controlled. This operation of the video equipment includes an operation of sending a predetermined video signal to the remote commander 201.

  In addition, the operation of the video equipment includes the operation of the control unit 113 of the image display system of FIG. 19 described above. In this case, the screen 206 constitutes a bird's-eye view image display unit 117, and displays an operation screen in a bird's-eye mode and an immersion mode as shown in FIGS. 32A and 32B, for example. In this case, the menu screen as described above is not displayed, and the bird's-eye view mode is initially displayed on the screen 206, and either the bird's-eye mode or the immersive mode operation screen is displayed according to the subsequent user operation. Is done. The video signal related to such an operation screen is wirelessly transmitted from the video device (corresponding to the image processing unit 113B in FIG. 19) and received by the transmission / reception unit 212 every time the screen content is changed.

  As described above, according to the remote commander 201 shown in FIG. 60, an image is displayed on the screen 206 arranged on the table 202 as furniture, and an instruction by a user operation on the screen 206 is detected by the touch panel 207 as a sensing unit. Therefore, it is not found when trying to operate like a conventional remote commander, and user convenience can be improved.

  FIG. 64 shows a part of the remote commander 201A in comparison with FIG. In FIG. 64, portions corresponding to those in FIG. 60 are denoted by the same reference numerals. In this remote commander 201A, the table 202 is fixed and held at a fixed location on the floor 2B. As described above, a projection window 213 for allowing the image light from the projector 210 to pass through is provided on the floor 2B where the table 202 is fixedly held.

  The projection window 213 may be formed by an opening, or may be formed by arranging a transparent plate material such as glass in the opening. Further, a lid (not shown) may be provided corresponding to the projection window 213, and the lid may be opened only when in use.

  The projector 210 is disposed in a space between the floor 2B and the underfloor 3B so as to correspond to the projection window 213 described above. The image light emitted from the projector 210 is irradiated onto the screen 206 disposed on the table surface 203 of the table 202 via the projection window 213 provided on the floor 2B, and the image is displayed on the screen 206. .

  The remote commander 201A is configured in the same manner as the remote commander 201 shown in FIG. 60 except for the configuration related to the arrangement position of the projector 210 as described above. The remote commander 201A can obtain the same effects as the remote commander 201, and the projector 210 is disposed in the space between the floor 2B and the underfloor 3B and is not disposed indoors. The table 208 for covering the projector 210 as in the remote commander 201 shown in FIG. 5 can be omitted, and the projector 210 is disposed in the space between the floor 2B and the floor 3B. The optical path can be obtained, and the thickness of the lower structure of the table 202 can be reduced to display an image on a large screen.

  FIG. 65 shows a part of the remote commander 201B in comparison with FIG. In FIG. 65, portions corresponding to those in FIG. 60 are denoted by the same reference numerals. In the remote commander 201B, the projector 210 is disposed on the floor 2B, and an optical path shortening lens 222 is provided on the optical path of the projector 210. In this case, the image light emitted from the projector 210 is applied to the screen 206 through the lens 222.

  The remote commander 201B is configured in the same manner as the remote commander 201 shown in FIG. 60 except for the configuration related to the lens 222 as described above. This remote commander 201B can obtain the same effect as the remote commander 201, and the optical path from the projector 210 to the screen 206 is shortened by the lens 222. Therefore, the thickness of the lower structure of the table 202 is reduced and increased. Video display on the screen is possible.

  FIG. 66 shows a part of the remote commander 201C in comparison with FIG. In FIG. 66, portions corresponding to those in FIG. 64 are denoted by the same reference numerals. In this remote commander 201C, the projector 210 is disposed in a space between the floor 2B and the floor 3B, and a mirror 234 for bending the optical path of the image light emitted from the projector 210 is provided.

  In this case, the projector 210 is arranged so that the image light is emitted in a direction along the floor surface. Thus, the image light emitted from the projector 210 is reflected in a direction orthogonal to the mirror 234 and is irradiated onto the screen 206 through the projection window 213 provided on the floor 2B.

  The remote commander 201C is configured similarly to the remote commander 201A shown in FIG. 64 except for the configuration related to the mirror 234 as described above. This remote commander 201C can obtain the same effect as the remote commander 201A, and also bends the optical path with a mirror 234. Even if the thickness of the lower structure of the table 202 is reduced, the image irradiated to the screen 206 A sufficient optical path of light can be secured and video display on a large screen becomes possible.

  FIG. 67 shows a part of the remote commander 201D in comparison with FIG. In FIG. 67, portions corresponding to those in FIG. 66 are denoted by the same reference numerals. In the remote commander 201D, an optical path shortening lens 245 is provided on the optical path of the projector 210. In this case, the image light emitted from the projector 210 is irradiated on the screen 206 via the lens 245 and the mirror 234.

  This remote commander 201D is configured similarly to the remote commander 201C shown in FIG. 66 except for the configuration related to the lens 245 as described above. The remote commander 201D can obtain the same effect as the remote commander 201C, and the optical path from the projector 210 to the screen 206 is shortened by the lens 245, so that the thickness of the lower structure of the table 202 is further reduced. Video display on a large screen is possible.

  68 and 69 show a state of the audiovisual room (room) 401 in which the remote commander 201E is arranged in comparison with FIGS. 60 and 61. FIG. In FIGS. 68 and 69, portions corresponding to those in FIGS. 60 and 61 are denoted by the same reference numerals. Here, the audiovisual room 401 is surrounded by the inner wall 2, the ceiling 2A, and the floor 2B. The table 202 constituting the remote commander 201E is placed on the floor 2B.

  A liquid crystal film 206A and a transparent touch panel 207 are laminated in this order on almost the entire table surface 203 made of a transparent plate material of the table 202. That is, in this remote commander 201E, the liquid crystal film 206A is arranged in place of the screen 206 of the rear projection projector of the remote commander 201 shown in FIGS.

  This liquid crystal film 206A is a sheet material formed by disposing liquid crystal between transparent films having two transparent electrodes. In the liquid crystal film 206A, the transmittance in the thickness direction varies greatly depending on the voltage value applied to the transparent electrode. Here, for example, by operating a switch provided on the side surface of the table 202, the voltage applied to the liquid crystal film 206A is switched, and the liquid crystal film 206A is in a transparent state and translucent enough to be used as a screen of the projector 210. It is possible to selectively switch to a different state.

  In this case, as shown in FIG. 68, the liquid crystal film 206A is set in a translucent state, and image light from the projector 210 is irradiated onto the liquid crystal film 206A, whereby an image is displayed on the table surface 203 (liquid crystal film 206A) of the table 202. Can be displayed. In addition, as shown in FIG. 69, the liquid crystal film 206A is in a transparent state, and the image light from the projector 210 passes through the liquid crystal film 206A and is irradiated onto the ceiling 2A, and an image can be displayed on the ceiling 2A. The projector 210 is configured to be able to adjust the focus following such switching of the image light irradiation target.

  The remote commander 201E is configured similarly to the remote commander 201 except for the configuration related to the liquid crystal film 206A as described above. The remote commander 201E can obtain the same effects as the remote commander 201, and can display an image on the ceiling 2A with the liquid crystal film 206A in a transparent state, thereby further improving the usability of the user.

  FIG. 70 shows the configuration of the remote commander 201F in comparison with FIG. In FIG. 70, portions corresponding to those in FIG. 60 are denoted by the same reference numerals. In this remote commander 201F, in place of the touch panel 207 and the signal processing unit 211 in the remote commander 201, an imaging device (camera) 266 and an image processing unit 267 are provided. The imaging device 266 and the image processing unit 267 constitute a sensing unit that detects an instruction by a user operation.

  Here, the imaging device 266 is arranged adjacent to the projector 210, images a user's operation of a finger or the like on the screen 206 together with an image displayed on the screen 206, and video data as an imaging result is image processing unit To 267. The image processing unit 267 processes the video data to detect an instruction by a user's operation such as a finger on the screen 206 and notifies the transmission / reception unit 212 of the instruction. The transmission / reception unit 212 wirelessly transmits the instruction by the user operation to the video equipment.

  The remote commander 201F is configured in the same manner as the remote commander 201 shown in FIG. 60 except for the configuration related to the sensing unit as described above. The remote commander 201F can obtain the same effect as the remote commander 201 and has an effect that it is not necessary to arrange the touch panel 207 for user operation on the table surface 203.

  FIG. 71 shows a part of the remote commander 201G in comparison with FIG. In FIG. 71, portions corresponding to those in FIG. 70 are denoted by the same reference numerals. In this remote commander 201G, the table 202 is fixed and held at a fixed location on the floor 2B. In this way, a projection window 213 for allowing the image light from the projector 210 to pass is provided in the portion of the floor 2B where the table 202 is fixedly held, similarly to the remote commander 201A shown in FIG.

  The projector 210 and the imaging device 266 are arranged in a space between the floor 2B and the underfloor 3B corresponding to the projection window 213 described above. The image light emitted from the projector 210 is irradiated onto the screen 206 disposed on the table surface 203 of the table 202 via the projection window 213 provided on the floor 2B, and the image is displayed on the screen 206. . In addition, the imaging device 266 captures an operation of a user's finger or the like on the screen 206 together with an image displayed on the screen 206 via a projection window 213 provided on the floor 2B.

  The remote commander 201G is configured in the same manner as the remote commander 201F shown in FIG. 70 except for the configuration relating to the arrangement positions of the projector 210 and the imaging device 266 as described above. The remote commander 201G can obtain the same effect as the remote commander 201F, and the projector 210 and the imaging device 266 are arranged in the space between the floor 2B and the underfloor 3B and are not arranged indoors. Does not disturb the indoor atmosphere.

  FIG. 72 shows a part of the remote commander 201H in comparison with FIG. In FIG. 71, portions corresponding to those in FIG. 70 are denoted by the same reference numerals. In this remote commander 201H, the image processing unit 267A that processes video data from the imaging device 266 distinguishes the instruction by the drag operation of the moving body on the screen 206 from the instruction by the simple touch operation on the screen 206. The instruction is detected and the transmission / reception unit 212 is notified of this instruction.

  The remote commander 201H is configured similarly to the remote commander 201F shown in FIG. 70 except for the configuration related to the image processing unit 267A. The remote commander 201H can obtain the same effects as the remote commander 201F, and the image processing unit 267A detects and detects an instruction by a simple touch operation on the screen 206 and an instruction by dragging a moving body. Thus, the types of instructions by user operations can be increased.

  In the above embodiment, the screen 206 is arranged on the table surface 203 made of a transparent plate material. However, the screen 206 itself may constitute the table surface 203. In the above-described embodiment, the remote commander using a table has been described. However, the respective configurations may be appropriately combined.

  In the above-described embodiment, the case where the image is projected by the rear projection type projector 210 has been described. However, the present invention is not limited to this, and the image may be projected by the front projection type projector. . In this case, the projector is arranged on the ceiling or the like instead of the arrangement of the projector below the floor.

  In the above-described embodiment, the furniture is a table. However, the same remote commander can be obtained by using other furniture such as a bookshelf, a desk, and a chest. In the above-described embodiment, the remote commander for video equipment has been described. However, the present invention is not limited to this, and can be widely applied to remote commanders for various equipment.

  (5) Display in the bird's eye image display unit

  In the image display system 101 shown in FIG. 19 described above, the bird's-eye image display unit 117 configuring the remote commander 116 includes, for example, a bird's-eye image and a photograph taken at a shooting point (shooting location) specified on the bird's-eye image. Images are displayed simultaneously (see FIG. 32A). Here, details of image processing for this display will be described.

  FIG. 73 shows a configuration of the image processing apparatus 301 for displaying a bird's-eye view image on which a captured image is arranged as described above. In FIG. 73, portions corresponding to those in FIG. 19 are denoted by the same reference numerals, and detailed description thereof is omitted as appropriate. The image processing apparatus 301 includes a media database 112, a control unit 113, a bird's eye image display unit 117, and a touch panel 118. The image display device 301 is configured by a computer having sufficient image processing capability.

  As described above, the media database 112 is formed by a recording / playback unit, for example, a large-capacity hard disk device. The media database 112 records various programs necessary for the operation of the image display device 301, image data of a plurality of types of bird's-eye images, image data of images taken at shooting points specified on each bird's-eye image, and the like. The stored image data is reproduced and output.

  Here, the bird's-eye view image is an image obtained by viewing the shooting point of the shot image from the sky, and as this bird's-eye view image, an image taken by actual shooting, an image by illustration, an image by computer graphics, a map image, or the like can be applied. In this embodiment, a map image is applied as a bird's-eye view image. That is, the media database 112 records and holds the map image image data D4 as the bird's-eye image data.

  As shown in FIG. 74 using a map M, the photographed images are various images photographed at the photographing points P1 to P3 set in the map image. In this embodiment, a panoramic image is applied as the photographed image. ing. That is, the media database 112 records and holds panoramic image data D3 as captured image data. This panoramic image is acquired by photographing the surroundings with a fisheye lens, and is formed in a circular shape. The panorama image at each shooting point is set with information on the shooting point as link information for specifying the shooting point on the map image.

  Instead of setting the link information on the panoramic image side, link information for specifying the corresponding panoramic image may be set at the shooting point on the map image side. Further, as this panoramic image, an imaging result using a hyperboloidal mirror may be applied, or an imaging result by a plurality of images taken by panning an imaging device (camera) by a certain angle may be applied. The panoramic image by the method can be widely applied. In addition, each panoramic image is photographed so that a specific direction has a certain direction, and altitude information indicating the altitude of the photographing point is attached and provided.

  The media database 112 is provided with the panoramic image data D3, the map image data D4, and the program necessary for the processing of the image processing apparatus 301 installed in advance. Instead of installation, the program may be provided by being recorded on various recording media such as an optical disk, a magnetic disk, and a memory card, or may be provided by downloading via a network such as the Internet.

  The bird's-eye view image display unit 117 is formed by, for example, a liquid crystal display panel, displays an image based on the video data DV2 output from the image processing unit 113B, and thereby displays an image in which a panoramic image is arranged on the map image.

  The touch panel 118 constitutes a user operation unit. With this touch panel 118, the user can perform an operation for changing the position of the viewpoint, an operation for switching scenes, and the like. The operation information from the touch panel 118 is notified to the display information selection unit 113A. As the user operation unit, an operation device such as a mouse, an acceleration sensor, or a joystick can be used instead of the touch panel 118.

  The control unit 113 is a computing unit that executes a program recorded in the media database 112, and configures various functional blocks by executing this program. That is, the control unit 113 includes an image processing unit 113B and a display image selection unit 113A. Further, the image processing unit 113B includes an omnidirectional image conversion unit 304A, a map image / panorama image arrangement unit 304B, and a viewpoint coordinate conversion unit 304C.

  As shown in FIGS. 75A and 75B, the omnidirectional image conversion unit 304A maps each pixel constituting the circular panorama image based on the imaging results A1 to A3 to a rectangular region that is elongated in the horizontal direction. The panoramic images A1 to A3 having a circular shape are subjected to image conversion processing into panoramic images B1 to B3 having a rectangular shape having a wide angle of view in the horizontal direction. For panoramic images taken using a hyperboloid mirror, panoramic images B1 to B3 having a rectangular shape with a wide angle of view in the horizontal direction can be generated by an image conversion process using a similar mapping process. . Further, panoramic images of a plurality of images captured by panning the imaging device by a certain angle can create such panoramic images B1 to B3 by connecting the plurality of images.

  The map image / panorama image arrangement unit 304B arranges the panorama image on the map image based on the map image data D4. In this case, the panoramic image is arranged to face the viewpoint. The viewpoint coordinate conversion unit 304C performs viewpoint coordinate conversion processing for converting the image in which the panoramic image is arranged in this way into an image viewed from the viewpoint direction, and outputs the video data DV2 as a result of the processing to the bird's-eye image display unit 117. To do.

  The display information selection unit 113A selects a map image and a panoramic image to be processed by the image processing unit 113B, that is, to be displayed on the bird's eye image display unit 117, in accordance with the notification of the operation information from the touch panel 118. In this case, the image data of the selected map image and panorama image is read from the media database 112 and supplied to the image processing unit 113B. Further, the display information selection unit 113A provides viewpoint information to the viewpoint coordinate conversion unit 304C of the image processing unit 113B in accordance with the notification of the operation information from the touch panel 118.

  Next, the processing procedure of the control unit 113 related to the display of the map image on which the panoramic image is arranged will be described with reference to the flowcharts of FIGS.

  In step ST1, the control unit 113 starts display processing when the power is turned on or when the user touches the touch panel 118, and proceeds to step ST2. Here, the control unit 113 captures from the media database 112 the image data D4 of the map image to be displayed in the initial state, the image data D3 of the panoramic image photographed at the photographing point specified on the map image, and further the photographing. The information of the shooting point indicating the point is read. The map image to be displayed in the initial state is a fixed one set in advance or one displayed at the end of the previous time.

  Subsequently, in step ST3, the control unit 113 converts the circular panorama images A1 to A3 obtained in step ST2 into the horizontally long panorama images B1 to B1 by the processing related to the functional blocks of the omnidirectional image conversion unit 304A. Image conversion processing is performed on B3 (see FIG. 75). The processes in steps ST2 and ST3 described above are initialization processes.

  Subsequently, the control unit 113 proceeds to step ST4, and performs landscape image display processing. FIG. 77 shows the procedure of landscape image display processing. That is, the control unit 113 starts landscape image display processing in step ST111 and proceeds to step ST112. Here, as shown in FIG. 80, the control unit 113 performs the processing related to the functional block of the map image / panorama image arrangement unit 304B to the shooting points P1 to P3 on the map image M based on the information of the shooting points. Landscape image display boards C1 to C3 that are planes of a certain size are arranged, and the boards C1 to C3 are inclined so as to face the direction of the viewpoint PS.

  Here, the “facing” is a relationship in which a straight line connecting the center of each board C1 to C3 and the viewpoint PS is orthogonal to the plane of each board. Here, only the direction along the map image M is opposed to the direction of the viewpoint PS, and the inclination (elevation angle) in the direction of looking up the viewpoint PS of each board C1 to C3 is changed as necessary. The boards C1 to C3 may stand upright on the map image M.

  Subsequently, in step ST113, the control unit 113 performs processing related to the functional block of the map image / panorama image arrangement unit 304B to convert the panorama images B1 to B3 into the sizes of the boards C1 to C3 as shown in FIG. It cuts out by the area | region AR of fixed width corresponding to. In this way, the region AR having a certain width is cut out from the panoramic images B1 to B3, and the horizontal viewpoint of the panoramic images D1 to D3 displayed on the boards C1 to C3 is limited as described later, thereby moving the viewpoint. Even if the boards C1 to C3, and therefore the panoramic images D1 to D3, are directly opposed to the PS, the panoramic images D1 to D3 are prevented from interfering with each other (so that they do not collide with each other). It is for improving.

  Here, it is assumed that the cutout area AR cut out from the panoramic images B1 to B3 corresponds to the viewpoint PS. That is, the control unit 113 sets the cutout area AR as an area that can be seen from the viewpoint direction. For example, when the viewpoint exists in the north direction of the map image M, the south part of the panoramic images B1 to B3 is cut out. On the other hand, when the viewpoint exists in the south direction of the map image M, the north part of the panoramic images B1 to B3 is cut out. The control unit 113 sequentially moves the cutout area AR as indicated by an arrow C by moving the viewpoint.

  Subsequently, in step ST114, the control unit 113 pastes the images D1 to D3 of the cutout area AR cut out from the panoramic images B1 to B3 as described above to the boards C1 to C3, respectively. Panorama images D1 to D3 photographed at the photographing points P1 to P3 are arranged at the photographing points P1 to P3.

  Subsequently, in step ST115, the control unit 113 performs a viewpoint so that the map image M on which the panoramic images D1 to D3 are arranged becomes an image viewed from the viewpoint PS by the processing related to the functional block of the viewpoint coordinate conversion unit 304C. The coordinate conversion process is executed, and the video data DV2 of the display image obtained by this process is supplied to the bird's eye image display unit 117. In this embodiment, viewpoint coordinate conversion processing is executed by parallel conversion processing, but various methods can be applied as necessary. Thereby, on the bird's-eye view image display unit 117, as shown in FIGS. 82A and 82B, an image in which the panoramic images D1 to D3 are arranged on the map image M is displayed as viewed from the viewpoint PS. Here, the display examples in FIGS. 82A and 82B have a relationship in which the viewpoint PS is arranged on the opposite side.

  In this case, the control unit 113 arranges marks M1 to M3 having a triangular shape with apexes facing downward on the map image M so that the photographing point is indicated by the tip, and the panoramic image is located above the marks M1 to M3. D1 to D3 are displayed so that the photographing locations corresponding to the panoramic images D1 to D3 can be grasped in detail.

  Further, based on the altitude information set for each panoramic image data, the length L of the marks M1 to M3 is set so as to be proportional to the altitude of the shooting point, so that the altitude of the shooting point can be roughly grasped. To. In addition, in such a mark, various shapes can be widely applied. For example, a mark and a panoramic image are displayed in a form as if a signboard is displayed on a pole stabbed in the ground, The altitude may be indicated by the length of the pole. Furthermore, the control unit 113 displays buttons B1 and B2 on the near side for the user to control the movement of the viewpoint.

  In this manner, the control unit 113 actually arranges the panoramic images D1 to D3 on the map image M in this way to form a two-dimensional image viewed from the viewpoint PS, and then on the two-dimensional image. The pixel values corresponding to the map image M and the panoramic images A1 to A3 are mapped, and the series of processes is executed.

  After step ST115, the control unit 113 ends the landscape image display process in step ST116, proceeds to step ST5 in FIG. 76, and performs viewpoint conversion processing. FIG. 78 shows the procedure of viewpoint conversion processing. That is, the control unit 113 starts viewpoint conversion processing in step ST211 and proceeds to step ST212. Here, based on the operation information from the touch panel 118, the control unit 113 determines whether or not the zoom-in operation has been performed by processing related to the functional block of the display image selection unit 113A. In step ST213, zoom-in viewpoint movement processing is performed, and then in step ST218, the viewpoint conversion processing ends.

  When the zoom-in operation is not performed in step ST212, the control unit 113 proceeds to step ST214 and determines whether or not the zoom-out operation is performed. If a zoom-out operation has been performed, the process proceeds to step ST215 to perform a zoom-out viewpoint movement process, and thereafter, in step ST218, the viewpoint conversion process ends.

  Here, the zoom-in and zoom-out viewpoint movement process described above is a process of changing the distance D from the map image M to the viewpoint PS by a certain value, as shown in FIG. In this case, in the zoom-in viewpoint movement process, the distance D is reduced by a certain value, and the viewpoint PS is brought close to the map image M by a certain value. On the other hand, in the zoom-out viewpoint movement process, the distance D is increased by a certain value, and the viewpoint PS is moved away from the map image M by a certain value. The display image selection unit 113A provides the viewpoint information thus changed to the viewpoint coordinate conversion unit 304C described above.

  When the zoom-in operation or the zoom-out operation is continuously performed, the viewpoint movement process in step ST213 or step ST215 is repeatedly performed, and the viewpoint PS is continuously changed. The viewpoint information changed by the viewpoint movement process of zooming in or zooming out in this way is given to the viewpoint coordinate conversion unit 304C, and the viewpoint coordinate conversion is performed so that the image is viewed from the viewpoint changed by the viewpoint coordinate conversion unit 304C. By executing the processing, the image displayed on the bird's-eye image display unit 117 is zoomed in or zoomed out as shown in FIG.

  If the zoom-out operation has not been performed in step ST214, it is determined in step ST216 whether or not a bird's-eye viewpoint movement operation has been performed. When the bird's-eye viewpoint movement operation is performed, the control unit 113 proceeds to step ST217, performs the bird's-eye viewpoint movement process by the process related to the functional block of the display image selection unit 113A, and then performs the viewpoint conversion process in step ST218. Exit.

  Here, the above-described bird's-eye viewpoint movement process is a process of moving the position of the viewpoint PS as shown in FIGS. 85A to 85C, for example. FIG. 85A is an example in which the viewpoint PS is moved at a predetermined speed so as to turn in the sky around the attention point. FIG. 85B is an example of moving on a spherical surface of a hemisphere centered on the point of interest at a predetermined speed and path. In these examples of FIGS. 85A and 85B, once set, it is determined that the bird's eye viewpoint movement operation is performed until the setting is canceled. On the other hand, FIG. 85C is an example in which the viewpoint PS is moved horizontally by the user dragging the screen, for example. The operation of the buttons B1 and B2 (see FIG. 82) for controlling the movement of the displayed viewpoint displayed on the screen is also included in this bird's-eye viewpoint movement operation.

  When the bird's-eye viewpoint movement operation is continuously performed, the viewpoint movement process in step ST217 is repeatedly performed, and the viewpoint PS is continuously changed. Information on the viewpoint changed by the bird's-eye viewpoint movement process is given to the viewpoint coordinate conversion unit 304C, and the viewpoint coordinate conversion process is executed so that the image is viewed from the viewpoint changed by the viewpoint coordinate conversion unit 304C. The Thereby, for example, as shown by arrows D and E1 to E3 in FIG. 80, the direction of the map image M and the boards C1 to C3 is changed in accordance with the movement of the viewpoint PS and displayed on the boards C1 to C3. The contents of the panoramic images D1 to D3 to be changed are also changed.

  In step ST218, the control unit 113 ends the viewpoint conversion process, and then proceeds to step ST6 in FIG. Here, based on the operation information from the touch panel 118, the control unit 113 determines whether or not a scene switching operation has been performed by processing related to the functional block of the display image selection unit 113A. If there is a scene switching operation, the process proceeds to step ST7.

  The scene switching operation by the user is performed by selecting a new map image (bird's-eye image) to be displayed on the bird's-eye image display unit 117 using the touch panel 118. In this case, the control unit 113 displays a menu screen alone or together with a map image on the bird's-eye image display unit 117 according to a user request, and the user selects a desired map image based on the menu screen. Good.

  FIG. 79 shows the procedure of the scene switching process. That is, the control unit 113 starts a scene switching process in step ST311, and proceeds to step ST312. Here, the control unit 113 indicates the image data D4 of the selected map image from the media database 112, the image data D3 of the panoramic image photographed at the photographing point specified on the map image, and further the photographing point. Read information about the shooting location.

  Subsequently, in step ST313, the control unit 113 converts the panoramic images A1 to A3 having a circular shape acquired in step ST312 into the horizontally long panoramic images B1 to B1 by processing related to the functional blocks of the omnidirectional image conversion unit 304A. Image conversion processing is performed on B3 (see FIG. 75). The processes in steps ST312 and ST313 described above are processes corresponding to the initialization processes in steps ST2 and ST3 described above.

  After step ST313, the control unit 113 ends the scene switching process in step ST314, and proceeds to step ST8 in FIG. In the scene switching process, the new map image data D4 and the like are read from the media database 112 as described above, and thus the landscape image display process described above is performed using the new map image data D4 and the like. A new map image M or the like is displayed on the bird's-eye view image display unit 117, and scene switching is performed.

  In the flowchart of FIG. 76, when the scene switching operation is not performed in step ST6, the process immediately proceeds to step ST8. Here, the control unit 113 determines whether or not to end the display on the bird's-eye view image display unit 117. For example, the control unit 113 determines that the process is ended when the power is turned off or the user performs an operation of touching a predetermined portion of the touch panel 118 twice in succession. Further, for example, the control unit 113 determines that the process is finished when a predetermined time has elapsed from the last time when the user operated the touch panel 118. Note that the termination determination condition is not limited to this.

  When it is not determined to end, the control unit 113 returns to step ST4 and repeats the same processing as described above. On the other hand, when determining that the process is to end, the control unit 113 proceeds to step ST9 and ends the display process.

  As described above, in the image display device 301 in FIG. 73, the display process is started when the power is turned on or when the user touches one of the touch panels 118. Initially, the image data D4 of the map image to be displayed in the initial state from the media database 112 to the control unit 113, and the image data D3 of the panoramic image photographed at the photographing point specified on the map image, and further Information on the shooting point indicating the shooting point is read. In the control unit 113, a display image is generated using the image data and the like. In this case, a panoramic image obtained by shooting at each shooting point of the map image is arranged and converted into an image viewed from the viewpoint. Thereby, the bird's-eye view image display unit 117 displays an image in which the panoramic image is arranged on the map image as viewed from the viewpoint (see FIGS. 82A and 82B).

  After the map image in which the panoramic image is arranged is displayed on the bird's-eye image display unit 117 in this way, the user performs operations for viewpoint conversion (zoom-in operation, zoom-out operation, bird's-eye viewpoint movement operation, etc.) on the touch panel 118. Then, the viewpoint moving process is performed by the control unit 113, and the image in which the panoramic image is arranged on the map image displayed on the bird's-eye view image display unit 117 changes with the movement of the viewpoint position.

  In addition, after the map image on which the panoramic image is arranged is displayed on the bird's eye image display unit 117 in this way, when the user performs a scene switching operation on the touch panel 118, a new selection is made from the media database 112 to the control unit 113. The image data D4 of the map image and the image data D3 of the panoramic image photographed at the photographing point specified on the map image, and further the information of the photographing point are read, and a new map image is displayed in the bird's eye image display unit 117. An image based on the data D4 or the like is displayed, and scene switching is performed.

  As described above, according to the image display device 301 shown in FIG. 73, the image displayed on the bird's-eye image display unit 117 is a panoramic image (captured image) photographed at the photographing point of the map image M (bird's-eye image). It will be arranged. Therefore, it is possible to easily grasp the geographical relationship between a plurality of panoramic images, that is, the panoramic images D1 to D3 in the above-described embodiment.

  Further, according to the image display device 301 shown in FIG. 73, the panoramic image arranged at the photographing point of the map image M is arranged so as to face the viewpoint PS, and the map image M on which the panoramic image is arranged is further viewed from the viewpoint. A display image is obtained by being converted into a viewed image. Therefore, the map image M is displayed as viewed from the viewpoint PS, and the panoramic image is displayed in a state of facing the viewpoint PS, so that an image that is consistent with the viewing direction can be displayed. This makes it easier to grasp the geographical relationship between a plurality of panoramic images.

  Further, according to the image display device 301 shown in FIG. 73, a partial region corresponding to the viewpoint is cut out and arranged at the shooting point of the map image M from the panoramic image that is a wide-angle image (see FIG. 81). ). Therefore, the panorama image displayed on the map image M corresponds to the viewpoint, that is, an image that can be viewed from the viewpoint direction, and the user can check the image, so the place introduced by the map image and the panorama image. Can be fully understood.

  73, the user can change the position of the viewpoint PS using the touch panel 118 that is a user operation unit. Therefore, the user can change the direction of the displayed map image M by changing the viewpoint position, and the content of the panorama image arranged on the map image M changes accordingly. It is easier to understand the relationship.

  Further, according to the image display device shown in FIG. 73, the panorama image is arranged on the map image M by cutting out and arranging a partial area of the panorama image (see FIG. 81). Therefore, even when a large number of horizontally long panoramic images are displayed, overlapping between the panoramic images is prevented and visibility is ensured, and this also makes it possible to easily grasp the geographical relationship between the panoramic images. Become.

  If there are few panoramic images to be arranged on the map image M, the panorama image may be arranged directly by omitting the clipping process. In this case, in the panorama image arranged on the map image M, By scrolling the display of each part in the panoramic image so that the part that can be seen from the viewpoint direction is the horizontal direction and center of the panoramic image, it can be seen from the viewpoint and actually in the field It can be aligned with the scenery that can be done.

  In the embodiment described above, the image display device 301 shown in FIG. 73 is applied to display a bird's-eye image on the bird's-eye image display unit 117 in the remote commander 116 of the image display system 101 shown in FIG. FIG. 86 shows an example in which the remote commander 116 is formed on a table that is furniture, and the bird's-eye view image display unit 116 is arranged on the table surface.

  Further, the image display device 301 shown in FIG. 73 can also be applied to a car navigation device as shown in FIG.

  In this case, for example, a corresponding panoramic image is arranged and displayed on a map image corresponding to a current position detected by a GPS (Global Positioning System) or the like and a destination designated by the user. At this time, the viewpoint is set in the default direction, and further, the viewpoint is set according to the direction of the automobile detected by GPS or the like, and the map image and the panorama image are displayed.

  In addition, selection of a destination on a map image is accepted by operating a touch panel arranged on the display screen, and selection of a destination is accepted by selecting a panoramic image. Further, the position of the viewpoint is moved in various directions by the same operation of the touch panel. In addition, the movement of the viewpoint includes a movement by rotation around the central portion of the displayed map and a parallel movement with respect to the displayed map. In the case of rotation, a desired place can be viewed from various directions. In the case of parallel movement, for example, a route along the moving route can be confirmed.

  Further, as shown in FIG. 87, the image display device 301 shown in FIG. 73 can be similarly applied to a terminal device or the like disposed in a station, a convenience store, or the like.

  Further, in the above-described embodiment, the case where the panoramic image is arranged and displayed on the map image has been described. However, the present invention is not limited to this. The present invention can be widely applied to display an image, and further to displaying an image with a normal angle of view.

  In the above-described embodiment, the case where image data is acquired and displayed from the media database 112 configured by an HDD or the like has been described. However, the present invention is not limited to this. It can be widely applied to acquisition.

In the above-described embodiment, the case where a still image is applied to a captured image has been described. However, the present invention is not limited to this, and a moving image may be applied to the captured image.
In the above-described embodiment, the case where the present invention is applied to the remote commander and the car navigation device of the image display system has been described. However, the present invention is not limited to this, and can be widely applied to various image displays such as an image viewer.

  The present invention can easily grasp the geographical relationship between a plurality of photographed images, and can be applied, for example, when a predetermined photographing point is selected from a bird's-eye view image.

It is a perspective view which shows the structure of the audio-visual system of a double wall structure. It is a figure which shows the flame | frame structure of an inner wall. It is a figure for demonstrating the frame structure of an inner wall. It is the figure which expanded and showed a part of flame | frame. It is a figure which shows the welding state of a horizontal and a vertical frame. It is a figure for demonstrating how to fix the weight direction support to the horizontal frame. It is a figure for demonstrating how to fix the weight direction support to the horizontal frame. It is a figure which shows the front-back direction support of a plate-shaped unit. It is a figure for demonstrating the method of fixing to the frame part of the front-back direction support. It is a figure for demonstrating the connection method of a some frame part. It is a figure which shows the other structure of an audiovisual system. It is a figure which shows the other structure of an audiovisual stem. It is a figure for demonstrating the installation state of a projector. It is a figure which shows the installation state (comparative example) of a projector. It is a figure which shows the other structure of an audiovisual system. It is a figure which shows the other structure of an audiovisual system. It is a figure which shows the other structure of an audiovisual system. It is a figure which shows the entrance for rooms, and the entrance for spaces. It is a block diagram which shows the structure of the image display system for displaying an image on a background image display part and a bird's-eye view image display part. It is a figure for demonstrating omnidirectional image data. It is a figure for demonstrating image distortion. It is a figure for demonstrating correction | amendment of image distortion. It is a figure for demonstrating correction | amendment of image distortion. It is a figure for demonstrating a vanishing point. It is a flowchart which shows the process sequence of the control part of an image processing apparatus. It is a figure which shows the relationship between a bird's-eye view image and the display of a background image display part. It is a figure for demonstrating the image display of a background image display part. It is a figure for demonstrating correction | amendment of the image display position by a vanishing point. It is a flowchart which shows the process sequence of the control part of an image processing apparatus. It is a flowchart which shows the process sequence of the control part of an image processing apparatus. It is a figure for demonstrating the input in the display of a background image display part. It is the figure which showed typically the example of a display of the bird's-eye view mode and immersion mode in a bird's-eye view image display part. It is a state transition diagram which shows the display state transition of a bird's-eye view image display part. It is a figure which shows the parallel display of a surrounding image. It is a flowchart which shows the process sequence of the control part which concerns on the parallel display of a surrounding image. It is a figure for demonstrating the mark detection of an intersection. It is a figure for demonstrating the relationship between the direction of the sun, and an imaging | photography direction. It is a figure which shows the change of the luminance level by the direction of imaging | photography. It is a figure for demonstrating illumination control information. It is a flowchart which shows the process sequence of the control part which concerns on the process of illumination control information. It is a block diagram which shows the other structure of an image display system. It is a perspective view which shows the mode (irradiation of image light to a screen) inside the audiovisual room (room) surrounded by the inner wall, ceiling, and floor. It is a disassembled perspective view which shows the structure of the moving mechanism of a projector. It is a perspective view which shows the mirror moving mechanism of a projector. It is a block diagram which shows the structure of a projector. It is a figure for demonstrating the movement along the ceiling surface of the projector arrange | positioned in the space between a ceiling and a ceiling back. It is a figure for demonstrating the movement along the floor surface of the projector arrange | positioned in the space between a floor and a floor. It is a figure which shows the example of a projection of the image light from the projector arrange | positioned in the space between a ceiling and a ceiling back. It is a figure which shows the example of a projection of the image light from the projector arrange | positioned in the space between a floor and a floor. It is a perspective view which shows the mode (irradiation of image light to a ceiling and a floor | bed) inside the audiovisual room (room) enclosed with the inner wall, the ceiling, and the floor. It is a perspective view which shows the mode (image light is irradiated to a wall and a window) inside the audio-visual room (room) surrounded by an inner wall, a ceiling, and a floor. It is a perspective view which shows the mode (irradiation of image light to a table surface from the floor side) inside the audiovisual room (room) surrounded by the inner wall, ceiling, and floor. It is a figure which shows the structure of the table with which image light is irradiated. It is a perspective view which shows the mode (irradiation of image light to a table surface from the ceiling side) inside the audiovisual room (room) surrounded by the inner wall, ceiling, and floor. It is a figure which shows the partial structure of the audiovisual room system which displays an image | video on a screen with one projector in two audiovisual rooms (rooms) continuous in an up-down direction (vertical direction). It is a figure which shows a part structure of the audiovisual room system which displays an image | video on a screen and a table with one projector in two audiovisual rooms (rooms) continuous in the up-down direction (vertical direction). It is a figure which shows a partial structure of the audiovisual room system which displays an image | video with one projector of two audiovisual rooms (room) continuous in the left-right direction (horizontal direction). It is a perspective view which shows the mode (irradiation of image light to a floor through the belt-shaped projection window of a ceiling) inside the audiovisual room (room) surrounded by the inner wall, the ceiling, and the floor. It is a perspective view which shows the inside (the image light is irradiated to the opposing inner wall through the strip | belt-shaped projection window of an inner wall) of the audiovisual room (room) enclosed with the inner wall, the ceiling, and the floor. It is a figure which shows the structure (arrangement | positioning a projector on a floor) of a remote commander. It is a perspective view which shows the table which comprises a remote commander. It is the side view and bottom view of the table base which comprises a table. It is a figure which shows the state which irradiates image light from the projector on the screen arrange | positioned on a table surface, and displays the image | video on the said screen. It is a figure which shows the other structure (arrangement | positioning a projector in the space between a floor and a floor) of the remote commander. It is a figure which shows the other structure (The projector is arrange | positioned on the floor and the lens for optical path shortening is used). It is a figure which shows the other structure (A projector is arrange | positioned in the space between a floor and a floor, and a light path is bent with a mirror). It is a figure which shows the other structure of a remote commander (a projector is arrange | positioned in the space between a floor and a floor, a light path is bent with a mirror, and also the lens for optical path shortening is used). It is a perspective view which shows the mode (the liquid crystal film which comprises a screen is a translucent state) inside the audiovisual room where the remote commander by a table is arrange | positioned. It is a perspective view which shows the mode (the liquid crystal film which comprises a screen is a transparent state) inside the audiovisual room where the remote commander by a table is arrange | positioned. It is a figure which shows the other structure (A sensing part consists of an imaging device etc. and arrange | positions a projector on a floor). It is a figure which shows the other structure of a remote commander (a sensing part consists of an imaging device etc., and arrange | positions a projector in the space between a floor and a floor). It is a figure which shows the other structure (The sensing part consists of an imaging device etc. and the drag | drug etc. of a moving body are detected) of a remote commander. It is a block diagram which shows the structure of the image processing apparatus for displaying the bird's-eye view image by which the picked-up image is arrange | positioned. It is a figure for demonstrating the panoramic image as a picked-up image. It is a figure for demonstrating the process of an omnidirectional image conversion part. It is a flowchart which shows the procedure of the display process in the control part which comprises an image display apparatus. It is a flowchart which shows the procedure of the landscape image display process in a display process. It is a flowchart which shows the procedure of the viewpoint conversion process in a display process. It is a flowchart which shows the procedure of the scene switching process in a display process. It is a figure for demonstrating arrangement | positioning of a landscape image display board and the inclination adjustment by a viewpoint at the time of arrange | positioning a panoramic image to a map image. It is a figure for demonstrating extraction of the one part area | region corresponding to the viewpoint from a panoramic image. It is a figure which shows the example of a display of the image which has arrange | positioned the panoramic image on the map image. It is a figure for demonstrating the viewpoint movement process of zooming in and zooming out. It is a figure which shows the example of a display of zoom-in in the bird's-eye view image display part, and zoom-out. It is a figure which shows the example of a bird's-eye viewpoint movement process. It is a figure which shows an example of the remote commander by which a bird's-eye view image display part is arrange | positioned on a table surface. It is a figure which shows the example of application to a car navigation apparatus. It is a figure which shows the example of application to terminal devices, such as a station and a convenience store.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Audiovisual room, 2 ... Inner wall, 2A ... Ceiling, 2B ... Floor, 3 ... Outer wall, 3A ... Back of ceiling, 3B ... Under floor, 4 ... Vertical frame 5 ... Horizontal frame, 6A-6D ... Plate unit, 8 ... Vibrator, 9 ... Diaphragm, 10 ... Projector, 11 ... Bay, 21, 31, 41, 51, 61 ... Audio-visual room, 24 ... Frame part, 25 ... Load direction support, 26 ... Front / rear direction support, 32 ... Mirror, 62 ... Room entrance, 63 ... Door, 64 ... entrance for space, 101 ... image display system, 102 ... data processing device, 111 ... image display device, 112 ... media database, 113 ... control unit, 113A ..Display information selection unit, 113B ... Image processing unit, 113C ... Sound field Management unit 113D ... Illumination information generation unit 113E ... Illumination information selection unit 115 ... Background image display unit 116 ... Remote commander 117 ... Bird's eye image display unit 118 ... Touch panel, 119 ... controller for illumination, 120 ... illumination, 122 ... speaker, 141 ... image display system, 142 ... image display device, 143 ... broadband network, 144A-144C ... Media database 201, 201A to 201H: remote commander, 202 ... table, 203 ... table surface, 204 ... leg, 206 ... screen, 206A ... liquid crystal panel, 207 ... Touch panel 208 ... Table base 210 ... Projector 212 ... Transmission / reception unit 211 ... Signal processing 213 ... projection window, 222, 245 ... optical path shortening lens, 234 ... mirror, 266 ... imaging device, 267 ... image processing unit, 267A, 301 ... image display Device 304A ... Omnidirectional image conversion unit 304B ... Map image / panoramic image arrangement unit 304C ... Viewpoint coordinate conversion unit 401 ... Audio-visual room 402 ... Projector 402a ... Controller, 402b ... Movement mechanism, 402c ... Projection mechanism, 402d ... Mirror moving mechanism, 402e ... Remote control receiving unit, 402f ... Video receiving unit, 402g ... Correction processing unit, 402h ..Position detection mechanism, 404 ... projection window, 405 ... screen, 430 ... window, 451 ... base, 452,454 ... rail, 453,455 ... 461 ... Cylinder, 462 ... Mirror, 463 ... Two-axis motor, 471 ... Remote commander, 472 ... Video equipment

Claims (8)

An image processing device that performs image processing for simultaneously displaying a bird's-eye image and a captured image captured at a capturing point specified on the bird's-eye image,
An image placement unit that obtains a display image by placing a shot image taken at the shooting point at the shooting point of the bird's-eye view image ,
The image placement section, an image, characterized in that arranged so as to face the captured image to the viewpoint, the further bird's-eye image in which the photographed image is arranged to convert the image seen from the viewpoint Ru obtain the display image Processing equipment. The captured image is a wide-angle image,
An image cutout unit that cuts out a partial region corresponding to the viewpoint from the wide-angle image;
The image processing apparatus according to claim 1 , wherein the image arrangement unit arranges the partial area cut out by the image cutout unit at the shooting point of the bird's-eye view image. The image processing apparatus according to claim 1 , further comprising a user operation unit that performs an operation for a user to change the position of the viewpoint. An image acquisition unit that acquires the bird's-eye image and the shot image shot at a shooting point specified on the bird's-eye image from a database;
The image processing apparatus according to claim 1, further comprising: a user operation unit that performs an operation of changing the bird's-eye image acquired by the image acquisition unit and the captured image. A database that stores a plurality of types of bird's-eye images and shot images taken at shooting points specified on each bird's-eye image;
A predetermined bird's-eye image and a photographed image taken at a photographing point specified on the predetermined bird's-eye image are obtained from the database, and a photographed image taken at the photographing point is taken at the photographing point of the predetermined bird's-eye image. An image processing unit for obtaining a display image by arranging
An image display unit for displaying an image obtained by the image processing unit ,
The image processing section, an image, characterized in that arranged so as to face the captured image to the viewpoint, the further bird's-eye image in which the photographed image is arranged to convert the image seen from the viewpoint Ru obtain the display image Display device. An image processing method for simultaneously displaying a bird's-eye view image and a photographed image photographed at a photographing point specified on the bird's-eye view image,
An image placement step of placing a photographed image photographed at the photographing point at the photographing point of the bird's-eye image so as to face the viewpoint;
An image conversion method comprising: an image conversion step of obtaining a display image by converting the bird's-eye view image in which the captured image is disposed at the imaging point in the image placement step to an image viewed from the viewpoint. In order to simultaneously display a bird's-eye image and a shot image taken at a shooting point specified on the bird's-eye view image,
An image placement step of placing a photographed image photographed at the photographing point at the photographing point of the bird's-eye image so as to face the viewpoint;
An image conversion method for causing a computer to execute an image processing method including: an image conversion step of converting the bird's-eye view image in which a captured image is arranged at the shooting location in the image placement step into an image viewed from the viewpoint to obtain a display image program. In order to simultaneously display a bird's-eye image and a shot image taken at a shooting point specified on the bird's-eye view image,
An image placement step of placing a photographed image photographed at the photographing point at the photographing point of the bird's-eye image so as to face the viewpoint;
An image conversion method for causing a computer to execute an image processing method including: an image conversion step of converting the bird's-eye view image in which a captured image is arranged at the shooting location in the image placement step into an image viewed from the viewpoint to obtain a display image A computer-readable recording medium on which a program is recorded.