JP2000341568A - Device, system and method for image pickup - Google Patents

Abstract

PROBLEM TO BE SOLVED: To inexpensively provide moving image of super wide angle to be at high resolution at the central part of an image by forming the surface of a convex mirror into center concavity shape, forming the central part into a low curvature surface and forming the peripheral part into a high curvature surface. SOLUTION: A block (center concavity mirror block) 1 is composed of a glass or transparent resin, a central concavity mirror 2 is provided, and the optical system of a center concavity visual image pickup device is constituted while at least providing the center concavity mirror 2, the center concavity mirror block 1, an optical member 4 and a sensor part 5. The surface of this center concavity mirror 2 is formed into a center concavity, the central part is formed into low curvature surface and the peripheral part is formed into high curvature surface. As a specific example, it is preferable that the surface of the center concavity mirror 2 be made into a curved surface constituted by rotating a curved line, which is expressed by Y=aX4 (a is a constant) with the central axis as Y axis, around the Y axis. The central concavity mirror 2 is installed at a position, where the central axis deviates from the central axis of light specified by the install position of the sensor part 5.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus and an image pickup method for obtaining a panoramic moving image, and is particularly suitable for application to a communication processing system for remotely using a digitally processed moving image signal.

[0002]

2. Description of the Related Art In recent years, so-called machine vision has been expected as a suitable automatic control device for obtaining a moving image of a complex and dynamically changing external world. As part of this application of machine vision, a foveated vision system for obtaining wide-angle images has been studied.

As a specific example of such a foveated vision system, “Active S” by Kuniyoshi et al.
"tereo Vision System with Foveated Wide Lenses", wherein a foveated wide-angle lens is disclosed. However, this lens has a relatively narrow field of view of 120 °,
There is a disadvantage that it is relatively complicated and expensive because of the structure of 11 elements in 10 groups.

As another specific example, “A foveated retina-like sensor using CCD tech” by Spiegel et al.
Here, there is disclosed a retina-like sensor in which a photoreceptor is arranged so as to have a high density at a central portion so as to be concentric and to have a low density at a peripheral portion. However, this sensor is expensive and unsuitable for mass production because a large number of photoreceptors are provided in a special arrangement as described above, and has the disadvantage that the latest various sensor manufacturing techniques cannot be used.

As another specific example, Greguss
U.S. Pat. No. 4,566,763, which discloses an image forming device capable of obtaining an image with a wide field of view.
g block) is disclosed. However, the resolution of this image forming body is not taken into consideration, and it is not possible to photograph the central part of the image with high definition. This is true even if a mirror having a conical surface, a spherical surface, a hyperbolic surface, or a parabolic surface is used as a mirror for obtaining a reflected moving image. It is difficult to get.

[0006]

As described above, in these conventional techniques, (1) the field of view of the optical system is narrow. (2) A high-density moving image cannot be captured at the central portion. (3) The shape and structure of the optical system are complicated and expensive. There was a serious drawback.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an inexpensive imaging apparatus, an imaging system, and an imaging method capable of obtaining an ultra-wide-angle moving image having a high resolution at a central portion of the image. An object of the present invention is to remove a distortion of a peripheral portion of the moving image and perform panorama of the moving image, which has not been performed.

[0008]

According to the present invention, there is provided an imaging apparatus comprising: a convex mirror; and a sensor section for capturing a moving image reflected from the convex mirror, wherein the surface of the convex mirror has a central fovea. The central portion has a low curvature surface and the peripheral portion has a high curvature surface.

In one aspect of the imaging apparatus of the present invention, the surface of the convex mirror is obtained by rotating a curve represented by Y = aX ⁴ (a: constant) around the Y axis with its central axis as the Y axis. It is a curved surface.

In one aspect of the imaging apparatus of the present invention, the convex mirror is installed at a position where a central axis thereof is shifted from a central axis of light defined by an installation position of the sensor unit.

According to one aspect of the image pickup apparatus of the present invention, a panoramic moving image is corrected by correcting a moving image signal of a moving image reflected from the convex mirror processed by the sensor unit and removing distortion at a peripheral portion of the moving image. An image processing unit is provided.

An imaging method according to the present invention is a method of imaging a moving image reflected from a convex mirror by a sensor unit, wherein the surface has a fovea-like shape, a central portion has a low curvature surface, and a peripheral portion has a high curvature. After obtaining a reflection image in which the central part has a high resolution and the peripheral part has a low resolution by the convex mirror taken in a plane, the sensor unit performs signal processing on the reflection moving image to generate a moving image signal, The moving image signal is corrected in accordance with a parameter determined in advance from a relative positional relationship between the convex mirror and the sensor unit, and a distortion of a peripheral portion of the moving image is removed to form a panoramic moving image.

In one embodiment of the imaging method of the present invention, the surface of the convex mirror is obtained by rotating a curve represented by Y = aX ⁴ (a: constant) around the Y axis with its central axis as the Y axis. It is a curved surface.

In one embodiment of the imaging method according to the present invention, the convex mirror is installed at a position where a central axis thereof deviates from a central axis of light defined by an installation position of the sensor unit.

An imaging system according to the present invention has a convex mirror whose surface has a fovea-like shape, a central portion having a low curvature surface and a peripheral portion having a high curvature surface, and a sensor for capturing a moving image reflected from the convex mirror. A first device having a first unit and a second device connected to the first device via a network and capable of remotely controlling a shooting state of the first camera unit;
The moving image captured by the device is transmitted to the second device, and the moving image is displayed on the second device.

In one embodiment of the imaging system of the present invention,
The surface of the convex mirror is a curved surface obtained by rotating a curve represented by Y = aX ⁴ (a: constant) around the Y axis with its central axis as the Y axis.

In one embodiment of the imaging system of the present invention,
The convex mirror is installed at a position where its central axis deviates from the central axis of light defined by the installation position of the sensor unit.

In one aspect of the imaging system of the present invention, a panoramic moving image is corrected by correcting a moving image signal of a moving image reflected from the convex mirror processed by the sensor unit and removing distortion at a peripheral portion of the moving image. An image processing unit is provided.

[0019]

According to the present invention, the surface of the convex mirror has a fovea-like shape, the central portion has a low curvature surface, and the peripheral portion has a high curvature surface.
A moving image having a high resolution in the central region and a low resolution in the peripheral region is obtained. Then, by correcting the peripheral part of the moving image, it becomes possible to obtain an ultra-wide-angle panoramic video that does not cause a sense of incongruity to human vision.

[0020]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments to which the present invention is applied will be described below in detail with reference to the drawings.

FIG. 1 is a schematic diagram showing an imaging system including a foveated visual imaging device according to the present embodiment. In the figure, reference numeral 2 denotes a convex mirror having a foveated curved surface formed of metal (hereinafter referred to as a foveated mirror), and 1 is a block made of glass or transparent resin and provided with a foveated mirror 2 (hereinafter, referred to as a block). , 3) a light transmitting surface, 4 an optical member such as a convex lens for imaging, and 5 a reflection moving image from the fovea mirror 2 via the optical member 4. It is a sensor unit. The optical system of the foveated visual imaging device is provided with at least the foveated mirror 2, the foveated mirror block 1, the optical member 4, and the sensor unit 5.

Further, reference numeral 6 denotes a distortion-removed image processing unit comprising a memory such as a CPU and a RAM, and a ROM for holding programs, and 7 is connected to the image processing unit 6 via a network and is located remotely. (The image displayed on the second device). With respect to the second device 7, the above 1 to 6
Is a foveated visual imaging device (first device). Reference numeral 8 denotes a high-resolution portion corresponding to the central portion in the image of the second device 7. The system includes the first device and the second device.

Although the second device 7 is not described in detail, an existing personal computer (P) having a function of connecting to a network and displaying hypertext data, etc.
C), a network PC, a workstation, and the like.

The surface of the foveal mirror 2 has a fovea-like shape, with a central portion having a low curvature surface and a peripheral portion having a high curvature surface. As a specific example, the surface of the foveated mirror 2 may be a curved surface obtained by rotating a curve represented by Y = aX ⁴ (a: constant) around the Y axis with its central axis as the Y axis. preferable.

The foveal mirror 2 is installed at a position where its central axis deviates from the central axis of light defined by the installation position of the sensor unit 5.

The operation of the foveated visual imaging device will be described below. First, an optical path that enters and reflects on the fovea mirror 2 will be described with reference to FIG. This figure is a cross-sectional view along the horizontal direction of the optical system of the foveated visual imaging device. For convenience, the central axis of the foveated mirror 2 and the central axis of light defined by the installation position of the sensor unit 5 coincide with each other. It is drawn as follows.

In FIG. 2, reference numeral 11 denotes a point 120 from the central axis L.
12 shows incident light from a central part 20 ° apart from the central axis L, 13 shows incident light from a central part which is 20 ° away from the central axis L, 13 shows reflected light of the incident light 11 reflected by the fovea mirror 2,
Represents reflected light of the incident light 12 reflected by the fovea mirror 2.

The incident light 11 is reflected by the fovea mirror 2 and becomes reflected light 13. The reflected light 13 passes through the optical member 4 and enters the sensor unit 5. Further, the incident light 12 is reflected by the fovea mirror 2 and becomes a reflected light 14. The reflected light 14 passes through the optical member 4 and enters the sensor unit 5.

FIG. 3 is a diagram showing characteristics of the foveal mirror 2 and the optical member 4 which are components of the optical system of the foveated visual imaging device. In FIG. 3, reference numeral 41 denotes an effective area of the sensor unit 5, and reference numeral 42 denotes a high-resolution portion of the central portion of the image. For the incident light in the directions of °, 10 °, and -10 °, the trajectory on the sensor unit 5 is defined as V0,
V10 and V-10 indicate the trajectories on the sensor unit 5 for the incident light in the directions of 0 °, 10 °, and −10 ° from the central axis in the vertical direction, respectively. Therefore, the reflected light 13 that has passed through the optical member 4 is incident on the locus H120 in FIG. 3, and the reflected light 14 is incident on the locus H20.

FIG. 4A shows an example of a moving image (original image) captured by the sensor unit 5 in this manner. Corresponding to the surface shape of the fovea mirror 2, an image with less distortion is formed at the central portion of the sensor section 5, and an image with more distortion is formed at the peripheral portion. As shown in the figure, an image in which four persons are at the table, the person located at the central part is photographed without distortion at high resolution, while the other three persons are photographed with distortion. Is bigger as you go around.

Then, the image signal of the moving image as shown in FIG. 4A is corrected by the image processing section 6 by the following panorama image forming process, and as shown in FIG. A moving image is obtained.

Hereinafter, the panoramic imaging process will be described in detail. To obtain a restored panoramic image (FIG. 4B), the pixels on the panoramic image are converted to the original image (FIG. 4B).
In (a)), which pixel corresponds to is calculated, and corresponding pixel position conversion processing of two images is performed. Specifically, as shown in FIG. 3, when the conversion table is defined every 10 °, the optical system is installed at the center of the spherical surface on which the mesh is drawn every 10 °, and the spherical surface is imaged. Measured by

For example, assuming that the intersection of V20 and H-30 is a point of 119 pixels upward from the center of the sensor unit 5 and 165 pixels to the left, the pixel address value is stored in the conversion table. In the case of every 10 °, 26 × 13 = 325 pixel address pairs are stored in the conversion table.

Here, assuming that the positions of the two images are (w, v), (w, v) = T (x / 10, y / 10) x: horizontal angle; -120 ≦ x ≦ 120 y : Vertical angle; −60 ≦ y ≦ 60. In the present embodiment, all pixels of the panoramic restored image (FIG. 4B) are used for the original image (FIG. 4B).
In (a)), the corresponding pixel is calculated, and the pixel value of the restored image is calculated as follows.

(W1, v1) = T (x / 10, y / 10) (w2, v2) = T ((x / 10) +1, y / 10) (w3, v3) = T ((x / 10 ) +1, (y / 10)
+1) (w4, v4) = T (x / 10, (y / 10) +1) Assuming that m = modulo (x, 10) and n = modulo (y, 10), w = [｛w1 * (10−m) ) + W2 * m｝ * (10−
n) + {w4 * (10-m) + w3 * m} * n] / 10
0 v = [{v1 * (10−n) + v4 * n} * (10−
m) + {v2 * (10−n) + v3 * n｝ * m] / 10
0

As described above, the addresses of the corresponding pixels of the two images are obtained, and an ultra-wide-angle panoramic video without distortion is obtained for the entire image. If the pixel value is taken as the nearest pixel value without performing interpolation, the pixel value becomes D (x, y).
D (x, y) = D (w, v). In the case where a finer angle is specified for each angle without using a conversion table for every 10 °, 10, 100 in the above formula group may be changed according to the angle.

As described above, according to the foveated visual imaging device of the present embodiment, the surface of the foveated mirror 2 has a low curvature surface at the central portion and a high curvature surface at the peripheral portion. According to the shape of the fovea mirror 2, a moving image having a high resolution at the central portion and a low resolution at the peripheral portion can be obtained. Then, by correcting the peripheral part of the moving image, it becomes possible to obtain an ultra-wide-angle panoramic video that does not cause a sense of incongruity to human vision.

[0038]

According to the present invention, the central portion has a low curvature, and the peripheral portion has a high curvature. The method of taking an image reflected on a convex curved mirror has a wider angle and a lower cost than the conventional one. It is possible to configure an imaging apparatus (imaging method) having the characteristics of foveated vision with high resolution and low peripheral area resolution.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing a foveated visual imaging device according to an embodiment.

FIG. 2 is a cross-sectional view of the optical system of the foveated visual imaging device taken along a horizontal direction.

FIG. 3 is a characteristic diagram showing characteristics of a foveated mirror and an optical member.

FIG. 4 is a schematic diagram showing an original image (a) and a restored image (b) obtained by the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Foveated mirror block 2 Foveated mirror 3 Transmissive surface 4 Optical member 5 Sensor part 6 Distortion removal image processing part

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/18 H04N 7/18 Z F term (Reference) 2H054 AA00 BB00 2H087 KA01 TA03 TA06 2H101 FF00 5C022 AA00 AB51 AB65 AB68 AC51 AC54 5C054 AA01 CC06 DA06 EA01 ED11 FD02 FD03 HA01

Claims (11)

[Claims] 1. An imaging apparatus comprising: a convex mirror; and a sensor unit for capturing a moving image reflected from the convex mirror, wherein the convex mirror has a fovea-like surface.
An imaging apparatus wherein a central portion has a low curvature surface and a peripheral portion has a high curvature surface. 2. The surface of the convex mirror is a curved surface obtained by rotating a curve represented by Y = aX ⁴ (a: constant) around the Y axis with its central axis as the Y axis. The imaging device according to claim 1. 3. The imaging device according to claim 1, wherein the convex mirror is installed at a position where a central axis thereof is deviated from a central axis of light defined by an installation position of the sensor unit. 4. An image processing means for correcting a moving image signal of a reflected moving image from the convex mirror processed by the sensor unit and removing a distortion of a peripheral portion of the moving image to form a super wide-angle panoramic moving image. Claim 1 characterized by comprising:
4. The imaging device according to any one of 3. 5. An imaging method for imaging a moving image reflected from a convex mirror by a sensor unit, wherein the surface has a fovea-like shape, a central portion has a low curvature surface, and a peripheral portion has a high curvature surface. After obtaining a reflection image in which the central part has high resolution and the peripheral part has low resolution by the convex mirror, the sensor unit performs signal processing on the reflection moving image to generate a moving image signal, and the convex mirror and the sensor An imaging method comprising: correcting a moving image signal in accordance with a parameter determined in advance from a relative positional relationship with a unit, removing distortion in a peripheral portion of the moving image, and forming a super wide-angle panoramic moving image. 6. The surface of the convex mirror is a curved surface obtained by rotating a curve represented by Y = aX ⁴ (a: constant) around the Y axis with its central axis as the Y axis. The imaging method according to claim 5, wherein: 7. The imaging method according to claim 5, wherein the convex mirror is installed at a position where a central axis thereof is shifted from a central axis of light defined by an installation position of the sensor unit. 8. A convex mirror whose surface has a fovea-like shape, a central portion having a low curvature surface and a peripheral portion having a high curvature surface,
A first device having a sensor unit that captures a moving image reflected from the convex mirror, and a second device that is network-connected to the first device and that can remotely control a shooting state of the first camera unit. Wherein the moving image captured by the first device is transmitted to the second device, and the moving image is displayed on the second device. 9. The surface of the convex mirror is a curved surface obtained by rotating a curve represented by Y = aX ⁴ (a: constant) around the Y axis with its central axis as the Y axis. The imaging system according to claim 8, wherein: 10. The imaging system according to claim 8, wherein the convex mirror is installed at a position where a central axis thereof is deviated from a central axis of light defined by an installation position of the sensor unit. 11. An image processing means for correcting a moving image signal of a moving image reflected from the convex mirror processed by the sensor unit and removing a distortion of a peripheral portion of the moving image to form a panoramic moving image. The imaging system according to claim 8, wherein: