JP2003069879A - Image input device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the burden of a user by relieving a restriction to the presenting position of an object by selecting a method optimum to obtain a high quality image in accordance with the field of view of each camera. SOLUTION: A single document is photographed by two digital cameras (step S2), and a circumscribed polygon L of a set S of correspondence points on the original is calculated in each image (step S9). When the entire document is decided to be included in the each image (step S11) according to whether or not correspondence points other than the set S exist outside the circumscribed polygon L (step S10), an image is generated by overlapping images (step S13), and when a portion of the original is decided not to be included in each image (step S12), an image is generated by connecting images (step S14).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image input device and a program.

[0002]

2. Description of the Related Art Conventionally, there are roughly two types of techniques for obtaining a high-resolution image from a plurality of images of a single subject having insufficient resolution.

One is a technique in which partial images are divided and photographed while partially overlapping each other, and the images are joined together.

The other is a technique for taking a plurality of images while shifting the entire image in subpixel units and combining the images to improve the resolution.

An example of the former is, for example, Japanese Patent Laid-Open No. 2000-115.
639 and Japanese Patent Laid-Open No. 2000-348159.

The technique disclosed in Japanese Unexamined Patent Application Publication No. 2000-115639 uses parameters such as the posture of a camera when a plurality of images that partially overlap each other are stitched together to obtain a high-resolution image. The image is joined with high accuracy.

Further, the technique disclosed in Japanese Patent Laid-Open No. 2000-348159 arranges reduced images in a pyramid shape when a plurality of images overlapping a part of the images are stitched together to obtain a high-resolution image. By calculating the amount of movement or the like during that time, the images are joined with high accuracy.

Examples of the latter include those disclosed in Japanese Patent Laid-Open Nos. 2000-217032 and 2000-224460. Such a technique acquires a plurality of images obtained by photographing the substantially same range by releasing the shutter a plurality of times, detects pixel shifts at the sub-pixel level, and then performs alignment.
By synthesizing the images, an image with high resolution / quality is obtained.

[0009]

However, in the technique of joining the images, it takes a lot of time to control the camera, and in the technique of synthesizing the images, it is necessary to set a certain distance between the camera and the manuscript. is there. Therefore, when a plurality of cameras are fixed and a high-resolution image is to be generated from the obtained plurality of images, there are many restrictions on the position of presenting the document regardless of which of the above-mentioned means is used, which facilitates photographing. There is a problem that it becomes a factor that prevents you from doing it.

An object of the present invention is to reduce the burden on the user by relaxing the constraint on the presentation position of the subject by selecting the optimum method for obtaining a high quality image according to the field of view of each camera. That is.

It is an object of the present invention to produce high quality images when the field of view of each camera includes the entire subject.

It is an object of the present invention to produce a high quality image when the field of view of each camera contains only a portion of the subject.

An object of the present invention is to produce a high quality image for each portion of the image.

An object of the present invention is to prompt the user to take another image when the images cannot be combined with each other.

An object of the present invention is to enable a visual field to be judged based on corresponding points.

An object of the present invention is to make it possible to judge that the entire subject is included in the image or a part of the subject is not included in the image based on the corresponding points.

[0017]

According to a first aspect of the present invention, a plurality of cameras for photographing a subject and a visual field for judging the visual field of each camera for the subject based on each image photographed by each camera. The image input device includes a determination unit and an image generation unit that generates a single image of the subject from the captured images by using a different method according to the determination.

Therefore, an image can be generated by selecting an optimum method for obtaining a high quality image according to the field of view of each camera, so that the constraint on the presentation position of the subject is relaxed and the burden on the user is reduced. be able to.

According to a second aspect of the present invention, in the image input device according to the first aspect, the image generating means is configured to, when the field of view of each of the cameras includes the entire subject, based on the determination. A single image of the subject is generated by a method of superimposing the captured images.

Therefore, it is possible to generate a high quality image when the field of view of each camera includes the entire subject.

According to a third aspect of the present invention, in the image input apparatus according to the first or second aspect, the image generation means includes the field of view of each camera including only a part of the subject according to the determination. If so, a single image of the subject is generated by a method of joining the photographed images.

Therefore, it is possible to generate a high quality image when the field of view of each camera includes only a part of the subject.

According to a fourth aspect of the present invention, in the image input apparatus according to the first or second aspect, the image generation means includes the field of view of each of the cameras based on the determination and includes only a part of the subject. In the case of a single field of view of each of the cameras, a method of superimposing the captured images is used for a portion where the fields of view of the cameras overlap, and a method of stitching the captured images is used for a portion other than the single field of the subject. An image is generated.

Therefore, it is possible to generate a high quality image for each portion of the image.

According to a fifth aspect of the present invention, in the image input device according to any one of the first to fourth aspects, the visual field determining means determines that there is no overlapping portion in the visual fields of the cameras with respect to the subject. In this case, there is provided a notifying means for notifying the camera and the subject of separation from the previous photographing, and a message for prompting the subject to re-photograph the subject.

Therefore, when the images cannot be combined with each other, the user can be prompted to take another image.

According to a sixth aspect of the present invention, in the image input device according to any one of the first to fifth aspects, the visual field determining means sets x as a distance to a corresponding point closest to the camera. In this case, a fixed threshold value α is used and “x + α”
The field of view is determined by determining that the corresponding points within the range are the set S of corresponding points of the subject.

Therefore, the visual field can be determined by the corresponding points.

According to a seventh aspect of the present invention, in the image input apparatus according to the sixth aspect, the visual field determining means includes a set S of corresponding points of the subject surrounded by a set of other corresponding points. When it is present, it is determined that the entire subject is included in the image, and when not, it is determined that a part of the subject is not included in the image.

Therefore, it can be determined from the corresponding points that the entire subject is included in the image or a part of the subject is not included in the image.

The invention described in claim 8 uses a visual field determination process of determining the visual field of each camera with respect to the subject based on each image obtained by photographing the subject with a plurality of cameras, and a different method according to this determination. It is a computer-readable program that causes a computer to execute an image generation process for generating a single image of the subject from each of the captured images.

Therefore, it is possible to select an optimum method for obtaining a high quality image according to the field of view of each camera and generate the image, so that the constraint on the presentation position of the subject is relaxed and the burden on the user is reduced. be able to.

According to a ninth aspect of the present invention, in the program according to the eighth aspect, in the image generation processing, when the field of view of each camera includes the whole of the subject by the determination, each of the photographing is performed. A single image of the subject is generated by a method of overlapping images.

Therefore, it is possible to generate a high quality image when the field of view of each camera includes the entire subject.

According to a tenth aspect of the present invention, in the program according to the eighth or ninth aspect, in the image generation processing, when the field of view of each camera includes only a part of the subject according to the determination. , A single image of the subject is generated by a method of joining the captured images together.

Therefore, it is possible to generate a high quality image when the field of view of each camera includes only a part of the subject.

The invention according to claim 11 is the program according to claim 8 or 9, wherein in the image generation processing, when the field of view of each camera includes only a part of the subject by the determination. , A method of superimposing the photographed images is used for a portion where the fields of view of the cameras overlap, and a method of joining the photographed images is used for the other portions to obtain a single image of the subject. To generate.

Therefore, it is possible to generate a high quality image for each part of the image.

[0039]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described.

FIG. 1 is a block diagram showing the overall configuration of an image input device 1 according to an embodiment of the present invention. Figure 1
As shown in FIG. 1, the image input device 1 is a computer such as a PC, performs various calculations, and centrally controls each unit of the image input device 1, and various ROMs and Rs.
A bus 4 is connected to the memory 3, which is a storage device made of AM.

A magnetic storage device 5 such as a hard disk, an input device 6 including a mouse and a keyboard, a display device 7, and a storage medium 8 such as an optical disk are read on the bus 4 via a predetermined interface. The storage medium reading device 9 and a plurality of, in this example, two digital cameras 10, 11 are connected.

A predetermined communication interface 13 for communicating with the network 12 such as the Internet may be connected. The storage medium 8 is C
Various media such as optical discs such as D and DVD, magneto-optical discs, and floppy (registered trademark) discs can be used. Further, as the storage medium reading device 9, specifically, an optical disc device, a magneto-optical disc device, a floppy disc device or the like is used depending on the type of the storage medium 8.

The magnetic storage device 5 stores an image input program that realizes the program of the present invention. This image input program is read from the storage medium 8 by the storage medium reading device 9, or is downloaded from the network 12 such as the Internet.
It is installed in the magnetic storage device 5. By this installation, the image input device 1 becomes operable. The image input program may be a part of a specific application software. Further, it may be one that operates on a predetermined OS.

FIG. 2 is a front view showing an example of the external appearance of the image input apparatus 1. The image input device 1 includes a main body 21 and an input / output device 22 connected to the main body 21. The input / output device 22 includes the display device 7 and the digital cameras 10 and 11 provided in, for example, the upper two corners of the display device 7. In FIG. 2, the screen of the display device 7 faces the front side, and the digital cameras 10 and 11 face the back side. Of course, the positions and the numbers of the digital cameras 10 and 11 are not limited to this.

The contents of the processing performed by the image input apparatus 1 based on the image input program will be described below.

FIG. 3 shows two digital cameras 10 and 11.
3 is a plan view of the input / output device 22 for explaining the visual field of FIG. The fields of view of the digital cameras 10 and 11 are shown by imaginary lines. In the case of using the technique disclosed in the above-mentioned Japanese Patent Laid-Open No. 2000-217032, the entire document serving as a subject is included in any field of view of the two digital cameras 10 and 11 (within the range shown by hatching in FIG. 4). Therefore, when the manuscript includes a portion that can be included in only one field of view of the two digital cameras 10 and 11 (the range is shown by hatching in FIG. 5), The image quality will be imperfectly improved.

On the other hand, when the technique disclosed in Japanese Patent Laid-Open No. 2000-348159 is used, the two digital cameras 1
When the entire original is included in any of the 0 and 11 fields of view (within the shaded area in FIG. 4), the size of the original image cannot be increased, so that the resolution can be substantially improved. I couldn't, and I had two digital cameras in the manuscript.
It is necessary to include a portion that falls within only one of the visual fields 0 and 11 (the range is shown by hatching in FIG. 5).

Therefore, in the image input apparatus 1, for example, a method of superposing a plurality of picked-up images as disclosed in Japanese Patent Laid-Open No. 2000-217032, and, for example, Japanese Patent Laid-Open No. 2000-348159.
The technique of joining a plurality of captured images as disclosed in Japanese Patent Laid-Open Publication is used properly according to the position of the document, so that the limitation when the user presents the document is reduced.

Therefore, the image input apparatus 1 specifically performs the processing shown in the flowchart of FIG. That is,
The user uses the digital camera 10, 11
If it is presented before, the operation of the input device 6 is instructed to execute the image input. When the execution of the image input is instructed by the operation of the input device 6, the CPU 2 (step S1
Y), the document of the subject is photographed by the digital cameras 10 and 11 (step S2). At this time, it is desirable that the digital cameras 10 and 11 simultaneously capture images as much as possible. The digital image data captured by the digital cameras 10 and 11 are stored in the memory 3 (step S3).

Then, the corresponding points between the two photographed images are calculated (step S4), and it is judged whether or not the entire original is included in the image (steps S8 to S12).

First, the corresponding points between the two images are calculated (step S4). Regarding the calculation of the corresponding points, many methods have been conventionally proposed, but any method or a combination of a plurality of methods may be used. If the corresponding points have been calculated (Y in step S5),
The distances from the digital cameras 10 and 11 to the corresponding points are obtained by the principle of triangulation (step S6). Here, when the corresponding points are not obtained at all (N in step S5), the document is the digital camera 1 indicated by the diagonal lines in FIG.
Since it is presented at a position outside the range of at least one field of view of 0 and 11, a message is displayed to instruct the document and the digital cameras 10 and 11 to be re-photographed apart from the previous photographing. The user is notified by the device 7 or the like (step S7).

In this method, the original is compared with other objects,
It is assumed that the image is presented at the position closest to the digital cameras 10 and 11. Therefore, if the distance to the corresponding point closest to the digital cameras 10 and 11 is x, then "x + α"
It is assumed that the corresponding points in the range of (α is a fixed threshold value) are a set S of corresponding points on the document (step S8). In the image P shown in FIGS. 7 and 8, S is the document. The points that are determined to be the corresponding points are shown above, and the points O are the points that are determined to be the corresponding points other than the original. If there is a corresponding point other than S on all sides of Y (Y in step S10), it is determined that the entire document is included in each image (step S11), otherwise (step S1).
0 N), it is determined that each image does not include a part of the document (step S12).

FIG. 7 shows S on the outside of all sides of the circumscribed polygon L.
There is a corresponding point other than, and it is determined that the entire original is included in the image P. In this case, when the line segment H orthogonal to the side is drawn from the apex of each side of the circumscribed polygon L to the outside of the image P, the area surrounded by the line segment H, the side, and the end of the image P is surely corresponded. Point O exists. Also,
FIG. 8 shows a case where one of the images P does not include a part of the document. In this case, the line segment H, the side, and the image P
There is an area surrounded by the edge of and the corresponding point O does not exist.

Calculation of corresponding points between two images (step S
If 4) and the determination as to whether or not the entire original is included in the image (steps S8 to S12) are completed, an image is generated based on the obtained information on the presence or absence of the original (step S8). S13, S14).

That is, when it is determined that the originals are completely included in each of the two images (step S11), the images are generated by the method of superimposing the images (step S13). ), Improve the image quality. In FIG. 9, two images, which are the overlapping portions of the entire portion of the image of the document G indicated by the arrow, are overlapped to generate an image. Several methods have been proposed for superimposing images, but as an example, Japanese Patent Laid-Open No. 2000-217
The method disclosed in 032 can be used.

If it is determined that a part of the original document is not included in each image (step S12), there is an overlap in each image, and therefore, a method of joining images is used. An image is generated (step S14),
Obtain a high resolution image. That is, in FIG. 10, two images, which are overlapping portions of only the portion indicated by the arrow in the image of the document G, are joined to generate an image. Regarding this method, some means have been proposed in the past, but for example, the method disclosed in Japanese Patent Laid-Open No. 2000-348159 can be used. Further, at this time, the image quality may be improved by applying the above-described image superimposing method only to a portion where the images overlap.

After the image generation (steps S13 and S14) is completed, the user is notified of the completion of the process by the display device 7 or the like (step S15), and the process is completed.

According to the image input apparatus 1 described above, an optimum method for obtaining a high quality image can be selected according to the field of view of each of the cameras 10 and 11 (step S10) so that an image can be generated (step S10). (S13, S14), the constraint on the presentation position of the document as the subject can be relaxed, and the burden on the user can be reduced.

Specifically, when the field of view of each of the cameras 10 and 11 includes the entire document (step S1).
Y of 0) A high quality image can be generated (step S13).

When the fields of view of the cameras 10 and 11 each include only a part of the subject (step S1).
(N of 0) A high quality image can be generated (step S14).

Further, when the images cannot be combined with each other (N in step S5), the user can be prompted to take another image (step S7).

As described above, when the images are overlapped and the images are joined together for each image portion, a high quality image can be generated for each image portion.

[0063]

According to the first aspect of the present invention, an image can be generated by selecting an optimum method for obtaining a high quality image according to the field of view of each camera, so that the constraint on the presentation position of the subject is relaxed. Thus, the burden on the user can be reduced.

According to the second aspect of the invention, in the image input device according to the first aspect, it is possible to generate a high quality image when the field of view of each camera includes the entire subject.

According to a third aspect of the present invention, in the image input device according to the first or second aspect, a high quality image is generated when the field of view of each camera includes only a part of the subject. You can

According to the invention described in claim 4, in the image input device according to claim 1 or 2, it is possible to generate a high quality image for each part of the image.

According to the invention described in claim 5, in the image input device according to any one of claims 1 to 4, when the images cannot be combined from the images, the user can be prompted to take another image. .

According to a sixth aspect of the present invention, in the image input device according to any one of the first to fifth aspects, the visual field can be determined by the corresponding points.

According to a seventh aspect of the invention, in the image input device according to the sixth aspect, the entire subject is included in the image or a part of the subject is not included in the image due to the corresponding points. Can be determined.

According to the eighth aspect of the present invention, an image can be generated by selecting an optimum method for obtaining a high-quality image according to the field of view of each camera, so that the constraint on the presentation position of the subject is relaxed. It is possible to reduce the burden on the user.

According to the ninth aspect of the invention, in the program according to the eighth aspect, it is possible to generate a high quality image when the field of view of each camera includes the entire subject.

According to a tenth aspect of the invention, in the program according to the eighth or ninth aspect, it is possible to generate a high quality image when the field of view of each camera includes only a part of the subject. .

According to the eleventh aspect of the invention, in the program according to the eighth or ninth aspect, it is possible to generate a high quality image for each portion of the image.

[Brief description of drawings]

FIG. 1 is a block diagram showing an electrical connection of an image input device according to an embodiment of the present invention.

FIG. 2 is a front view of the image input device.

FIG. 3 is a plan view illustrating a field of view of the image input device.

FIG. 4 is a plan view of the same.

FIG. 5 is a plan view of the same.

FIG. 6 is a flowchart illustrating a process performed by the image input device.

FIG. 7 is an explanatory diagram illustrating a process performed by the image input device using corresponding points of an image.

FIG. 8 is an explanatory diagram of the same.

FIG. 9 is a plan view illustrating a field of view of the image input device.

FIG. 10 is an explanatory diagram of the same.

[Explanation of symbols]

1 Image input device 10 cameras 11 cameras S corresponding point O corresponding point P image G subject

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 5B057 BA11 CA08 CA13 CA16 CB08                       CB12 CB16 CD02 CE08 CE10                       DA07 DA15 DA20 DC02 DC05                 5C022 AA13 AB61 AB62 AB68 AC27                       AC42 AC54 AC78 CA07                 5L096 AA09 CA05 DA03 EA14 FA05                       FA09 FA66 FA77 FA78

Claims (11)

[Claims] 1. A plurality of cameras for photographing an object, a visual field judging means for judging a visual field of each camera for the object based on each image photographed by each camera, and a different method depending on the judgment. An image input device comprising: an image generating unit that generates a single image of the subject from each of the captured images. 2. The image generating means generates a single image of the subject by a method of superimposing the photographed images when the field of view of each camera includes the whole of the subject according to the determination. The image input device according to claim 1, which is a thing. 3. The image generation means, when the field of view of each camera includes only a part of the subject according to the determination, a single image of the subject is obtained by a method of joining the captured images. The image input device according to claim 1 or 2, which is generated. 4. The image generating means, when the field of view of each of the cameras includes only a part of the subject according to the determination, the captured image is taken for an overlapping part of the field of view of each of the cameras. The image input device according to claim 1 or 2, wherein a single image of the subject is generated by using a method of superimposing and a method of combining the captured images for other portions. 5. When the visual field determining means determines that there is no overlapping portion in the visual fields of the cameras with respect to the subject, the camera and the subject are separated from the previous photographing, and the subject with the cameras is separated from the subject. The image input device according to any one of claims 1 to 4, further comprising informing means for informing a message prompting re-imaging. 6. The field-of-view determining means uses a fixed threshold value α when the distance to the corresponding point closest to the camera is x, and the corresponding point in the range of “x + α” is the subject. The image input device according to any one of claims 1 to 5, wherein the image field is determined by determining that the set of corresponding points is S. 7. The field-of-view determining means determines that the entire subject is included in the image when the set S of corresponding points of the subject is surrounded by a set of other corresponding points. The image input device according to claim 6, wherein if not, it is determined that a part of the subject is not included in the image. 8. A field-of-view determination process for determining the field-of-view of each camera with respect to the subject based on each image obtained by shooting the subject with a plurality of cameras, and a method different depending on the determination is used to determine the subject from each of the captured images. A computer-readable program that causes a computer to execute an image generation process for generating a single image of. 9. The image generation processing generates a single image of the subject by a method of superimposing the captured images when the field of view of each camera includes the entire subject in the determination. The program according to claim 8, which is a program. 10. The image generation process, when the field of view of each camera includes only a part of the subject according to the determination, a single image of the subject is obtained by a method of joining the captured images. The program according to claim 8, which is generated. 11. The image generation processing, when the field of view of each of the cameras includes only a part of the subject according to the determination, the captured images are taken for an overlapping part of the field of view of each of the cameras. 10. The program according to claim 8, wherein a single image of the subject is generated by using a method of superimposing and a method of joining the captured images in other portions.