JP5950675B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus having an imaging element that reads an image signal through a plurality of paths.

  In recent years, with the increase in resolution of digital cameras and digital video cameras, the accuracy of object detection has also improved, and there is a demand for feedback to shooting, recording, and display.

  In Patent Document 1, high-resolution video is input from a plurality of cameras, and feature recognition is performed by comparing features.

JP 2004-192378 A

  The delay in the processing and display of video data when shooting in real time is directly related to stress during operation. When a subject is detected, recognized, and tracked, the processing delay must be reduced as much as possible. However, with the recent increase in resolution, video data processing time is increasing. In order to prevent this, it is conceivable to use a plurality of readout paths of the image sensor and to divide the processing for each path. However, with such a configuration, there may be a case where a subject that crosses the boundary of the divided areas cannot be detected.

  For example, based on the edge information obtained from the video, when extracting partial areas of shapes that seem to be organs constituting the face such as eyes and mouth, and satisfying that these are in a predetermined positional relationship, There is a method of determining an area including a partial area as a face. When a face is detected using such a method, the face cannot be detected if the image of the area including the eyes and the image of the area including the mouth are read out through different paths.

  There is a method to detect the object after pasting the video data read through multiple paths in the latter stage of the memory, but this process is later, so a delay occurs and the operability deteriorates. Connected.

(Object of invention)
An object of the present invention is to provide an imaging apparatus capable of realizing subject detection with high speed and high accuracy even with high resolution.

  In order to achieve the above object, an image pickup apparatus according to the present invention is an image pickup device that reads out an image signal generated by photoelectric conversion for each pixel through a plurality of paths, and an image pickup that drives readout of the image sensor through a plurality of paths. Element driving means, a plurality of subject detecting means for detecting a subject from video data of each of the plurality of paths, a result integrating means for integrating detection results output from the subject detecting means, and video data read by the plurality of paths Division control means for dividing the area, and the division control means controls the division based on the output of the result integration means so that the subject does not straddle the boundary of the divided areas. The image sensor driving means drives each of a plurality of paths according to the result of the division.

  According to the present invention, high-speed and high-precision subject detection can be realized even with high resolution.

It is a figure which shows the structure of the imaging device which is Example 1 of this invention. It is explanatory drawing of the image division | segmentation of Example 1. FIG. 6 is a flowchart showing an operation common to Embodiments 1 and 2. It is a figure which shows the structure of the imaging device which is Example 2 of this invention. FIG. 10 is an explanatory diagram of image division according to the second embodiment.

  The mode for carrying out the invention is as described in Examples 1 and 2 below.

  FIG. 1 is a diagram illustrating an overall configuration of an image pickup apparatus that is Embodiment 1 of the present invention. An image in which incident light is imaged by the lens 101 is incident on the sensor 104. In the sensor 104, photoelectric conversion is performed for each pixel, and an image signal generated by the photoelectric conversion is divided into four channels for speeding up, and is simultaneously read out through a plurality of paths. The sensor 104 represents one or a plurality of image sensors. For each channel, there are analog front end circuits 105 to 108 (hereinafter abbreviated as AFE), which perform AD conversion and generate digital video data. The AFEs 105 to 108 are circuits that perform noise removal, signal amplification, and digital signal conversion.

  Each channel has face detection means 109 to 112 for detecting a face within the angle of view of each path. The video data used for the face detection is stored in the memory 113, signal processing is performed by the signal processing unit 114 as a single video data, and recording and display are performed. Further, the signal processing unit 114 can specify who the detected face is by using recognition data registered in advance. The face detection results are integrated by the result integration unit 115, the face position in one piece of video data is calculated, the recording by the signal processing unit 115, the processing of the display image, and the lens driving (focus control) by the lens driving unit 102. Used for processing weights. Further, the division control unit 116 corrects the division of the region based on the face detection position so that the region where the face is predicted to be present in the next field processing (or frame processing) does not cross between the readout paths. . That is, the region boundaries are divided so that the subject does not straddle. Based on this information, the sensor driving means 103 performs sensor readout driving in the next field (or frame). The sensor driving means 103 constitutes an image sensor driving means.

  The detection mode selection unit 117 outputs the video data input to the face detection unit A 109 according to the presence / absence of the face from the result from the result integration unit 118 and the shooting mode, from the output of the AFE_A 105 which is the video data of the channel A, It is possible to selectively input to the face detection means A whether the output from the resizing means 115 which has resized the video data of all angles of view. The output of the AFE_A 105 constitutes the first video data read at high resolution, and the output of the resizing means 115 constitutes the resized low resolution second video data.

  201 in FIG. 2A illustrates image division. The entire imaging surface of the sensor 104 is divided into a plurality of rectangular regions R0 to R5. R2 and R3, which are areas where the face is predicted to exist, are divided so as to be read out by one path. Other areas are divided into a plurality of rectangular areas R0, R1, R4, and R5. The rectangular areas R0, R1, R4, and R5 are allocated so that the amount of video data transferred through the read paths A to D is equal. As shown in FIG. 202, the video data from the region R2 in the channel A, the video data from the region R3 in the channel B, the video data from the regions R0 and R1 in the channel C, and the regions R4 and R5 in the channel D. Are distributed respectively. Then, reading is performed over one field period (or one frame period). As a result, as shown in 203, faces existing in the regions R2 and R3 are normally detected.

  FIG. 3 is a flowchart showing the operation of the first embodiment. After the power is turned on (S301), it is determined whether or not the image is taken (S302, before shooting starts (during the initial operation after turning on the power)), an initial search is first performed (S303). The face is detected from the entire low-resolution video (S304), that is, the position of the subject in the entire angle of view is analyzed, whether or not a face is detected is determined (S305), and if detected, The division setting of the next field is performed so that one face is not divided into videos of different areas and the data amount of the video read out in each path is approximately equal (S306). (S308).

  First, after the start of photographing, it is determined whether or not division setting is made in the initial search before photographing (S307). If the division setting is not set, the face is detected from the entire low-resolution video obtained by reducing the image obtained from the sensor 104 as in the pre-shooting operation after the power is turned on (S313 to S315). If a face is detected, the next field division setting is performed so that one face is not divided into videos of different areas and the data amount of the video read through each path is evenly approached (S316). ), Recognition processing at a low resolution is performed (S317). The signal processing means 114 compares the shape of the organs constituting the face such as the eyes and mouth in the detected face and the mutual positional relationship between the organs with recognition data registered in advance for each individual. As a result of the comparison, if the degree of similarity is high, it is determined that the detected face is the same person as the individual indicated by the recognition data.

  If it is determined in step 307 that the division setting has been made, the face detection process is performed on each of the high-resolution divided videos obtained by dividing the image obtained from the sensor 104 into a plurality of pieces without reducing the image (S308). It is determined whether or not a face has been detected (S309). If a face has been detected, the division setting of the next field is maintained without being changed (S310). On the other hand, when the face is not detected (the face has disappeared from the angle of view), the division setting is canceled (S311). Then, recognition processing is performed on the divided high-resolution video (S312). By performing recognition processing on a high-resolution video, there may be cases where an individual who has failed to recognize a low-resolution video can be recognized. This is repeated for each field (S318).

  If the face is not detected in step 309 during shooting, the division setting is canceled in the next field, so low resolution analysis processing (face detection processing) is performed in step 314.

  Similarly, if the size of the face exceeds the amount of video data distributed to one of the multiple paths during shooting, the face is not detected. Therefore, in the next field, low resolution analysis processing (face Detection process).

  Reference numerals 204 to 206 in FIG. 2B illustrate the initial search mode. When the face initial position is not known after the power is turned on, or when the face cannot be detected from each divided area and the division setting is once canceled, the detection mode selection unit 117 enters the initial search mode. As shown in 204, the entire area is set as one detection target, and as shown in 205, the low-resolution detection process is performed only in channel A in one field period, and the face is detected as shown in 206. Is done.

  FIG. 4 is a diagram illustrating an overall configuration of an image pickup apparatus that is Embodiment 2 of the present invention. Portions similar to those in the first embodiment are denoted by the same reference numerals.

  The difference from the first embodiment is that each channel has subject detection means 409 to 412, and detects a subject within the angle of view of each path based on comparison with subject information previously held in subject information holding means 418. . The subject detection results are integrated by the result integration unit 118, and the subject position in one piece of video data is calculated. The subject detection units 409 to 412 are circuits for detecting the subject indicated by the subject information held in the subject information holding unit 418. The subject information holding unit 418 can update or replace the subject information based on data transmitted from the outside. That is, the subject information holding means 418 holds subject information for detecting a dog, subject information for detecting a car, or subject information for detecting a child, thereby subject detection means 409 to 412. It is possible to detect an arbitrary subject.

  Reference numeral 501 in FIG. 5A illustrates image division. R2 and R3, which are areas where the subject is predicted to exist, are divided so as to be read out by one path. The other areas are divided into a plurality of rectangular areas R0, R1, R4, R5. The divided rectangular areas R0, R1, R4, and R5 are allocated to be uniform in the read paths A to D. As shown in 502, the channel A reads the region R2, the channel B reads the region R3, the channel C reads the regions R0 and R1, and the channel D reads the regions R4 and R5 over one field period. As a result, as shown in 503, subjects existing in the regions R2 and R3 are normally detected.

  The operation of the second embodiment is exactly the same as that of the first embodiment shown in FIG.

  If the subject is not detected in step 309 during shooting, the division setting is canceled in the next field, so low resolution analysis processing (face detection processing) is performed in step 314.

  Similarly, if the size of the subject exceeds the amount of video data distributed to one of the plurality of paths during shooting, the subject is not detected, so low resolution analysis processing (subject detection is performed in step 314 in the next field. Process).

  Reference numerals 504 to 506 in FIG. 5B illustrate the initial search mode according to the second embodiment. When the subject initial position is not known after the power is turned on, or when the subject cannot be detected from each divided area (subject undetected state), and the division setting is once canceled, the detection mode selection means 117 activates the initial search mode. enter. As shown in 504, the entire area is set as one detection target, and low-resolution detection processing is performed in one field period using only channel A as shown in 505, and a subject is detected as shown in 506. Is done.

102 Lens driving means 103 Sensor driving means 104 Sensor (imaging device)
109 to 112 Face detection means 115 Resize means 116 Division control means 117 Detection mode selection means 118 Result integration means

Claims (9)

An image sensor that reads out an image signal generated by photoelectric conversion for each pixel through a plurality of paths;
Image sensor driving means for driving readout in a plurality of paths of the image sensor;
A plurality of subject detecting means for detecting a subject from video data of each of the plurality of paths;
A result integrating means for integrating the detection results output by the subject detecting means;
Division control means for dividing an area of video data read by the plurality of paths,
The division control means controls the division based on the output of the result integration means so that the subject does not straddle the boundary of the area to be divided,
The image pickup device, wherein the image pickup device driving means drives each of a plurality of paths according to a result of division.   The division control unit divides the entire imaging surface of the image sensor into a plurality of rectangular areas, and the video data of the divided areas is divided into the plurality of paths so that the amount of video data transferred through the plurality of paths is equalized. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is distributed to the image pickup apparatus.   The division control unit analyzes the position of the subject in all angles of view output by the result integration unit, and controls the division of the region so that no subject crosses the boundary of the region. Item 3. The imaging device according to any one of Items 1 and 2.   The imaging apparatus according to claim 1, wherein the division control unit analyzes the position of the subject in a field or frame period.   The subject detection unit resizes the first video data read from the image sensor through the plurality of paths or the video data of the full angle of view to reduce the resolution to be lower than the first video data. The image pickup apparatus according to claim 1, wherein any one of the above is selectively input.   The subject detection means receives the second video data during the initial operation after power-on, and receives the first video data when the initial position of the subject is detected after the start of shooting and during the initial operation. The imaging apparatus according to claim 5, wherein:   The imaging apparatus according to claim 5, wherein the subject detection unit receives the second video data when the subject is not detected during photographing.   The subject detection means receives the second video data when the size of the subject exceeds the amount of video data distributed to one of the plurality of paths during shooting. 5. The imaging device according to 5.   The imaging apparatus according to claim 1, wherein the subject is a face.

Images