JP2012070222A - Imaging device, imaging method, and imaging program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To bring a a person as a subject into a state suitable for imaging depending on their features or behaviors and enable recording of high-quality images.SOLUTION: Personal authentication is performed on a person to be a subject in photography with a camera, and an image engine 15 saves features or behaviors of each individual, such as the degree of tension, smiling, blinking, or red-eye, in an operation history database 32 as subject-related information. When the authenticated person is photographed next time, a subject-related information acquisition and attention necessity determination part 31 reads the subject-related information from the operation history database 32 and instructs an attention execution part 33 to call an attention depending on the subject-related information. The attention execution part 33 calls an attention, and then, an imaging part 11 performs imaging.

Description

  The present invention relates to an imaging apparatus, an imaging method, and an imaging program.

  Patent Documents 1 to 5 listed below allow image processing parameters to be optimally set using setting values and sample setting values of past users.

  The electronic camera system described in Patent Document 1 corrects captured image data captured from an electronic camera according to a user instruction, records a correction value for each correction item as correction history information, and based on the correction value. A computer that calculates a representative correction value that is an average value, and an electronic camera that captures and stores the representative correction value calculated by the personal computer and corrects the image processing conditions at the time of shooting based on the representative value. By configuring the camera system, it is possible to record with the desired image quality with high accuracy from the history of past image processing.

  The camera device described in Patent Literature 2 is used when a user changes a shooting parameter to shoot a new subject or when a shooting parameter is automatically changed by directing a lens toward a new subject. A sample image of a shooting scene category estimated according to the partially changed shooting parameter is displayed on a part of the image. As a result, the user can know the existence of a shooting scene for setting shooting conditions similar to the current shooting scene while performing the work of determining the subject and composition, and when the user selects a sample image, It is said that the corresponding shooting conditions can be automatically set immediately.

  The imaging information providing system described in Patent Document 3 provides information used for imaging by a sample provider to a general user via a server, so that even a user who is not familiar with camera handling can perform the imaging conditions. In this way, excellent image shooting can be performed.

  The image processing method described in Patent Document 4 records the shooting type / photographing information, display method, processing procedure and processing time of the image used by the user, the processing contents according to the shooting model / shooting conditions, and then the same conditions. When a photographed image is input, an appropriate processing procedure is automatically selected from the processing parameter group stored automatically.

  In the imaging support method described in Patent Document 5, a mobile phone terminal including a camera imaging unit having an imaging function stores past captured images and shooting information including shooting conditions in association with each other in a shooting history database. A shooting situation analysis engine that makes it possible to retrieve the shooting information by the shooting history utilization unit is provided, and this shooting situation analysis engine enables new shooting based on shooting conditions of past shooting data.

  In these Patent Documents 1 to 5, image processing parameters are optimally set based on past user setting values and sample setting values. Further, these image processing parameters depend on scenes such as landscape, portrait, and night view.

  However, even if the image processing parameters are set optimally, if the person who is the subject has a tense and stiff face or the eyes are meditated, a satisfactory captured image cannot be obtained. Patent Documents 1 to 5 do not mention that the subject is brought into a state suitable for imaging depending on the characteristics and wrinkles of each person.

  In Patent Document 6, a feature extraction process is performed on a photographed image, the shape of the subject's eyes and mouth is extracted, a coefficient is calculated based on the extracted shape, and the calculated coefficient is a predetermined value determined in advance. It is determined whether or not the threshold value is less than or equal to the threshold value. If the threshold value is not more than the predetermined threshold value, an advice message for correcting the facial expression of the subject is generated and output, thereby detecting the facial expression and posture of the subject. A digital camera is described in which advice can be given appropriately according to the posture.

  In Patent Document 7, the movement of the subject, facial expression of the subject, the state of the subject's eyes, the orientation of the subject's face, and the like are detected, and the movement of the subject is determined according to these detection results. An image pickup device that stops, prompts a change in the facial expression of the subject, prompts a change in the state of the subject's eyes, and visually displays each instruction information for prompting a change in the orientation of the face of the subject. Is described.

  Further, in Patent Document 8, a human face image is detected from an input image, a facial expression evaluation value corresponding to each facial expression is calculated from the detected face image, and the calculated facial expression evaluation value and facial expression evaluation are calculated. The facial expression recognition that updates the relationship with the facial expression determination threshold provided corresponding to the value based on the facial image and determines the facial expression based on the updated facial expression evaluation value and the facial expression determination threshold An apparatus is described.

JP 2002-262174 A JP 2006-186695 A JP 2007-36716 A JP 2007-219728 A JP 2010-41527 A JP 2003-219218 A JP 2009-246832 A JP 2009-253848 A

  As described above, Patent Documents 1 to 5 optimally set image processing parameters based on past user setting values and sample setting values. It does not depend on the subject to be in a state suitable for imaging.

  The techniques disclosed in Patent Document 6 and Patent Document 7 provide advice and messages according to the facial expression and state of the subject currently being photographed. However, those disclosed in these patent documents do not perform personal authentication of a person who is a subject, and give advice or a message regardless of the characteristics or habits of each person. Therefore, even for a subject that does not require advice or a message, the calculation load increases and power consumption occurs because the facial expression and state of the subject are continuously detected. Further, the techniques disclosed in Patent Documents 6 and 7 provide advice and messages based on the facial expression and state of the subject, and thus cannot cope with changes in the subject immediately before imaging. For example, it is impossible to prevent a subject that is easily blinked from blinking suddenly at the moment of being imaged.

  Patent Document 8 can increase the accuracy of facial expression recognition by updating the facial expression evaluation value. Further, in this patent document 8, a person is determined by performing personal authentication on a subject, and whether or not shooting is performed based on a facial expression detected in the past and the current facial expression of the subject is determined. It is described that it is judged.

  However, Patent Document 8 determines whether or not to perform shooting according to a facial expression, and does not lead the subject to a state suitable for imaging depending on characteristics or habits of each person who is the subject. .

  In view of the above-described problems, the present invention works on a person who is a subject, leads the subject to a state suitable for imaging depending on the characteristics and habits of each person who is the subject, and provides a high-quality image. An object of the present invention is to provide an imaging apparatus, an imaging method, and an imaging program that can record the image.

  In view of the above-described problems, the present invention is based on the characteristics and habits of the subject, and determines whether the subject needs to be acted on, and determines whether or not the approach determining unit needs to act on the subject. And an imaging unit that performs imaging after the action execution unit performs an action.

  The present invention personally authenticates a subject, acquires subject-related information corresponding to the subject subject to personal authentication from a storage unit based on a personal authentication result, and applies the subject to the subject based on the subject-related information acquired from the storage unit In this imaging method, it is determined whether or not an action is necessary, and when an action is necessary, an image is taken after the action is performed on the subject.

  The present invention includes a step of personally authenticating a subject, a step of acquiring subject-related information corresponding to the subject that has been personally authenticated based on a personal authentication result from a storage unit, and a subject-related information acquired from the storage unit The computer-executable imaging program includes a step of determining whether or not the subject needs to be acted, and a step of taking an image after performing the act on the subject when the act is necessary. .

  According to the present invention, personal authentication is performed on a person who is a subject at the time of camera shooting, and the characteristics and habits of each individual are stored as subject-related information as a history. Since the work is performed in accordance with the subject-related information, the subject is brought into a state suitable for imaging depending on the characteristics and habits of each person as the subject, and a high-quality image can be recorded.

It is a block diagram based on the function of the portable terminal device with a camera concerning one embodiment of the present invention. 4 is a flowchart showing processing when camera shooting is performed using subject-related information of each individual held in an operation history database in the embodiment. It is explanatory drawing of the operation history database in the portable terminal device with a camera concerning the embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram based on functions of a camera-equipped mobile terminal device 1 according to an embodiment of the present invention.

  As illustrated in FIG. 1, the mobile terminal device 1 includes a wireless communication unit 10, an imaging unit 11, an image recording processing unit 12, an image recording unit 13, an image reproduction processing unit 14, an image engine 15, and a subject. The analysis engine 16, the position detection unit 17, and the display unit 18 are included.

The wireless communication unit 10 communicates with a base station wirelessly to realize a telephone function, a mail function, an internet connection function, and the like.
The imaging unit 11 photoelectrically converts the captured subject image light and outputs an imaging signal based on the subject image light. As the imaging unit 11, a CCD (Charge Coupled Device) imaging device or a CMOS (Complementary Metal Oxide Semiconductor) imaging device is used.

The image recording processing unit 12 performs image processing on the imaging signal from the imaging unit 11. Image recording processing includes camera processing such as gamma correction, sharpness, and white balance correction, and image compression processing using a JPEG (Joint Photographic Experts Group) method or the like.
The image recording unit 13 records the imaging signal processed by the image recording processing unit 12 on a recording medium. As the recording medium, a built-in semiconductor memory, a card type flash memory, or the like is used.

  The image reproduction processing unit 14 reproduces the image signal recorded in the image recording unit 13. The reproduction process includes an image expansion process by the JPEG method or the like. The image reproduced by the image reproduction processing unit 14 is displayed on the display unit 18. For example, a liquid crystal display is used as the display unit 18.

  The image engine 15 analyzes the image captured by the image capturing unit 11 from the image recording processing unit 12 or the image reproduced from the image recording unit 13 from the image reproduction processing unit 14, and the characteristics of the person serving as the subject Acquire subject-related information based on yam. The image engine 15 includes a personal authentication unit 21, a face detection unit 22, a smile detection unit 23, a blink detection unit 24, a red-eye detection unit 25, and the like. As a result, personal authentication of the person of the subject (for example, identification of a person by face authentication) is performed, and facial tension, smile, blink, red-eye, etc. can be acquired as subject-related information.

  The subject analysis engine 16 includes a subject-related information acquisition and action necessity determination unit (subject-related information acquisition unit) 31, an operation history database (storage unit) 32, an action execution unit 33, and a rule database (rule storage unit) 34. And an action control unit 35.

  The subject-related information acquisition and action necessity determination unit 31 acquires subject-related information for each person of the subject using the image engine 15, and reads / writes to / from the operation history database 32. That is, the subject-related information acquisition / working necessity determination unit 31 performs personal authentication at the time of camera shooting, analyzes the person's tension level, smile level, blink level, red-eye level, etc. Subject-related information such as features and wrinkles is extracted, and the subject-related information is weighted according to shooting conditions and described in the operation history database 32. In addition, when photographing a person who has already been personally authenticated, the subject-related information acquisition and action necessity determination unit 31 reads the setting file from the operation history database 32 and captures the shooting condition (tension of the subject (person who has been personally authenticated)). Instructions are output to the action execution unit 33 based on the degree of smile, the degree of smile, the degree of red-eye, the blinking frequency, and the like. At that time, the mobile terminal device 1 includes a position detection unit 17 that measures the current location, and a position-related information providing unit 36 that stores in advance information about each location (photographing location). The action necessity determination unit 31 acquires position information at the time of shooting and bean knowledge about the position as necessary, and presents the acquired bean knowledge to the subject via the work execution unit 33 and the sound reproduction unit 41. To do.

  The action execution unit 33 executes an instruction based on the subject photographing conditions from the subject-related information acquisition and the action necessity determination unit 31. The action from the action execution unit 33 is, for example, to output a sound related to the knowledge about the shooting location from the sound reproduction unit 41 when the subject is easily strained, and to relieve the tension of the subject. When the subject is a low subject, the sound reproduction unit 41 makes a “yes cheese” or joke to direct the subject to laugh, and when the subject is prone to red eyes, the red eye from the flash lamp 42 When the subject is a subject that is easy to blink, the voice reproduction unit 41 instructs the timing to end the blink, or prompts the user to pay attention to the imaging device, and blinks. It is to suppress. Further, the work execution unit 33 can display advice on the display unit 38 and set image processing parameters in the image recording unit 12.

  The rule database 34 is a database describing operations to be executed. The rule group described in the rule database 34 is as follows.

(1) At the time of camera shooting, personal authentication is performed on the subject, and the image engine 15 extracts features and wrinkles for each individual, and if necessary, stores them in the operation history database 32 together with position information at the time of shooting. To do.

(2) At the time of personal authentication, for a person who is determined to have a high degree of tension by the face detection unit 22 of the image engine 15, at the next shooting, a sound related to background knowledge is output from the audio playback unit 41 Relieve tension.

(3) At the time of personal authentication, for the person who has determined that the smile level of the smile detection unit 23 of the image engine 15 is low, at the next shooting, the subject is laughed by pronouncing “yes cheese” or a joke.

(4) At the time of personal authentication, red-eye correction is applied to a person who is determined to be easily red-eyed by the red-eye detection unit 25 of the image engine 15 and emits red-eye prevention light from the flash lamp 42 at the next shooting.

(5) At the time of shooting with the camera, the blink detection unit 24 of the image engine 15 detects blinking, and the person who has taken a picture again is instructed the timing to end the blinking at the next shooting, or the imaging device Outputs prompts to keep an eye on and suppresses blinking.

(6) When photographing a camera, if the subject is a personally authenticated subject, the setting file is read from the operation history database 32, the photographing conditions suitable for the subject are read, and feedback or advice is given to the photographer, or the read photographing conditions are set. Based on the camera shooting.

  The mobile terminal device 1 includes a best shot mode and a normal shooting mode as shooting modes. When shooting in the best shot mode, the mobile terminal device 1 performs the above-described action. On the other hand, when shooting in the normal shooting mode, the mobile terminal device 1 does not perform the above-described action. On the other hand, the mobile terminal device 1 acquires subject-related information indicating the characteristics and wrinkles of the subject and stores them in the operation history database 32 in both the best shot mode and the normal shooting mode.

  The behavior control unit 35 reads a rule from the rule database 34 and performs processing according to the read rule when shooting in the best shot mode.

  FIG. 2 is a flowchart showing a process when shooting is performed by using the subject-related information of each individual held in the operation history database 32 to perform an action.

  In FIG. 2, when it is determined that the personal authentication has been completed (step S1), the subject related information acquisition and action necessity determination unit 31 reads the subject related information of the individual from the operation history database 32 (step S2). Then, the subject-related information acquisition and action necessity determination unit 31 first determines whether the subject has a high degree of tension from the information on the degree of tension included in the subject-related information read from the operation history database 32 (step S3). . When the degree of tension is large, the subject-related information acquisition and action necessity determination unit 31 reads the bean knowledge about the shooting location from the operation history database 32 and causes the work execution unit 33 to output the sound about the bean knowledge about the shooting location. The sound reproduction unit 41 outputs a sound related to knowledge about the shooting location to relieve the subject's tension (step S4).

  Next, the subject-related information acquisition and action necessity determination unit 31 determines whether the subject has a low smile level from the smile level information included in the subject-related information (step S5). If the degree of smile is low, the subject-related information acquisition / working necessity determination unit 31 instructs the work execution unit 33 to output “yes cheese” or a joke. Or a joke is output to make the subject smile (step S6).

  Next, the subject-related information acquisition and action necessity determination unit 31 determines whether or not the subject has a large red-eye degree (subject that tends to have red eyes) from the red-eye degree information included in the subject-related information (step S7). ). When the red-eye degree is large, the subject-related information acquisition and action necessity determination unit 31 instructs the action execution unit 33 to emit red-eye prevention light and causes the flash lamp 42 to emit red-eye prevention light. (Step S8).

  Next, the subject-related information acquisition and action necessity determination unit 31 determines whether or not the subject is easily blinked from the blink information included in the subject-related information (step S9). When it is easy to blink, the subject-related information acquisition and action necessity determination unit 31 outputs an instruction to the action execution unit 33 to output a timing to endure blinking or a prompt to watch the imaging apparatus. And the voice reproducing unit 41 outputs an instruction for timing to endure blinking or a prompt to watch the image pickup apparatus (step S10).

  In this way, the subject is acted on by the processing of step S1 to step S9. Thereafter, the imaging unit 11 captures a subject image (step S11).

  As described above, in one embodiment of the present invention, personal authentication is performed on a person who is a subject at the time of camera shooting, and each person's characteristics and wrinkles are stored as subject-related information as a history. When shooting a person, it is possible to work according to subject-related information. In other words, when shooting a person who has been personally authenticated next time, if the subject is subject to tension from subject-related information, it is possible to show the knowledge about the shooting location to relieve the subject's tension, and to smile When it is a low-grade subject, you can make the subject laugh by saying “yes cheese” or a joke, and when it is a subject that is prone to red eyes, it can emit red-eye prevention light, Further, it is possible to suppress blinking by instructing the timing to endure blinking or prompting the user to watch the imaging apparatus.

  For example, when the user A photographs the person B with the camera-equipped mobile terminal device 1, the person B is often nervous about camera photography. In this case, when the person B is photographed, the face detection unit 22 of the image engine 15 detects that the degree of tension is high. At this time, subject-related information extracted from the image engine 15 is as follows.

Person authenticated by personal authentication function: Person B
Subject-related information: Tension level large setting file: File associated with subject-related information for person B

  Based on this, as shown in FIG. 3A, “Large” is described in the “Tension” column of the file associated with the person B in the operation history database 32.

  Suppose that user A takes a picture of person B at sightseeing spot X at a later date. At this time, when the user A activates the camera and sets the best shot mode, the subject is recognized as the person B by the personal authentication unit 21 of the image engine 15. Then, the setting file (FIG. 3A) associated with the person B is read from the operation history database 32 by the subject related information acquisition unit and the action necessity determination unit 31.

  As shown in FIG. 3A, from the setting file of the operation history database 32, the person B is determined to have a high degree of tension. When the degree of tension is large, the subject-related information acquisition / working necessity determination unit 31 acquires information indicating the current position (imaging location) from the position detection unit 17, and the bean for each location stored in advance by the position-related information providing unit. Read knowledge about the current position from the knowledge. Then, the subject-related information acquisition and action necessity determination unit 31 outputs the read bean knowledge and an instruction to show the bean knowledge to the action execution unit 33. Based on this instruction, the work execution unit 33 outputs a sound related to the knowledge about the shooting location from the sound reproduction unit 41. Thereby, the person B can be relieved of tension.

  Further, for example, when the user A takes a picture of the person C with the camera-equipped mobile terminal device 1, it is assumed that the person C often appears in red eyes. Here, when the person C is photographed, red-eye “large” is detected by the red-eye detection unit 25 of the image engine 15, and subject-related information extracted from the image engine 15 is as follows.

Person authenticated by personal authentication function: Subject person C
Subject-related information: Red eye degree large setting file: File associated with subject-related information for subject person C

  Based on this, as shown in FIG. 3B, “Large” is described in the “red eye degree” column in the setting file of the person C in the operation history database 32.

  It is assumed that user A takes a picture of person C at a later date. When the user A activates the camera and sets the best shot mode, the subject is recognized as C by the personal authentication function. Then, the subject related information acquisition unit and the action necessity determination unit 31 read the setting file (FIG. 3B) associated with “person B” from the operation history database 32.

  As shown in FIG. 3B, it is determined from the setting file of the operation history database 32 that the person C has a high red eye degree. When the red-eye degree is large, the subject-related information acquisition and action necessity determination unit 31 gives a red-eye prevention instruction to the action execution unit 33, and red-eye prevention light is emitted from the flash lamp 42. Thereby, the red eye of the person C can be prevented.

  As described above, the mobile terminal device 1 acts on the subject as necessary based on the subject-related information indicating the feature or habit of the subject, and performs imaging after the approach. As a result, the subject can be guided to a state suitable for imaging, such as releasing the tension of the subject, and imaging can be performed, and a high-quality image can be recorded.

At that time, since the mobile terminal device 1 determines whether or not the action is required using the subject-related information, it is possible to prevent the subject from being changed immediately before shooting. For example, it is possible to perform an action to prevent blinking on a subject that is easy to blink, even if the subject has not yet blinked, thereby preventing blinking at the moment of shooting.
In addition, since the mobile terminal device 1 determines whether or not an action is necessary using the subject-related information, it is not necessary to detect the facial expression or state of the subject after determining that the subject does not require an action. Calculation load and power consumption can be reduced.

  Moreover, since the portable terminal device 1 acquires subject-related information at the time of shooting, the user does not need to perform a special operation for acquiring subject-related information. In this respect, the burden on the user can be reduced.

  In the above-described example, the embodiment applied to the camera-equipped mobile terminal has been described. However, the present invention is not limited to this, and can be similarly applied to various imaging apparatuses such as a normal digital camera.

  In the above-described example, the tension, smile, blink, and red-eye are used as the characteristics and wrinkles of each individual that is the subject. However, what is used as the characteristics and wrinkles of each individual is limited to this. It is not something. Also, in the above example, when working on the characteristics and habits of each individual, when the subject is subject to tension, when the subject has a low smile level by showing the tips on the shooting location to ease the subject's tension In addition, when the subject is laughing by pronouncing a joke, it emits red-eye prevention light when the subject is prone to red-eye, and when there are many blinks, it indicates the endurance timing or gazes at the imaging device However, the approach to individual characteristics and habits is not limited to this.

  In addition, the method for identifying the subject for the mobile terminal device 1 to read the subject-related information is not limited to the above-described personal authentication method. For example, various methods capable of identifying a subject can be used, such as a user inputting identification information of a subject by an operation input.

It should be noted that a program for realizing the function of each unit of the mobile terminal device 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute processing of each unit. May be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like without departing from the gist of the present invention.

1: mobile terminal device 10: wireless communication unit 11: imaging unit 12: image recording processing unit 13: image recording unit 14: image reproduction processing unit 15: image engine 16: subject analysis engine 17: position detection unit 18: display unit 21 : Personal authentication unit 22: face detection unit 23: smile detection unit 24: blink detection unit 25: red-eye detection unit 31: subject-related information acquisition and action necessity determination unit 32: operation history database 33: action execution unit 34: rule database 35: Action control unit 36: Position related information providing unit 41: Audio reproduction unit 42: Flash lamp

Claims (5)

An action necessity determination unit that determines whether or not to act on the subject based on the characteristics and habits of the subject;
When the action necessity determination unit determines that the action is necessary, an action execution unit that works on the subject; and
An imaging unit that captures an image after the action execution unit has worked,
An imaging apparatus comprising: In addition, a storage unit that stores wrinkles and features for each person as subject-related information;
A rule storage unit for storing a rule in which the subject-related information and the necessity of working are associated;
A personal authentication unit for personal authentication of the subject;
A subject-related information acquisition unit that acquires, from the storage unit, subject-related information corresponding to the subject that has been personally authenticated based on the personal authentication result;
The said action necessity determination part applies the rule which the said rule memory | storage part memorize | stores in the subject relevant information which the said subject relevant information acquisition part acquires, The necessity determination of the subject with respect to a subject is determined. Item 2. The imaging device according to Item 1.   The imaging apparatus according to claim 2, wherein the wrinkles and characteristics of each person stored in the storage unit are used as subject-related information at the next and subsequent shootings. Personally identify the subject,
Based on the personal authentication result, subject related information corresponding to the personally authenticated subject is acquired from the storage unit,
Based on the subject-related information acquired from the storage unit, determine whether or not to act on the subject,
An imaging method in which, when the action is required, an image is taken after the subject is executed. Personal authentication of the subject;
Obtaining subject-related information corresponding to the subject subject to personal authentication from the storage unit based on the personal authentication result;
Determining whether or not to act on the subject based on the subject-related information acquired from the storage unit;
An imaging program that can be executed by a computer, including a step of performing imaging after performing an action on the subject when the action is required.

Images