JP4436101B2 - robot - Google Patents

Description

The present invention relates to a robot in which facial expressions, particularly shadows and colors caused by light, can be seen naturally.

In order to realize a human- or animal-shaped robot, its head has a face and hair, and it is necessary that the human head does not feel uncomfortable.

Therefore, the “communication robot” of Patent Document 1 discloses a technique for creating a facial expression by mechanically driving a rubber film by an electromagnetic motor.
However, when a facial expression is mechanically realized, there are problems such as a complicated structure, high cost, and failure.

On the other hand, attempts have been made to realize facial expressions using an electronic display device.
For example, in “Intelligent robot having facial expression” of Patent Document 2, a robot face is realized by drawing a plurality of prepared facial expression patterns on an electronic display device.
However, in this method, the face is expressed without being influenced by the surrounding environment, for example, light, wind, etc., compared to the case of the mechanical type described above, and it becomes uncomfortable, and the robot and the human have a conversation. When such a thing is assumed, it is desired that the face is expressed more realistically.

In order to solve such a problem, the “robot” of Patent Document 3 has an electronic display and a means for detecting ambient light or wind, so that facial expressions can be expressed without any sense of incompatibility in conjunction with the surrounding situation. A technology that realizes a robot that changes the speed is disclosed.

JP-A-6-261982 JP 2001-215940 A JP 2003-187267 A

If the 3D computer graphics technology and the means for detecting external environmental factors are used like the technology described in Patent Document 3, the human or animal face linked to the environment is realistically represented on the display. It is possible to display. Then, using this as the face of the robot can realize a realistic face at a low cost and with few failures compared to the case of the mechanical type described above.

However, in the technique described in Patent Document 3, data is input to the three-dimensional model display unit by using the light detection unit and the ambient light environment is taken in, but the light source is not estimated. Therefore, much information cannot be obtained without increasing the number of light detection means.
I couldn't express a real face.

Therefore, the problem to be solved by the present invention is to reduce the number of light detection means by using an electronic display, and to estimate the state of the light source from the obtained information. The object is to provide a robot capable of realizing a head.

In order to solve the above problems, a robot according to the present invention provides three-dimensional model data representing a human or animal head, three-dimensional model display means for displaying the data on a display, and light around the display. In the robot having a camera to detect , the three-dimensional model display means updates the data of the three-dimensional model based on the output of the camera, and displays it on the display by the three-dimensional model display means. The camera image data is binarized and then labeled, and it is determined whether there is an object having a size greater than or equal to a predetermined size. Considering the labeled object as the light source in order from the largest, the size, shape, direction, color, distance, type of the light source is calculated from the image data, It updates the data relates to a light source in the data of the dimensional model, and the structure further comprises a light source estimation means.

Further, in the robot according to the present invention, the light source estimation unit obtains the size, shape, and type of the light source by summing the number of pixels of the portion that has been labeled and regarded as the light source, and each of the light sources constituting the light source Add all the x and y coordinate position information of the pixel and divide by the number of pixels to find the center point of the object, which is the center position of the light source, and input the color at the same position as the center position of this light source from the camera The image was obtained by referring to the image, and this was used as the color of the light source .

Further, in the robot according to the present invention, when there is no object regarded as a light source in the image data obtained by the camera, the light source estimation unit divides the image data vertically and horizontally, finds the brightest range, and has a light source in that direction. The light source is set in that direction.

Furthermore, the robot of the present invention is configured such that when there is no object regarded as a light source in the image data from the camera, the direction in which the light source exists is estimated from the image data, and the shooting direction of the camera is moved in that direction.

According to the above-described robot according to the present invention, there is provided a three-dimensional model display means for displaying data on a display having three-dimensional model data representing a human or animal head, and a display on which the head is displayed. It has light detection means for detecting the intensity, color, etc. of ambient light, and estimates the light source itself existing around the display in the light source estimation means from the detection result of the light detection means,
Update the data on the light source in the data of the three-dimensional model according to the estimation result of the light source,
Since it is configured to be displayed on the display by means of the three-dimensional model display means, it is possible to realistically express the brightness, direction, color, etc. of ambient light with a small number of light detection means, and to make the robot's head inexpensive and comfortable. There is an effect that can be realized.

In the robot according to the present invention, a camera is used as the light detection unit, and the size, shape, direction, color, and distance of the light source in the light source estimation unit are calculated from image data obtained by the camera and shooting information of the camera. The type may be estimated, and the data on the light source in the data of the three-dimensional model may be updated based on the estimation result, and displayed on the display. In this case, the camera has two-dimensional information. Therefore, even with one camera, the robot according to the present invention can be realized. If two cameras are used, an accurate light source distance can be obtained by using a method such as a binocular stereo method. This makes it possible to estimate a more realistic light source.

Further, in the robot according to the present invention, when there is no object regarded as a light source in the image data by the camera, it is estimated from the image data that the light source exists, for example, in a bright direction, and the light source is set in that direction. By setting, the robot according to the present invention can be realized.

Furthermore, in the robot according to the present invention, when there is no object regarded as a light source in the image data obtained by the camera, it is estimated from the image data that the light source exists in a direction where the light source exists, for example, in a bright direction. The configuration may be such that the shooting direction is moved. In this case, the light source can be reliably captured as image data, accurate light source information can be obtained from the image data, and the light source can be estimated with higher accuracy. .

Hereinafter, the best embodiment of the robot according to the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing an example of a system configuration of a robot according to the present invention. As shown in this figure, the robot according to the present invention includes a display 1 that displays the head of the robot, a three-dimensional model display unit 2 that displays a three-dimensional model on the display 1, and the three-dimensional model display unit 2.
Analyzing three-dimensional model data such as light source data 3, material data 4 and shape data 5 as data to be given to the camera 7, and a camera 7 which is one or a plurality of light detection means, and data from the camera 7 The light source estimation unit 6 updates the data to be given to the three-dimensional model display unit 2.
The three-dimensional model display unit 2 and the light source estimation unit 6 are processing units inside the computer, and the three-dimensional model data such as the light source data 3, the material data 4 and the shape data 5 are data inside the computer.

As shown in FIG. 2, the camera 7 is arranged around the display 1 that displays the head, so that the state of the light L striking the display 1 can be detected.
The camera 7 can detect light intensity, color, and the like. The advantage of using the camera 7 as a light detection means is that since the camera can obtain two-dimensional information, the robot according to the present invention can be realized even with one camera, and if the two cameras are used, By using a technique such as the binocular stereo method, an accurate distance of the light source can be obtained, and a more realistic light source can be estimated.

The size, shape, direction, color, distance, and type of the light source are estimated from the image information of one or a plurality of cameras 7. The light source estimation unit 6 performs this estimation process. The light source estimation unit 6 updates the light source data 3 and the like inside the computer. The three-dimensional model display unit 2 uses the updated light source data 3, shape data 5 representing the shape of the head, and material data 4 indicating the color and texture corresponding to the shape data 5 of the head. An image of the head of the robot is displayed on the display 1.
In addition, although the said display 1 can obtain only a rectangular thing at present, the display close | similar to the shape of the head of a human or an animal can also be used.

FIG. 3 is a flowchart for explaining an example of the operation of the robot according to the present invention.
As shown in this flowchart, first, image data is obtained from the camera 7 (P1).
Next, the size, shape, direction, color, distance, and type of the light source are estimated from this image data (P
2). Then, the light source data 3 is updated based on the estimation result, and is displayed on the display 1 together with other three-dimensional model data 4 and 5 (P3).
By repeating the above processing, the light around the robot's head changes, and at the same time, the robot's head displayed on the display 1 can be changed in real time according to the surrounding light. For example, if the light hits from the right, the robot's face appears as if it hits the right. If the surroundings are darkened, the robot's face on the display 1 will be black. It is displayed as if illuminated.

FIG. 4 is a flowchart for explaining in detail an example of the operation of the estimation portion (P2) of FIG.
In this embodiment, first, the image data obtained from the camera 7 is binarized (Q1). Next, a labeling process is performed (Q2). Next, it is determined whether there is an object having a size larger than a predetermined size (Q3). When there is an object having a size greater than or equal to a predetermined size, the labeled object is regarded as the light source in order from the largest, and the position, size, shape, and type of the light source are calculated from the image data (Q4). Next, compared with the image data of the camera 7,
The color of the light source is extracted (Q5). In this way, the surrounding light source is estimated, and the position of the light source in the real space is obtained from the shooting information such as the direction and position of the mounted camera 7 and the light source position in the image (Q8).
These processes are performed, and the operation of the light source estimation part (P2) in FIG. 3 ends.
The binarization process (Q1) and the labeling process (Q2) are performed, and it is determined whether there is an object having a size larger than a predetermined value (Q3). If not, the average value of the luminance of the entire image is obtained, and the presence or absence of the light source is estimated by obtaining the ambient brightness (Q6). That is, if there is no average luminance value of the entire image, it is estimated that there is no light source around. Otherwise, it is determined that there is a light source, and the light source position is estimated (Q7).
).

  As an example of the method of estimating the light source position in Q7 in FIG. 4, for example, the image data is divided into four parts in the vertical and horizontal directions, the brightest range is determined, and it is estimated that there is a light source in that direction. Then, based on this estimation result, the position of the light source in the real space is obtained (Q8), and the operation of the light source estimation part (P2) in FIG. By this method, it is possible to estimate the position of the light source even when there is no object regarded as the light source on the image data.

FIG. 5 is a diagram for explaining in detail an example of the preprocessing of the estimation means for estimating the light source from the images (Q1) and (Q2) in FIG.
The input image (R1) from the camera 7 has x, y coordinate position information, RGB for each pixel.
Color information and brightness information indicating brightness. A binarization process is performed to extract only pixels with high luminance from this input image (R2). However, if binarization processing is performed, even if adjacent pixels are bright, they are not treated as the same object, and therefore the size and shape of the light source cannot be calculated. Therefore, in order to recognize a collection of bright pixels as one object, labeling processing is performed (R3). Here, the pixels of the labeled image (R3) are given numbers for identifying objects in addition to position information and luminance information. By performing these processes, the size of the object can be obtained by summing the shape of the bright part in the image and the number of pixels.

FIG. 6 is a flowchart for explaining in detail an example of the estimation means for estimating the light source from the image that is the (Q4), (Q5), and (Q8) portions of FIG.
When an input image is pre-processed as described above and an object having a size greater than or equal to a predetermined size is present in the labeled image (Q4 start), a plurality of images are regarded as light sources in order from the largest object in the labeled image. Create (S1). Next, the size, shape, and type of the light source are obtained by summing the shape of the parts constituting each light source and the number of pixels, and the position information of the x and y coordinates of each pixel constituting each light source is obtained. All are added, and the center point of the object is obtained by dividing by the number of pixels, and this is set as the center position of the light source (S2). Then, RGB information is obtained by referring to the color at the same position as the center position of the light source based on the input image from the camera 7, and this is used as the color of the light source (S3).
Based on the position information of the light source in the image obtained by these processes and the shooting information such as the position, orientation, and magnification of the camera 7, it is calculated in which direction in the real space and how far away the light source is. (S4). With the processing so far, the position of the light source in two dimensions can be estimated. Therefore, a light source is created at a predetermined distance in the direction obtained by the image in the computer graphic created in three dimensions (S5). By doing so, the light source is estimated from the two-dimensional information to the three-dimensional information.
Here, the light source is estimated using one camera 7, but if two cameras are used, an accurate light source distance can be obtained by using a technique such as a binocular stereo method.
A more realistic light source can be estimated.

FIG. 7 is an explanatory diagram of the three-dimensional model display unit 2 that displays the light source information obtained above.
The figure explains the light that enters the eye E when the light L hits the curved surface S. FIG. Light entering the eye includes diffused light, reflected light, and ambient light, and a method for calculating the light entering the eye when these light components hit inside the computer has been established. When the light entering this eye is performed on each pixel on the display 1, the three-dimensional model is displayed on the display 1 as a realistic image with a shadow.
As this method, there are methods such as phone shading and Guro shading. Such a processing algorithm is realized, for example, in a graphic library called OpenGL. If the light source data 3, the material data 4 and the shape data 5 are inputted, a three-dimensional image with a shadow on the display 1 can be easily obtained. It is done. This processing unit is the three-dimensional model display unit 2.

FIG. 8 is a view showing an example of an overview of the head of the robot in which the camera 7 as the light detection means is movable.
In this embodiment, the motor 8 is used to move the camera 7, and the camera 7 can be moved vertically and horizontally. Although only the camera 7 is moved here, the camera and the display that is the head of the robot may be moved simultaneously.

FIG. 9 is a flowchart for explaining an example of the operation of the movable camera 7 in FIG.
In this flowchart, (Q1) to (Q5) and (Q8) have the same operations as those in the flowchart of FIG. 4 described above, and thus description thereof is omitted.
When binarization processing (Q1) and labeling processing (Q2) are performed and it is determined whether there is an object having a size greater than or equal to a predetermined value (Q3), An average value of the luminance of the entire image is obtained, and it is determined whether or not there is brightness in the surroundings (T1). If it is determined that there is no brightness in the surroundings, it is estimated that there is no light source in the surroundings (T2), and thereafter the same operation as in the flowchart of FIG. 4 is followed. On the other hand, if it is determined that there is brightness in the surroundings, the image is divided into four parts, top, bottom, left, and right, the brightness of each range is obtained, and the brightest direction is detected (T3). Then, the light source is estimated to be in the brightest direction, and the camera is moved in that direction by the motor 8 attached to the camera 7 (T4). After the movement, the above-described light source estimation is performed based on the camera image data obtained again, thereby realizing light source estimation more accurately reflecting the surrounding light source information.

Although the embodiment of the robot according to the present invention has been described above, the present invention is not limited to the embodiment described above, and within the scope of the technical idea of the present invention described in the claims, Naturally, various modifications and updates are possible.

It is the figure which showed an example of the system configuration | structure of the robot which concerns on this invention. It is the figure which showed an example of the general view of the head of the robot which concerns on this invention. It is a flowchart for demonstrating an example of operation | movement of the robot which concerns on this invention. It is a flowchart for demonstrating in detail an example of operation | movement of the light source estimation part by the fixed camera of the robot which concerns on this invention. It is a figure for demonstrating in detail an example of the pre-processing of the image for the light source estimation of the robot which concerns on this invention. It is a flowchart for demonstrating in detail an example of the estimation means which estimates a light source from the image after the pre-processing of the robot which concerns on this invention. It is explanatory drawing of the three-dimensional model display part of the robot which concerns on this invention. It is the figure which showed an example of the general view of the head of the robot which concerns on this invention carrying a movable camera. It is a flowchart for demonstrating in detail an example of operation | movement of the light source estimation part by the movable camera of the robot which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display 2 3D model display part 3 Light source data 4 Material data 5 Shape data 6 Air volume estimation part 7 Camera 8 Motor L Light S Curved surface E Eye

Claims (4)

Three-dimensional model data representing a human or animal head, three-dimensional model display means for displaying the data on a display, and a camera for detecting light around the display , the three-dimensional model display means Is a robot that updates the data of the three-dimensional model based on the output of the camera and displays it on the display by the three-dimensional model display means , and binarizes the image data of the camera, Next, a labeling process is performed, and it is determined whether or not there is an object having a size larger than a predetermined size. If there is an object having a size larger than a predetermined size, the labeled objects are lighted in order from the largest. The size, shape, direction, color, distance, and type of the light source is calculated from the image data, and data on the light source in the data of the three-dimensional model is obtained. New to the robot, characterized in that it further comprises a light source estimation means. The light source estimation means calculates the size, shape, and type of the light source by summing the number of pixels of the portion that has been labeled and regarded as the light source, and the position of the x and y coordinates of each pixel constituting each light source Add all the information, find the center point of the object by dividing by the number of pixels, this is the center position of the light source, obtain the color at the same position as the center position of this light source with reference to the input image from the camera, The robot according to claim 1, wherein this is a color of a light source . When there is no object regarded as a light source in the image data from the camera, the light source estimation means divides the image data vertically and horizontally, finds the brightest range, estimates that there is a light source in that direction, and the light source in that direction characterized in that to set the claim 1, wherein the robot. If there is no object regarded as a light source in the image data by the camera, to estimate the direction in which the light source from the image data exists, characterized by moving the photographing direction of the camera in that direction, according to claim 1 Robot .

Images