JP5306940B2 - Moving image content evaluation apparatus and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image content evaluation device easily and objectively evaluating moving image content by using point-of-gaze map data and saliency map data. <P>SOLUTION: A point-of-gaze data analysis unit obtains an eyesight distribution by using point-of-gaze data including a point-of-gaze coordinate value about image content. An image analysis unit calculates saliency map data corresponding to a plurality of vision properties by applying a parameter for image analysis. A comparison processing unit calculates the degree of matching between the point-of-gaze map data and the saliency map data. A parameter determination unit determines an optimum parameter for evaluation based on the degree of matching. The moving image content to be evaluated without point-of-gaze map data is analyzed based on moving image content for learning having point-of-gaze map data, saliency map data, and a parameter for evaluation, and a point-of-gaze distribution of an observer is estimated. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a moving image content evaluation apparatus and a computer program for evaluating moving image content.

  Broadcasting content, content provided for network distribution services, and content such as video advertisements displayed on information provision devices installed in public facilities can be viewed at the time of video production so that the viewer's attention is drawn. Has been devised. For example, a director or cameraman who is a content creator decides a composition or camera work that is intended to attract or attract attention from viewers based on experience-based know-how or video production expertise. There are many cases. Under such circumstances, producers are required to objectively know how content is viewed by viewers and use it for production.

  By the way, with the act of human beings looking at things, the line of sight moves and is directed in a direction that attracts attention and interest in the field of view. Therefore, various studies have been conducted using eye movement, which is a human biological reaction. For example, an eye movement of an observer observing a moving image displayed on the screen is captured by a camera, and the movement of the line of sight is measured from the captured image, and the position of the moving image on the screen (note A technique for determining whether or not it is directed to the (viewpoint) is known. And the technique which evaluates a video content based on the measurement result of a motion of an observer's gaze using the technique is proposed (for example, refer to patent documents 1 and patent documents 2).

  The video content evaluation technique described in Patent Document 1 is a technique for imaging and analyzing the movement of the eyeball of a subject and presenting the eye movement data obtained thereby in synchronization with the reproduction of the video. Specifically, this document describes the number of blinks, pupil diameter change, reaction time, eye movement speed, eye retention time, and number of stops for each frame image from the state of the eyeball of the subject imaged by an infrared camera. , And a video content evaluation apparatus for calculating the position thereof. And it is disclosed that those calculation results are visualized by a graph or the like.

  The image evaluation apparatus described in Patent Literature 2 obtains the distribution of a gazing point for each frame image of moving image data using a probability density function based on coordinate data of the gazing point measured for a plurality of viewers. The total entropy is calculated. Then, using these gazing point groups comprehensively, the probability of which position the line of sight gathers in the frame image is calculated as the degree of concentration with respect to the frame image, and the content is evaluated. At this time, it is possible to visualize where the gazing points are gathered in the frame image by displaying a contour map showing the distribution of the degree of concentration superimposed on the frame image.

  On the other hand, a technique related to a saliency map that is used to estimate a distribution of ease of attention when a viewer views an image is known (for example, see Non-Patent Document 1). In the evaluation of an image using this saliency map, it is not necessary to conduct a visual evaluation experiment by a subject, and it is possible to evaluate the ease of attention of an image only by analyzing the physical characteristics of the image data. it can.

  The saliency map can be obtained by, for example, a two-step process including a feature map generation step and a feature map synthesis step. Of these, in the feature map generation step, a feature map is generated by performing image analysis on one or more visual attributes of the frame image. For example, six attributes of color, intensity, orientation, orientation, contrast, flicker, and motion can be used as visual attributes. In this case, in the feature map generation step, six feature maps are generated. In the feature map synthesis step, a weighted linear sum of the feature maps generated for each visual attribute is calculated to obtain a saliency map.

JP 2004-282471 A JP 2007-310454 A

Laurent Itti, Christof Koch, Ernst Niebur, “A Model of Salientity-Based Visual Attention for Rapid Scenario Analysis,” IEEE Transactons Int. 20, no. 11, pp. 1254-1259, November 1998.

  However, in the conventional technology that evaluates the image content based on the visual line measurement result of the subject, it is necessary to repeat the visual line measurement experiment every time the image content to be evaluated is changed. In other words, with this prior art, it is an iterative process that once the image content is produced, the eye gaze measurement experiment is performed to analyze the distribution of the gazing point, and the image content is produced again based on the result and the eye gaze measurement experiment is performed again. Is required. Thus, the work of repeating the image content production and the evaluation experiment requires a lot of time and labor. In addition, when the number of samples of the subject is increased, the labor is further increased. Therefore, there is a need for a method for simply and efficiently evaluating image content while suppressing the number of subjects' samples.

  In the evaluation method using the above-described saliency map, it is not necessary to perform a gaze measurement experiment on the subject. However, when the saliency map obtained from the sample image is compared with the line-of-sight measurement experiment result for the sample image, the region with high saliency and the region where the gazing point concentrates may not match. This is not only the visual characteristics (bottom-up factor) of the image itself, which is the attribute used to calculate the saliency map, but also the subject's preference, interest, experience, knowledge This is thought to be due to factors such as top-down factors.

  In order to reduce the influence of such top-down factors as much as possible, a method of setting image analysis parameters related to physical characteristics in image analysis processing for generating a saliency map to a statistically appropriate value is required. It is done. However, conventionally, when using a saliency map calculation tool, it is usual to use the initial setting value as it is or to use a value obtained through experience. In other words, the conventional technique has not sufficiently studied the method for setting the image analysis parameter.

  The present invention has been made on the basis of the above problem recognition, and makes it easy and objective for moving image content using the distribution of gaze points based on gaze measurement and the distribution of saliency based on image analysis processing. It is an object of the present invention to provide a moving image content evaluation apparatus and a computer program that can be evaluated in a simple manner.

[1] In order to solve the above-described problem, a moving image content evaluation apparatus according to an aspect of the present invention provides an image relating to visual attributes for moving image contents of a plurality of learning moving image contents and evaluation target moving image content. An image analysis unit that performs video analysis based on analysis parameters and generates saliency map data indicating saliency distribution corresponding to pixels included in the moving image content; and visual acuity distribution related to the learning moving image content A comparison processing unit that calculates a degree of coincidence that is an index of similarity between the gazing point map data and the saliency map data based on the gazing point map data and the saliency map data; and the image analysis unit The ratio of the learning video content based on a result of video analysis based on different image analysis parameters for each of the learning moving image contents. Processor selects based on the Evaluation Technical parameters for image analysis degree of coincidence between said gazing point map data calculated the saliency map data, as an initial value of the evaluation parameter of the parameter for image analysis said selected A parameter determining unit that determines a final evaluation parameter by a steepest gradient method using an initial value of the evaluation parameter, and determines the final evaluation parameter as an optimal image analysis parameter; and the evaluation target moving image a similar image content determination unit that determines a moving image content for similar learning is a moving image content for learning that is similar to the content, the parameters determined with respect to the similar learning moving image contents determined by the similar image content determination unit the image analysis unit is the evaluation pair based on the optimal image analysis parameter section decides A gazing point map estimation unit that outputs the saliency map data generated by performing video analysis of moving image content as gazing point map data estimated as the visual acuity distribution of the evaluation target moving image content; It is characterized by.
In the above configuration, the visual acuity distribution represents the distribution of the visual acuity of the person at the gaze point and its surroundings. At this time, the visual acuity at the peripheral portion thereof gradually decreases with respect to the visual acuity at the gazing point according to the distance from the gazing point. Moreover, based on the result of measuring the gazing point for a plurality of subjects, a superimposition of the visual acuity distributions of the plurality of subjects may be used as the above visual acuity distribution. This visual acuity distribution represents the degree of human gaze, and can be said to be a visual acuity distribution.
The saliency distribution is, in other words, an attractive distribution. Attraction is the degree of ease of visual attention.
The visual attributes may be one type or a plurality of types. The image analysis parameter represents weighting for each visual attribute, for example.
In addition, the parameter determination unit calculates and calculates the similarity between the gazing point map data related to one learning moving image content for obtaining the evaluation parameter and the gazing point map data related to a plurality of other learning moving image contents. Based on the similarity, the gaze point map data related to one or a plurality of learning moving image contents is selected, and the image analysis parameter associated with the selected gaze point map data is selected. Next, the parameter determination unit is calculated by the image analysis unit based on the gazing point map data relating to one learning moving image content for obtaining the evaluation parameter and the selected one or more image analysis parameters. Of the matching degrees calculated by the comparison processing unit based on the saliency map data, an image analysis parameter indicating the highest matching degree is determined as an evaluation parameter. Further, the parameter determination unit similarly determines the evaluation parameters for other learning moving image contents.
In other words, the parameter determination unit determines one or a plurality of learning based on the similarity between the gazing point map data of the learning moving image content and the gazing point map data of other learning moving image content for obtaining the evaluation parameter. An image corresponding to the gazing point map data having the highest degree of coincidence with the saliency map data of the learning moving image content corresponding to the narrowed gazing point map data, narrowed down to the gazing point map data of the moving image content An evaluation parameter is determined based on the analysis parameter.
The similar image content determination unit uses, for example, a feature map relating to a predetermined visual attribute using an evaluation parameter for generating common saliency map data for the evaluation target moving image content and the learning moving image content. By determining the similarity of the obtained saliency map data, a learning moving image content similar to the evaluation target moving image content is determined. Alternatively, clustering processing based on the characteristics of moving image content may be performed, and similarity between moving image contents may be determined based on the result.

  [2] Furthermore, the moving image content evaluation apparatus according to an aspect of the present invention is based on gazing point data including coordinate values of a gazing point, which is a result of measuring the line of sight with respect to the plurality of learning moving image contents. A gazing point data analysis unit that generates gazing point map data indicating visual acuity distribution corresponding to pixels included in the learning moving image content is further provided, and the comparison processing unit relates to each of the plurality of learning moving image contents. Based on the saliency map data and the gazing point map data generated by the gazing point data analysis unit, a degree of coincidence that is an index of similarity between the gazing point map data and the saliency map data is calculated. It is characterized by that.

[3] Further, the computer program of the present invention causes the computer to perform video analysis based on image analysis parameters related to visual attributes for each of the plurality of moving image contents for learning and the moving image content to be evaluated. And an image analysis unit that generates saliency map data indicating saliency distribution corresponding to pixels included in the moving image content, gazing point map data indicating visual acuity distribution related to the learning moving image content, and the saliency map Each of the learning moving image content in the image analysis unit and a comparison processing unit that calculates a degree of coincidence that is an index of similarity between the gazing point map data and the saliency map data. Calculated by the comparison processing unit based on the results of video analysis based on different image analysis parameters Serial select based-out parameters for image analysis degree of coincidence between the gazing point map data wherein the saliency map data, to determine the parameters for the selected image analysis as the initial value of the evaluation parameter, the evaluation A parameter determination unit that obtains a final evaluation parameter by the steepest gradient method using an initial value of a parameter for determination, and determines the final evaluation parameter as an optimal image analysis parameter; and learning similar to the evaluation target moving image content a similar image content determination unit that determines a moving image content for similar learning a use moving image content, wherein the parameter determining unit to said similar learning moving image content as determined by the similar image content determination unit has determined the the evaluation target moving image content is the image analysis unit based on the optimal image analysis parameters The saliency map data generated by performing a video analysis, gazing point map estimation unit for outputting as a gazing point map data that is estimated to the visual acuity distribution of the evaluation target moving image contents, to function as a.

[4] Furthermore, in order to solve the above-described problems, the following mode may be adopted.
Note indicating the visual acuity distribution corresponding to the pixels included in the learning moving image content based on the gazing point data including the coordinate value of the gazing point as a result of measuring the line of sight for each of the plurality of learning moving image contents. Pixels included in the learning moving image content by generating viewpoint map data, performing video analysis for each of the plurality of learning moving image contents for each parameter based on a plurality of image analysis parameters related to visual attributes And generating saliency map data indicating a saliency distribution corresponding to each of the plurality of learning moving image contents based on the gazing point map data and the saliency map data. And a degree of coincidence that is an index of similarity between the saliency map data and each of the plurality of learning moving image contents Then, the image analysis parameter when the degree of coincidence between the gazing point map data and the saliency map data is the highest is determined as the evaluation parameter of the learning moving image content, and the input evaluation target moving image The learning moving image content most similar to the content is determined from the plurality of learning moving image contents, and the video of the evaluation target moving image content is determined based on the evaluation parameters of the determined most similar learning moving image content A moving image content evaluation method comprising: generating saliency map data by performing analysis, estimating and outputting the saliency map data as gazing point map data of the evaluation target moving image content.

  According to the present invention, in order to easily and objectively evaluate image content, it is based on the distribution of the gazing point obtained by the gaze measurement experiment of the learning moving image content and the video analysis processing on the learning moving image content. Image analysis parameters can be acquired using similarity to the saliency distribution. Then, a line-of-sight measurement experiment is performed by using the image analysis parameters of the learning moving image similar to the evaluation target moving image content as the image analysis parameters for generating the saliency map for the evaluation target moving image content. Even if there is no evaluation target moving image content, it can be estimated as the distribution of the viewer's gaze point.

It is a block diagram which shows the function structure of the image content evaluation apparatus by one Embodiment of this invention. It is the figure which showed the example of the data structure stored in the data storage part in the embodiment. 2 is a functional configuration diagram of a gazing point data generation device in the embodiment. FIG. It is a data block diagram of the gaze point data recorded on the storage part in the embodiment. In the embodiment, the gazing point data analysis unit is a flowchart showing a procedure for generating gazing point map data for each frame image from gazing point data for image content. In the same embodiment, it is a figure showing typically signs that a subject looked at by observing a display screen, and coordinates of a gazing point on a display screen. It is an example of the three-dimensional graph showing the relationship between the eccentricity with respect to a gaze direction and the peripheral visual acuity value in the same embodiment. It is an example of the three-dimensional graph of gaze point map data in the embodiment. In the same embodiment, it is a flowchart which shows the procedure in which an image analysis part produces | generates the saliency map data about an image content. It is the figure which showed the data structure of the parameter for image analysis which the image analysis part sets in the same embodiment. 4 is an example of a three-dimensional graph of a saliency map generated for a referenced frame image in the embodiment. 5 is a flowchart illustrating a procedure in which a comparison processing unit calculates a degree of coincidence between gaze point map data and saliency map data for image content in the embodiment. In the embodiment, the parameter determination unit is a flowchart showing a procedure for determining an evaluation parameter for generating saliency map data by using gaze point map data and image analysis parameters related to learning moving image content. . It is a flowchart which shows the procedure of the estimation process of the gaze point map data of the image content evaluation apparatus in the embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a functional configuration of an image content evaluation apparatus according to an embodiment of the present invention. In the figure, the image content evaluation apparatus 1 is similar to a data storage unit 11, a gazing point data analysis unit 12, an image analysis unit 13, a comparison processing unit 14, a parameter determination unit 15, and an image input unit 21. An image content determination unit 22 and a gazing point map estimation unit 23 are included.

  FIG. 2 is a schematic diagram illustrating a configuration example of data stored in the data storage unit 11. The data storage unit 11 is realized using a semiconductor memory, a magnetic hard disk, or the like. As shown in FIG. 5A, the data storage unit 11 stores image content for allowing a subject to observe or perform image analysis processing. This image content is data that can be controlled in units of frame images, and includes a time code for each frame image. The contents of the image content used in the present embodiment are those in which the influence of the top-down factor is reduced as much as possible or eliminated.

  For example, as described in known literature (Ran Carmi and Laurrent Itti, “Causal Saliency Effects During Natural Vision”, Proc. Of Symposium on Eye Tracking Research & Applications, pp. 11-18, March, 2006.) A moving image or an unknown moving image that does not include a cognitive meaning in the content of the image content for the subject is used as the image content. Or even if it is a general moving image, it is considered that the influence of the top-down factor can be reduced or eliminated by using a moving image having a playback time of about several seconds. A moving image is used as image content. Thereby, it is possible to capture the movement of the line of sight induced only from the physical characteristics of the image without giving a margin to the subject to consider and without depending on the top-down factor.

  The image content may be moving image content including a plurality of frame images, or may be still image content that is a single frame image. In the present embodiment, an example in which moving image content (evaluation target moving image content, learning moving image content) is used as the image content will be described.

  Further, the data storage unit 11 matches the gazing point data, the gazing point map data, the feature map data, and the saliency map data in correspondence with the frame images (# 1 to #N) of the image content. Store the degree.

  The gazing point data is data including coordinate values of the gazing point obtained by measuring the line of sight of one or a plurality of subjects. The gazing point map data is data indicating a distribution of gazing points calculated in consideration of the peripheral visual field of the gazing point based on the gazing point data. The feature map data is data indicating the distribution of feature amounts obtained for each visual attribute of the frame image. The saliency map data is data indicating a distribution of saliency (ease of attention to an image) obtained by calculating a weighted linear sum of feature map data, that is, an attractiveness distribution. The degree of coincidence is data indicating the degree of coincidence, which is an index of similarity between the gazing point map data and the saliency map data. Each map data shown here is data of a matrix corresponding to a pixel of a frame image having the number of horizontal pixels W × the number of vertical pixels H, and each element of the matrix is a scalar value.

  As shown in FIG. 2B, the data storage unit 11 stores an evaluation value and an image analysis parameter in association with the image content. The evaluation value is a value obtained by evaluating the degree of coincidence between the gazing point map data and the saliency map data as the entire image content. Image analysis parameters (especially image analysis parameters in the case of moving image content are called video analysis parameters) are settings for calculating saliency map data by linearly summing each feature map data Information, including weight data for each visual attribute.

Returning to FIG. 1, the gazing point data analysis unit 12 generates gazing point data evaluation index data related to the gazing point by analyzing the gazing point data for one or a plurality of subjects stored in the data storage unit 11. In other words, the gazing point data analysis unit 12 generates gazing point map data indicating a visual acuity distribution corresponding to the pixels included in the image content based on the gazing point data including the coordinate value of the gazing point corresponding to the image content. .
The image analysis unit 13 generates saliency evaluation index data, which is evaluation index data related to saliency, by image analysis processing using the physical feature amount of the frame image. In other words, the image analysis unit 13 calculates feature amount data for each visual attribute corresponding to the pixel based on the image content, and handles the pixel based on the feature amount data and the weight data determined for each visual attribute. The saliency map data indicating the distribution of the saliency is generated.

The comparison processing unit 14 compares the generated gazing point data evaluation index data and the saliency evaluation index data, respectively, and calculates the degree of coincidence between the gazing point distribution and the saliency distribution. In other words, the comparison processing unit 14 is a match that is an index of similarity between the gazing point map data and the saliency map data based on the gazing point map data and the saliency map data regarding the image content. Calculate the degree.
The parameter determination unit 15 determines an evaluation parameter used for calculating the saliency map data in the image analysis unit 13 based on the degree of coincidence.

When the evaluation target moving image content to be evaluated is supplied from the outside, the image input unit 21 inputs the evaluation target moving image content to the image content evaluation device 1 and stores it in the data storage unit 11.
The similar image content determination unit 22 determines a similar learning moving image content that is a learning moving image content similar to the evaluation target moving image content from among a plurality of learning moving image contents.
The gazing point map estimation unit 23 uses the evaluation parameter in the similar learning moving image content as an evaluation parameter for calculating the saliency map data for the evaluation target moving image content, and the saliency map generated by the image analysis unit 13. The data is output as estimated gaze point map data.

  Next, means for generating gaze point data stored in advance in the data storage unit 11 and a generation method will be described. The gazing point data is generated when the gazing point data generation device, which is a device different from the image content evaluation device 1, performs gaze measurement for one or more subjects. This gaze point data generation process is positioned as a pre-process for the image content evaluation process.

  FIG. 3 is a block diagram illustrating a functional configuration of the gazing point data generation device. In the figure, the gazing point data generation device 3 includes an image reproduction unit 31, an image display unit 32, a gazing point data measurement unit 33, a gazing point data recording unit 34, and a storage unit 35. . The image reproduction unit 31 reads out and reproduces image content for allowing the subject to observe from the storage unit 35. The image display unit 32 displays the reproduced image content on the screen. The gaze point data measurement unit 33 measures the movement of the eyeball of the subject who observes the moving image displayed on the image display unit 32, and measures the coordinate value of the gaze point that is the position on the screen. The gazing point data recording unit 34 records the coordinate value of the gazing point in the storage unit 35 in synchronization with the reproduction of the image content.

  Next, a more specific configuration and operation of the gazing point data generation device 3 will be described. The gazing point data measuring unit 33 includes a gazing point measuring device 33a for measuring the movement of the eyeball of the subject. This gaze point measuring device 33a may be based on the prior art, for example, a type that detects a gaze point by attaching a visual sensor to the face of the subject, or a type that measures a line of sight by attaching a contact lens or goggles. be able to.

  The gaze point measuring device 33a of the present embodiment uses a measurement method based on the pupil corneal reflection method that detects the point of gaze by remote sensing the movement of the eyeball based on the captured image. The gaze point measuring device 33a irradiates the eyeball portion of the subject who observes the screen from a position at a certain distance from the display surface of the image display unit 32 with near infrared light, and uses the camera to display a corneal reflection image on the surface of the eye. Take measurements to image. Then, the center point of the pupil and the corneal reflection point are detected from the captured image, and the direction of the line of sight and the coordinate value of the gazing point on the screen are calculated geometrically.

  The resolution of measurement by this remote sensing is about 0.5 to 1 degree. Taking an HDTV (High Definition Television) image with an effective pixel count of 1,920 pixels per line as an example, when the viewing angle in the horizontal direction of the screen is 30 degrees, it is about 32 to 64 pixels. Corresponds to resolution. The gazing point data recording unit 34 records the coordinate value of the gazing point measured by the gazing point data measuring unit 33 in the storage unit 35 as the gazing point data in synchronization with the reproduction of the image content by the image reproducing unit 31. That is, each frame image included in the image content is associated with the gazing point data.

  FIG. 4 is a schematic diagram showing the data structure of the gazing point data. As shown in the figure, the gazing point data includes the frame number of the frame image of the image content, the time code of the frame, and the coordinate value of the gazing point for each subject. The time code is time information counted from the first frame image of the image content, and is represented by “hour: minute: second.number of frames”. The frame number is a number that is incremented by 1 in the time code chronological order, with the first frame image of the image content being 1. The coordinate value of the gazing point is a two-dimensional coordinate system on a frame image of horizontal pixel count W × vertical pixel count H with the coordinate at the upper left corner of the display area of the frame image displayed on the image display unit 32 as the origin. expressed. For example, when the time code is “0: 00: 05.00”, the coordinate value of the gazing point of the subject 1 is (175, 122), that of the subject 2 is (168, 145), and so on. It is (166,260).

  The image content evaluation apparatus 1 uses the gazing point data stored in the storage unit 35 by the above gazing point data generation process by taking it into the data storage unit 11. This completes the description of the gazing point data generation process, which is a pre-process.

Next, the operation will be described by dividing the image content evaluation process by the image content evaluation apparatus 1 into a gaze point data analysis process, an image analysis process, a comparison process, and a parameter determination process.
<Gaze point data analysis processing>
FIG. 5 is a flowchart illustrating a procedure in which the gazing point data analysis unit 12 generates gazing point map data for each frame image from the gazing point data regarding the image content. In step S51, the gazing point data analysis unit 12 refers to the gazing point data stored in the data storage unit 11 in units of frame images. In step S52, when there is no gazing point data to be referred to (step S52: NO), the process of this flowchart is ended. On the other hand, if there is gazing point data (step S52: YES), the process proceeds to step S53. In step S <b> 53, the gazing point data analysis unit 12 reads the gazing point data of all subjects for the referenced one frame image.

  Next, in step S54, the gazing point data analysis unit 12 generates gazing point map data considering the peripheral visual field. Here, the generation process of the gazing point map data will be described in detail. According to the knowledge in the field of visual science regarding eye movement, the relational expression (1) is established for the relation between the line of sight and the peripheral visual field. Here, E is the degree of eccentricity with respect to the visual line direction, Vf is the visual acuity in the visual line direction, Es is a predetermined constant, and V is the peripheral visual acuity value.

  FIG. 6 is a diagram schematically illustrating a state in which a subject turns his / her line of sight by observing the display screen and coordinates of a gazing point on the display screen. FIG. 6A shows a moving image on the display screen 61 from a position on the extension of the axis perpendicular to the display screen 61 and passing through the center point of the screen, and away from the display screen 61 by a distance L. It shows a state of observing. When the coordinate value of the gazing point at the time indicated by a certain time code is (GX, GY), it indicates that the subject's line of sight is directed to the point P (GX, GY). Here, attention is paid to the visual field at the coordinates (X, Y) on the display screen 61 that is an angle of the eccentricity E with respect to the line of sight of the subject. Here, since the visual acuity distribution in the peripheral visual field at the point P can be approximated to be a perfect circle, the radius R with the gazing point coordinates (GX, GY) as the center as shown in FIG. Visual acuity is constant on the circumference. In the figure, coordinate values X and Y take integer values in the range of 1 ≦ X ≦ W and 1 ≦ Y ≦ H, respectively. Here, W and H are the number of horizontal pixels and the number of vertical pixels of the frame image, respectively.

  At this time, the peripheral visual acuity value V [X, Y] at the coordinates (X, Y) is expressed by Expression (2) obtained by modifying Expression (1).

  Here, atan is an arctangent function. In addition, since the direction of the line of sight with respect to an arbitrary pixel on the screen is substantially perpendicular to the screen, approximation using an arctangent function can be performed to calculate the eccentricity E [X, Y]. The visual acuity Vf in the line-of-sight direction may be set for each subject or may be a common setting.

  FIG. 7 is a three-dimensional graph showing the peripheral visual acuity value V [X, Y] in the degree of eccentricity E [X, Y] with respect to the line-of-sight direction, which is obtained by the gaze point data analysis unit 12 by calculation of Expression (2). The figure shows an example of a frame image having a horizontal pixel count W = 320 pixels and a vertical pixel count H = 240 pixels.

  The peripheral visual acuity value V [X, Y] obtained by the calculation of Expression (2) is expressed as matrix data of the number of elements of horizontal pixel count W × vertical pixel count H, and visual acuity distribution based on subject's gaze data Represents. This is called gaze point map data. That is, the gazing point map data GMs (f) of the subject s in the frame number f is expressed as in Expression (3).

  Returning to FIG. 5, in the process of step S <b> 54, the gazing point data analysis unit 12 linearly sums the gazing point map data for all the subjects for the referenced frame image, and the gazing point map data that is linearly summed The gazing point data evaluation index data in the frame image is used. That is, the gazing point map data GM (f) in the frame image with the frame number f is obtained by the calculation of Expression (4).

  The constant cs may be a different value for each subject or may be a constant value (for example, CS = 1.0 for all subjects).

  FIG. 8 is a three-dimensional graph of the gazing point map data GM (f) obtained by the gazing point data analysis unit 12 by the calculation of Expression (4). The figure shows an example of a frame image having a horizontal pixel count W = 320 pixels and a vertical pixel count H = 240 pixels.

  Returning to FIG. 5 again, in step S55, the gazing point data analysis unit 12 records the gazing point map data GM (f) in the referenced frame image in the data storage unit 11. Then, the process returns to step S51.

  As described above, the gazing point data analysis unit 12 calculates the gazing point data evaluation index data in consideration of the visual acuity distribution of the peripheral visual field, thereby adapting the characteristics of the human eye movement from the measured gazing point. Evaluation index data can be created. That is, even when the number of subjects is small, it is possible to efficiently obtain the concentration of the gazing point with respect to all the pixels of the frame image.

  Note that the gazing point data analysis unit 12 clusters the gazing point distributions for a plurality of subjects by using a clustering method based on existing technology, and then adds all the distributions for each cluster to form a mixed normal distribution. You may make it obtain | require viewpoint map data.

<Image analysis processing>
As the visual attributes used when generating the saliency map data, the image analysis unit 13, for example, as described above, for example, color, intensity, orientation, contrast, flicker, And six attributes of motion. The color attribute is a color attribute value that is a pixel color value. The brightness attribute is a brightness attribute value of a pixel. The azimuth attribute is, for example, the sum of the line component strength of each of the four azimuths of 0 degrees, 45 degrees, 90 degrees, and 135 degrees when the horizontal direction is the reference azimuth for each pixel to obtain an azimuth attribute value. Is. Note that the strength of the line component of a predetermined azimuth is calculated based on, for example, the ratio between the image differential value in the direction of the azimuth and the image differential value in the direction orthogonal thereto. The contrast attribute is a contrast attribute value that is a contrast value calculated by a ratio between a pixel value of a region including the pixel and a pixel value of another region. The blinking attribute is an attribute value calculated based on the frequency itself and the amplitude of the frequency component when the change in the pixel value in the time direction of the region including the pixel has a predetermined frequency component. The motion attribute is an attribute value that takes into account the size of the pattern and its moving speed when a predetermined pattern in the frame image moves in a predetermined direction as time passes. In generating the saliency map data, attribute values corresponding to one or more types of visual attributes may be used, but in the present embodiment, the six types of attribute values are used as physical feature amounts corresponding to the visual attributes. Use.

  FIG. 9 is a flowchart illustrating a procedure in which the image analysis unit 13 generates saliency map data for image content. In step S91, the image analysis unit 13 sets parameters for image analysis for generating saliency map data. The image analysis parameters are six weight data corresponding to physical feature amounts in the image analysis processing of the image content.

  FIG. 10 shows the data structure of the image analysis parameters set by the image analysis unit 13. As shown in the figure, the image analysis parameter has weight data corresponding to six types of physical feature amounts. wc is weight data of the physical feature quantity CC corresponding to the color attribute, wi is weight data of the physical feature quantity CI corresponding to the brightness attribute, wo is weight data of the physical feature quantity CO with respect to the orientation attribute, and wr is a contrast attribute. , Wj is the weight data of the physical feature quantity CJ corresponding to the blinking attribute, and wm is the weight data of the physical feature quantity CM corresponding to the motion attribute.

  Returning to FIG. 9, next, in step S92, the image analysis unit 13 refers to the image content stored in the data storage unit 11 in units of frame images. In step S93, when there is no frame image to be referred to (step S93: NO), the processing of this flowchart is ended. On the other hand, when the frame image is referred to (step S93: YES), the process proceeds to step S94. In step S94, the image analysis unit 13 reads the referenced frame image.

  Next, in step S95, the image analysis unit 13 generates saliency map data. The image analysis unit 13 generates saliency distribution data estimated based on the feature amounts corresponding to all visual attributes based on the image analysis parameters set in the process of step S91. Specifically, the image analysis unit 13 performs image analysis processing related to six visual attributes on the read frame image to generate a feature map for each visual attribute. Then, the image analysis unit 13 calculates a weighted linear sum of these feature maps to generate saliency map data that is saliency evaluation index data. The image analysis unit 13 calculates a weighted linear sum F [i, j] of feature amounts corresponding to the pixel (i, j) of the frame image having the number of horizontal pixels W × the number of vertical pixels H by the following equation: Calculate using (5).

  Then, the saliency map data SM (f) for the entire frame image is expressed as in Expression (6).

  FIG. 11 is an example of a three-dimensional graph of the saliency map generated for the referenced frame image. The figure shows an example of a frame image having a horizontal pixel count W = 320 pixels and a vertical pixel count H = 240 pixels.

  Returning to FIG. 9, next, in step S <b> 96, the image analysis unit 13 records the saliency map data in the referenced frame image in the data storage unit 11. Then, the process returns to step S92.

<Comparison process>
When gaze point data evaluation index data and saliency evaluation index data are created using image content that has at least reduced the influence of top-down factors, the areas where the gazing point concentrates and the areas with high saliency in the image , At least partially overlap or close. Therefore, the comparison processing unit 14 obtains a degree of coincidence that is an index of similarity between the distribution of the gazing point data evaluation index data and the distribution of the saliency evaluation index data. As the values of the respective matrices are similar between the gazing point map data that is the gazing point data evaluation index data and the saliency map data that is the saliency evaluation index data, the value indicating the degree of coincidence increases. Specifically, the comparison processing unit 14 compares both data of the gazing point map data GM (f) represented by Expression (4) and the saliency map data SM (f) represented by Expression (6). To calculate the degree of coincidence.

FIG. 12 is a flowchart illustrating a procedure in which the comparison processing unit 14 calculates the degree of coincidence by comparing the gazing point map data and the saliency map data for one image content. In step S <b> 121, the comparison processing unit 14 refers to gaze point map data for one frame image included in a certain image content stored in the data storage unit 11.
Then, in the next step S122, it is determined whether or not there is data referred to in step S121. If there is referenced gazing point map data (step S122: YES), the process proceeds to the next step S123. On the other hand, if there is no gazing point map data to be referred to, that is, if the processing from step S123 on all the frame images included in the image content is completed (step S122: NO), the processing in step S127 is performed. move on.

  In step S123, the comparison processing unit 14 reads gaze point map data for the referenced frame image. Next, in step S <b> 124, the comparison processing unit 14 reads the saliency map data for the referenced frame image from the data storage unit 11. In step S125, the comparison processing unit 14 calculates coincidence evaluation map data from the read gazing point map data and saliency map data. Next, in step S126, the degree of coincidence in the referenced frame image is calculated. Then, the process returns to step S121.

The matching degree calculation processing in steps S125 and S126 will be described with three specific examples.
The process by the 1st method calculates | requires the difference value of each element of gaze point map data and saliency map data, and makes it coincidence evaluation map data. That is, the comparison processing unit 14 matches the gazing point map data GM (f) and the saliency map data SM (f) with a coincidence evaluation map that is an absolute value of a difference between elements corresponding to the position (i, j) of the matrix. Data DM [i, j] is calculated by equation (7).

  Then, the comparison processing unit 14 compares the coincidence evaluation map data DM [i, j] calculated by the equation (7) with a predetermined threshold value, and the difference value DM [i having a smaller value than the threshold value. , J]. And the number which is a count result is made into a coincidence degree.

  The process according to the second method is to obtain the similarity between the gazing point map data and the saliency map data. That is, the comparison processing unit 14 creates a histogram from each of the gazing point map data GM (f) and the saliency map data SM (f), and calculates the absolute value of the difference between the two in element units of the histogram. The total value is used as the degree of coincidence. In the histogram, when each element of the gazing point map data GM (f) and the saliency map data SM (f) can take a value from 0 to 1, for example, each element is divided into 10 parts by 0.1. The number of occurrences of data for each category is counted.

  The process by the 3rd method calculates | requires the product for every element of gaze point map data and saliency map data, and makes it coincidence evaluation map data. That is, the comparison processing unit 14 matches the evaluation map data MM that is the product value of elements corresponding to the position (i, j) of the matrix for the gazing point map data GM (f) and the saliency map data SM (f). [I, j] is calculated by the equation (8).

  Then, the comparison processing unit 14 compares the coincidence evaluation map data MM [i, j] calculated by the equation (8) with a predetermined threshold value, and a product value MM [i that has a value larger than the threshold value. , J]. And the number which is a count result is made into a coincidence degree.

  Then, the comparison processing unit 14 repeats the processing from step S121 to S126 to calculate the degree of coincidence for all the frame images constituting the target image content, and then proceeds to the processing of step S127. In step S127, the comparison processing unit 14 calculates an evaluation value that is the matching degree of the entire image content based on the matching degree of each frame image.

  The evaluation value is calculated as follows. For example, an average value of coincidences for all the frame images constituting one image content is obtained and used as an evaluation value. Alternatively, an integral value with respect to a temporal change in the degree of coincidence of all frame images is obtained and used as an evaluation value.

  Next, in step S128, the comparison processing unit 14 records the calculated evaluation value in the data storage unit 11.

  In the above-described gazing point data generation process, the gazing point data is measured in synchronization with the reproduction of the image content, and the gazing point data obtained at the same time as the frame image reproduction time is recorded. However, the human eye has a characteristic of moving the line of sight after a short time lag from when an image enters the field of view as a physiological response. In consideration of this characteristic, the calculation time of the gazing point data corresponding to the frame image may be delayed from the reproduction time of the frame image by a time corresponding to the time lag.

  That is, the value of the number of frames corresponding to this time lag is stored in advance, and the comparison process generates saliency map data when calculating the degree of coincidence by comparing the gazing point map data with the saliency map data. The degree of coincidence may be calculated using gaze point map data delayed by the number of frames from the time.

  Note that the delay time Td of the collection time of the gazing point map data to be compared corresponding to the generation time of the saliency map can be obtained as follows. By analyzing a frame image of a certain time code and the frame images before and after the frame image, a location where a change in physical characteristics in a partial region at the same position in the frame image is larger than a predetermined threshold is detected. When the time code of the frame image thus detected is T1, the movement of the subject's line of sight is analyzed, and if a saccade is detected at a time point after the time Tb has elapsed from the time code T1, And the time code T2 = T1 + Tb at that time are recorded. In this case, the time Tb can be regarded as the delay time Td. The saccade is a rapid eye movement that occurs when the line of sight is shifted, and is also called a jumping eye movement.

<Parameter determination process>
To optimize the combination of weight data for image analysis parameters used to generate image content saliency map data, for example, calculation using various image analysis parameters using the steepest gradient method Can be used to search for an optimal solution. However, simply using such a method requires a huge amount of calculation and time. Therefore, in the present embodiment, the parameter determination unit 15 uses the gazing point map data of the learning moving image content and the image analysis parameter, for which the degree of coincidence has been calculated in advance, and is used for the determination of the evaluation parameter. Processing for determining an optimal image analysis parameter as an evaluation parameter for generating saliency map data of the moving image content (evaluation parameter determination target learning moving image content) is executed.

  The data storage unit 11 includes a learning image database (not shown) inside. The learning image database stores one or a plurality of learning moving image contents, and further includes gazing point map data and saliency map data of each of these learning moving image contents, and these gazing point map data and saliency map. Data of the degree of coincidence with the data and image analysis parameters used when calculating the saliency map data are stored. When there are a plurality of learning moving image contents, the weight data of the image analysis parameters is made different for each learning moving image content. Here, the learning image database may not have saliency map data corresponding to the learning moving image content among the data items. Conversely, the learning image database may not hold data on the degree of coincidence among the above data items. At this time, the degree of coincidence can be calculated from the gazing point map data and the saliency map data of the moving image content for learning, and the pair of the gazing point map data and the saliency map data substantially has the degree of coincidence. It is data that represents.

  As the image analysis parameters for each learning moving image content, all six weight data are set to the same value, and the weights are set to be equal (for example, wc = wi = wo = wr = wj = wm = 1.0). Alternatively, the weight data is adjusted so that the degree of coincidence between the gazing point map data of the learning moving image content and the saliency map data is higher than a predetermined reference value.

  FIG. 13 shows the saliency map data of the evaluation parameter determination target learning moving image content in the learning image database in which the parameter determination unit 15 uses the gazing point map data and the image analysis parameters related to the learning moving image content. 5 is a flowchart showing a procedure for determining an optimum image analysis parameter (evaluation parameter) for generating the image.

First, in step S <b> 131, the parameter determination unit 15 reads gaze point map data about the evaluation parameter determination target learning moving image content from the data storage unit 11.
Next, in step S <b> 132, the parameter determination unit 15 refers to gazing point map data for a certain learning moving image content excluding the evaluation parameter determination target learning moving image content stored in the data storage unit 11.
Then, in the next step S133, it is determined whether or not there is data referred to in step S132. If there is the gazing point map data of the referenced learning video content (step S133: YES), the process proceeds to the next step S134. On the other hand, when there is no gazing point map data to be referred to, that is, when the processing of steps S134 and S135 has been completed for all of the learning moving image contents in the parameter determination unit 15 (step S133: NO), the process proceeds to step S136. move on.

Next, in step S134, the parameter determination unit 15 reads from the data storage unit 11 the gazing point map data of the learning moving image content referenced in the process of step S133. Next, in step S135, the parameter determination unit 15 calculates the similarity between the gazing point map data of the read evaluation parameter determination target learning moving image content and the gazing point map data of the learning moving image content. The process returns to step S132.
The similarity calculation method in the process of step S135 is the same method as the method of calculating the degree of coincidence by comparing the gazing point map data and the saliency map data in the comparison process executed by the comparison processing unit 14 described above. Is used. For example, the parameter determination unit 15 calculates a difference value for each element of the gazing point map data of the evaluation parameter determination target learning moving image content and the gazing point map data of the learning moving image content. Then, the difference value is compared with a predetermined threshold value, and the number of difference values having a value smaller than the threshold value is measured to obtain the similarity.

Processing after proceeding from step S133 to step S136 is as follows.
In step S136, the parameter determination unit 15 selects the gazing point map data according to a predetermined selection criterion from the gazing point map data of the one or more learning moving image contents whose similarity is calculated. The predetermined selection criterion is, for example, a criterion for selecting gazing point map data whose similarity calculated by the parameter determination unit 15 in step S135 exceeds a predetermined similarity criterion value. Alternatively, the criterion may be that a predetermined number of gazing point map data is selected in descending order of similarity.

  Next, in step S137, the parameter determination unit 15 reads from the data storage unit 11 the image analysis parameter and the degree of coincidence associated with the gazing point map data of the selected learning moving image content. Next, in step S138, when there are a plurality of read matching degrees, the parameter determining unit 15 selects an image analysis parameter corresponding to the highest matching degree among them and evaluates it. Determine the initial value of the parameter. Further, when the read degree of coincidence is one, the parameter determining unit 15 determines the image analysis parameter corresponding to the degree of coincidence as the initial value of the evaluation parameter.

  Next, in step S139, the parameter determination unit 15 searches the six weight data in detail using the evaluation parameter determined above as an initial value to search for the optimal evaluation parameter. For example, the parameter determination unit 15 searches for the optimum value of the weight data in detail using the steepest gradient method. Here, the following example is given. The parameter determination unit 15 selects the first weight data from the six weight data constituting the determined evaluation parameter, changes the value, and supplies the six weight data to the image analysis unit 13. Next, the image analysis unit 13 calculates the saliency map data of the moving image content for evaluation parameter determination target learning using the evaluation parameters of the supplied six weight data, and returns control to the parameter determination unit 15. . Next, the parameter determination unit 15 passes control to the comparison processing unit 14. The comparison processing unit 14 calculates the degree of coincidence between the saliency map data of the evaluation parameter determination target learning moving image content and the gazing point map data of the evaluation parameter determination target learning moving image content, and sends it to the parameter determination unit 15. Return control. The parameter determination unit 15 calculates the degree of coincidence by changing the value of the weight data between a desired range as described above, and searches for weight data having the highest degree of coincidence. Further, the parameter determination unit 15 searches for the weight data having the highest degree of coincidence for the second to sixth weight data as well as the first weight data.

  Next, in step S140, the parameter determination unit 15 determines the searched six weight data as final evaluation parameters that are optimum values of the evaluation parameters. Next, in step S141, the parameter determination unit 15 records the final evaluation parameters in the data storage unit 11.

  In the above parameter determination process, when the number of learning moving image content samples is small, or when the number of image analysis parameter setting patterns for learning moving image content is small, moving image content for evaluation parameter determination target learning It is possible that only a result with a low degree of similarity between the gazing point map data and the gazing point map data of the learning moving image content can be obtained. In such a case, the parameter setting pattern is increased using the feature map data used in the generation process of the saliency map data of the learning moving image content.

  Specifically, based on the comparison between the saliency map data of the learning moving image content and the individual feature map data, physical features having a large influence on the saliency map data of the learning moving image content are selected. To do. Next, the degree of coincidence is calculated while changing the weight data for the selected physical feature within a desired range. Then, the parameter with the highest degree of coincidence calculated is employed as an additional image analysis parameter.

  In addition, the degree of matching of the final evaluation parameter with the image content differs for each image content. Therefore, the evaluation criteria are changed according to the use of the image content (for broadcasting, data distribution, etc.), the purpose of use (for unspecified viewers, for specific viewers, etc.), and the final evaluation parameters are set according to the evaluation criteria. A plurality of types may be provided.

<Evaluation of image content>
There is no gazing point data in image content that has not been subjected to a gaze measurement experiment by a viewer, such as image content being produced. The distribution of the gazing point of the viewer is estimated for such image content without gazing point data. FIG. 14 is a flowchart of processing for estimating gaze point map data of the evaluation target moving image content input from the outside. In step S <b> 241, when the evaluation target moving image content is supplied from the outside to the image input unit 21, the image input unit 21 inputs this and stores it in the data storage unit 11.
Next, in step S242, the similar image content determination unit 22 learns moving image content in which gazing point map data, saliency map data, and optimized evaluation parameters exist for the evaluation target moving image content. From the group, a similar learning moving image content is determined as a similar learning moving image content.

Next, in step S243, the gazing point map estimation unit 23 refers to the evaluation parameter in the determined moving image content for similarity learning, and uses this for evaluation for calculating the saliency map data for the evaluation target moving image content. Use as a parameter to generate saliency map data.
That is, the gazing point map estimation unit 23 sets the similar learning moving image content determined by the similar image content determination unit 22 as the evaluation target moving image content, and based on the evaluation parameters determined by the parameter determination unit 15, the image analysis unit 13 performs video analysis of the moving image content to be evaluated to generate saliency map data.
Then, the gaze point map estimation unit 23 stores the saliency map data in the data storage unit 11.
Next, in step S244, the gazing point map estimation unit 23 reads the saliency map data stored in the data storage unit 11 by the processing in step S243, and outputs this to the outside as estimated gazing point map data. Thereby, it is possible to estimate the distribution of the viewer's gaze point for the evaluation target moving image content.

  The following two examples are given as methods for determining the similarity between the evaluation target moving image content and the learning moving image content.

  The first method uses a common evaluation parameter for generating saliency map data for the evaluation target moving image content and the learning moving image content, or a plurality of feature maps related to one visual attribute, or a plurality of feature maps. The similarity between the saliency map data obtained from the feature maps related to the visual attributes of the images is determined.

  In the second method, clustering processing based on the characteristics of moving image content is performed, and the similarity between moving image contents is determined based on the result. As an example, literature (Keiichiro Hoashi, 3 others, “Proposal of CGM moving image content search method using frame clustering”, IEICE Pattern Recognition / Media Understanding Study Group, pp. 87-92, 2007 10 The process of determining the similarity between moving images by the video analysis processing of moving image content described in “Month)” is applied.

  By displaying the gaze point distribution estimated as described above superimposed on the reproduction display screen of the evaluation target moving image content, it is possible to present an evaluation result that is easy to understand visually for the evaluator.

  As described above, in this embodiment, the gaze distribution measured when the viewer views the learning moving image content and the visual characteristic saliency map obtained by analyzing the learning moving image content are analyzed. The viewer line-of-sight parameter is acquired by using the degree of coincidence, and the database is created. It is possible to estimate the viewer's gaze distribution by analyzing the video image content to be evaluated. In other words, even for image content that has not undergone a line-of-sight measurement experiment, it is possible to easily relate to the distribution of gazing points by using gazing point map data, saliency map data, and saliency map generation parameters of other image contents Evaluation can be made.

  As described above, in this embodiment, when generating a saliency map that represents the distribution of ease of attention when a human views an image, the eye movement obtained by an actual human observation action is generated. The evaluation parameters were adjusted so that the similarity of the distribution with the gaze point distribution obtained based on the above was increased. Thereby, according to the present embodiment, an evaluation result close to a subjective image evaluation by a human can be easily obtained by an objective evaluation method using the physical characteristics of the image.

  Further, in the present embodiment, the attention point map data is generated in consideration of the visual acuity distribution of the peripheral visual field, so that even when the number of subjects is small, the concentration degree of the attention point with respect to all the pixels of the frame image is improved. You can often ask.

  Further, in the present embodiment, in order to determine the optimum final evaluation parameter for the evaluation parameter determination target learning moving image content, the distribution of the gazing point is similar using the existing learning moving image content and The parameter for image analysis that increases the degree of coincidence is determined as the initial value of the parameter for evaluation. Then, using the evaluation parameters, the final evaluation parameters are determined by adjusting the evaluation parameters so that a more appropriate saliency map can be obtained while changing the six weight data. Thereby, according to this embodiment, the optimal final evaluation parameter can be obtained while improving the temporal efficiency.

  Further, in the present embodiment, by evaluating the desired evaluation target moving image content using various learning moving image contents and repeatedly performing an experiment for obtaining a final evaluation parameter, the evaluation target moving image content, The correlation with the setting pattern of the weight data of the final evaluation parameter can be obtained.

  Further, the evaluation target moving image content is evaluated in advance using various learning moving image contents, and an experiment for obtaining a final evaluation parameter is repeatedly performed. It is desirable to obtain the correlation with the setting pattern of the parameter weight data. As a result, according to the image content evaluation apparatus according to the present embodiment, the image content can be obtained only by using the physical characteristics of the image data that is an objective evaluation material, without performing a line-of-sight measurement experiment for obtaining gazing point data. Can be estimated and gaze point map data can be estimated.

  Note that the functions of the image content evaluation apparatus according to the above-described embodiment may be realized by a computer. In this case, a computer program for realizing the control function may be recorded on a computer-readable recording medium, and the computer program recorded on the recording medium may be read by the computer system and executed. . Here, the “computer system” includes an OS (Operating System) and hardware of peripheral devices. The “computer-readable recording medium” refers to a portable recording medium such as a flexible disk, a magneto-optical disk, an optical disk, and a memory card, and a storage device such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case may be included and a program that holds a program for a certain period of time may be included. Further, the above program may be for realizing a part of the functions described above, or may be realized by a combination with the program already recorded in the computer system. .

  As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

  The present invention can be used for evaluation of image content in the production process of content used for broadcasting and network distribution, for example. Similarly, it can be used for evaluation of image content in the production process of content related to video advertisements presented to the public in public facilities.

DESCRIPTION OF SYMBOLS 1 Image content evaluation apparatus 11 Data storage part 12 Gaze point data analysis part 13 Image analysis part 14 Comparison processing part 15 Parameter determination part 21 Image input part 22 Similar image content determination part 23 Gaze point map estimation part

Claims (3)

Video analysis is performed based on image analysis parameters related to visual attributes for each of the plurality of learning moving image contents and evaluation moving image content, and the saliency corresponding to the pixels included in the moving image content An image analysis unit that generates saliency map data indicating a distribution;
Based on the gazing point map data indicating the visual acuity distribution related to the learning moving image content and the saliency map data, the degree of coincidence as an index of similarity between the gazing point map data and the saliency map data is determined. A comparison processing unit to calculate,
The gazing point map data and the saliency map calculated by the comparison processing unit based on the result of video analysis performed on the learning moving image content based on different image analysis parameters in the image analysis unit. select based-out parameters for image analysis coincidence degree between data, to determine the selected image analysis parameters were as initial value of the evaluation parameter, steepest using the initial value of the evaluating parameter A parameter determination unit for determining a final evaluation parameter by a gradient method, and determining the final evaluation parameter as an optimal image analysis parameter ;
A similar image content determination unit for determining similar learning moving image content that is learning moving image content similar to the evaluation target moving image content;
Video analysis of the similar image the image analysis unit is the evaluation target moving image content based on said optimal image analysis parameter the parameter determination unit has determined to the similar learning moving image contents determined by the content determination unit The saliency map data generated by performing as the gazing point map data estimated as the visual acuity distribution of the evaluation target moving image content;
A moving image content evaluation apparatus comprising: Note indicating the visual acuity distribution corresponding to the pixels included in the learning moving image content based on the gazing point data including the coordinate value of the gazing point as a result of measuring the line of sight with respect to the plurality of learning moving image contents. A gazing point data analysis unit for generating viewpoint map data is further provided.
The comparison processing unit, based on the saliency map data regarding each of the plurality of learning moving image contents and the gazing point map data generated by the gazing point data analysis unit, The moving image content evaluation apparatus according to claim 1, wherein a degree of coincidence that is an index of similarity with map data is calculated. Computer
Video analysis is performed based on image analysis parameters related to visual attributes for each of the plurality of learning moving image contents and evaluation moving image content, and the saliency corresponding to the pixels included in the moving image content An image analysis unit that generates saliency map data indicating a distribution;
Based on the gazing point map data indicating the visual acuity distribution related to the learning moving image content and the saliency map data, the degree of coincidence as an index of similarity between the gazing point map data and the saliency map data is determined. A comparison processing unit to calculate,
The gazing point map data and the saliency map calculated by the comparison processing unit based on the result of video analysis performed on the learning moving image content based on different image analysis parameters in the image analysis unit. select based-out parameters for image analysis coincidence degree between data, to determine the selected image analysis parameters were as initial value of the evaluation parameter, steepest using the initial value of the evaluating parameter A parameter determination unit for determining a final evaluation parameter by a gradient method, and determining the final evaluation parameter as an optimal image analysis parameter ;
A similar image content determination unit for determining similar learning moving image content that is learning moving image content similar to the evaluation target moving image content;
Video analysis of the similar image the image analysis unit is the evaluation target moving image content based on said optimal image analysis parameter the parameter determination unit has determined to the similar learning moving image contents determined by the content determination unit The saliency map data generated by performing as the gazing point map data estimated as the visual acuity distribution of the evaluation target moving image content,
Computer program to function as.

Images