WO2011058626A1

WO2011058626A1 - Image processing device and slide show display

Info

Publication number: WO2011058626A1
Application number: PCT/JP2009/069211
Authority: WO
Inventors: 俊輔高山; 晃司山本; 恒青木
Original assignee: 株式会社東芝
Priority date: 2009-11-11
Filing date: 2009-11-11
Publication date: 2011-05-19

Abstract

An image processing device is provided with an acquiring unit (101) for acquiring an image composed of a plurality of pixels, a dividing unit (102) for dividing the image into a plurality of regions on the basis of the difference between the pixel values of a pixel and another pixel adjacent to the former pixel, an extracting unit (103) for extracting the contour of each region, a selecting unit (104) for calculating the polygonality ratio and convexity/concavity ratio of the contour and selecting from among the regions a region having a polygonality ratio equal to or greater than a first threshold value or having a convexity/concavity ratio equal to or smaller than a second threshold value, and a determining unit (106) for determining the whole image including the region selected by the selecting unit as an image of an artifact.

Description

Image processing apparatus and slide show display apparatus

The present invention relates to an image processing apparatus and a slide show display apparatus using the apparatus.

As a technique for detecting an artificial structure such as a building from an image, there is a method using a line segment in the image (see, for example, Patent Document 1). In this method, an artificial structure is detected based on conditions such as whether there are parallel line segments or intersecting line segments.

Japanese Patent No. 3480369

Patent Document 1 is premised on application to an automatic rendezvous of a spacecraft. For example, when an image of a space station and a planet is input, a region where a space station as an artificial structure exists is extracted.

However, in the case of a complex image containing many objects, if an artificial structure is simply extracted by the parallelism or orthogonality between nearby line segments, the line segment relationship with the contour of another nearby object There is a problem that also takes into account. Also, irregular lines and curves are not detected.

The present invention has been made in view of the above, and provides an image processing apparatus and a slide show display apparatus that can determine an artificial structure without being influenced by surrounding objects even in a complex image including many objects. For the purpose.

An image processing apparatus according to the present invention includes an acquisition unit that acquires an image including a plurality of pixels, a division unit that divides the image into a plurality of regions based on a difference in pixel values between the pixel and a pixel adjacent to the pixel, An extraction unit that extracts a contour for each region, and calculates a polygon rate and a concavo-convex rate of the contour, and the polygon rate is equal to or higher than a first threshold value, or a region where the concavo-convex rate is equal to or lower than a second threshold value A selection unit that selects an image from the plurality of regions, and a determination unit that determines an image including the region selected by the selection unit as an image of an artifact.

A slide show display device according to the present invention includes the above-described image processing device, and includes a display unit that displays an image of the artifact in a plurality of images.

According to the image processing device and the slide show display device of the present invention, it is possible to determine an artificial structure without being influenced by surrounding objects even in a complex image including many objects.

1 is a block diagram of an image processing apparatus according to an embodiment. The figure for demonstrating the case where the inclination of an image is corrected when it inclined. The figure which shows an example of the outline extraction result by an extraction part. The figure which shows an example of the area | region division result by a division part. The flowchart which shows an example of operation | movement of an extraction part. The block diagram of a selection part. The figure for demonstrating the calculation method of the average value of the curvature by a 1st selection part. The figure for demonstrating how to give the curvature score which a 1st selection part calculates. The figure for demonstrating the calculation method of the uneven | corrugated rate by a 2nd selection part. The block diagram of the modification of a selection part. The figure which shows an example of the corner point calculation result by a corner point calculation part. The block diagram of a calculation part. 6 is a flowchart illustrating an example of the operation of the image processing apparatus according to the embodiment. The figure which shows an example of the slide show created from the photograph in which the face is reflected. The figure which shows an example of the slide show which inserted the photograph of the artificial structure between the photographs of the person. 1 is a diagram illustrating an example of an apparatus that can include a slide show display apparatus including an image processing apparatus according to an embodiment.

Hereinafter, an image processing apparatus and a slide show display apparatus according to embodiments of the present invention will be described in detail with reference to the drawings. Note that, in the following embodiments, the same numbered portions are assumed to perform the same operation, and repeated description is omitted.
Hereinafter, an object having an artificial and regular structure such as a building, a signboard, or a vehicle is referred to as an artificial structure, and an object including the artificial structure in an image is referred to as an artificial structure image. Here, the building includes a residence, a building, a castle, a temple, a torii, and the like. Signs include signs, monuments, foreheads, etc. Vehicles include trains, ships, airplanes, cars and the like. Polygon refers to all polygons including triangles and quadrangles. In addition, a score indicating the degree to which an artificial structure is included in the image is referred to as a structure score.

Next, the image processing apparatus according to the present embodiment will be described with reference to FIG.
The image processing apparatus according to the present embodiment includes an acquisition unit 101, a division unit 102, an extraction unit 103, a selection unit 104, a calculation unit 105, and a determination unit 106. The acquisition unit 101 acquires one or more image frames 151. The dividing unit 102 divides the image frame 151 acquired by the acquiring unit 101 into a plurality of regions based on the pixel values. Details of the dividing unit 102 will be described later with reference to FIG. The extraction unit 103 extracts the outline of each region divided by the division unit 102. Details of the extraction unit 103 will be described later with reference to FIGS. The selection unit 104 selects only contours having a polygon ratio equal to or higher than the first threshold value or an unevenness rate equal to or lower than the second threshold value from the contours extracted by the extraction unit 103. The polygon ratio is an index indicating how close the shape is to a polygon having a contour, and indicates how close the contour is to a quadrangle, for example. Details of the selection unit 104 will be described later with reference to FIGS. Based on the shape of each contour selected by the selection unit 104, the calculation unit 105 calculates a structural score indicating the likelihood of an artificial structure. Details of the calculation unit 105 will be described later with reference to FIG. The determination unit 106 determines whether the image is an image including many artificial structures from the height of the structural score calculated by the calculation unit 105, and obtains a determination result 152.

Each part will be described in detail below.
First, the acquisition unit 101 will be described.
The acquisition unit 101 acquires an image frame 151 including a plurality of pixels. When there are a plurality of images, they are read into the memory one by one. For example, the memory is installed in the acquisition unit 101. When the input to the acquisition unit 101 is a video, the acquisition unit 101 acquires one image from several frames converted into an image. Next, the acquisition unit 101 converts the input image into a specific resolution specified in advance. This conversion uses an image expansion / contraction technique such as a bicubic method. If the original image is a vertically long image and the converted image is horizontally long, the original image is rotated by 90 ° and then stretched. The acquisition unit 101 outputs the converted image frame to the division unit 102.

The acquisition unit 101 may be configured to determine whether the input image has been photographed with an inclination, and to correct the inclination of the image when the input image has been inclined. Line segments in the image are detected and classified into a plurality of directions (for example, a total of 8 directions inclined by 22.5 ° from the horizontal direction), and the total length of the line segments is calculated for each direction. If the direction with the largest total line segment length is the horizontal direction or the vertical direction, it is not inclined, and if it is any other direction, it is determined that the direction is inclined.

Detecting a line segment is performed by acquiring an edge image by an edge detection method such as the Canny algorithm, and performing a Hough transform on the edge image. If it is tilted, the detected tilt direction is set as the horizontal direction, and the center portion of the image is cut out and rotated as shown in FIG. The acquisition unit 101 may be configured to determine whether the input image has been taken with an inclination, and to correct the inclination of the image if the input image has been inclined. Line segments in the image are detected and classified into a plurality of directions (for example, a total of 8 directions inclined by 22.5 ° from the horizontal direction), and the total length of the line segments is calculated for each direction. If the direction with the largest total line segment length is the horizontal direction or the vertical direction, it is not inclined, and if it is any other direction, it is determined that the direction is inclined.

Detecting a line segment is performed by acquiring an edge image by an edge detection method such as the Canny algorithm, and performing a Hough transform on the edge image. If it is tilted, the detected tilt direction is set as the horizontal direction, and the center portion of the image is cut out and rotated as shown in FIG.

Next, the dividing unit 102 in FIG. 1 will be described with reference to FIGS. 3 and 4.
The dividing unit 102 divides the image area from the difference between the pixel value of the pixel of the converted image frame and the pixel adjacent to the pixel. Here, the pixel value difference includes an RGB value, an HSV value, a Euclidean distance of a luminance value, and the like. H represents hue, S represents saturation, and V represents luminance. Also, a numerical difference binarized using the luminance as a threshold value may be used.

There are various methods for dividing an image into a plurality of regions, such as a k-means method, a region expansion method, and a division / integration method, and any method may be used. In this embodiment, a division / integration method will be described as an example.

First, consider the entire image as one partial area. When it is determined that the target area is not uniform, the target area is divided into two equal parts. The division process is repeated for each partial region until all the regions are uniform. There are various division conditions, but there is a technique using a density histogram as a representative one. The density value (luminance value, RGB value, HSV value, etc.) of each pixel in the region is quantized, and the number of pixels of each density value is counted. If the ratio of the maximum number of pixels among the density values to the entire area is high, it is determined as a uniform area, and if it is low, it is determined as non-uniform and divided.

Next, for the partial areas adjacent to each partial area in the top, bottom, left, and right, if the areas when they are integrated satisfy the conditions, integration is performed. As the integration condition, the same density histogram as the division condition is used. The integration is repeated until there is nothing that can be integrated, and the final region division result is obtained.

The dividing unit 102 assigns a different label to each divided area, and creates an area image (area division result) with the area label to which each pixel belongs as the pixel value of the pixel. This region division result is, for example, as shown on the left in FIG. The dividing unit 102 outputs the area division result to the extracting unit 103. An example of the region division result is shown in FIG. Different areas are displayed in different patterns. Each sky, building roof, or wall in the image is divided into separate areas.

Next, the extraction unit 103 in FIG. 1 will be described with reference to FIG. FIG. 5 is a flowchart illustrating an example of the operation of the extraction unit 103.
The extraction unit 103 extracts the outline of each region in the image divided by the division unit 102. The extraction unit 103 performs the processing from step S502 on all the regions divided by the division unit 102 (step S501). One pixel to which the label of the region currently being processed is assigned is searched from the region image (step S502). The region image is scanned sequentially from the upper left pixel, and the first found pixel is stored as the first contour pixel (step S502).

Subsequently, contour pixels are obtained along the boundary between the area being processed and another area. Of the pixels adjacent to the contour pixel (near 4 or 8) having the region label currently being processed, the pixel adjacent to the pixel having another region label is set as the next contour pixel (step S503). ). If there are a plurality of pixels that satisfy the condition, the next contour pixel is determined based on the relationship between the immediately preceding contour pixel and the current contour pixel. For example, if the immediately preceding contour pixel is at the upper right of the current contour pixel, the neighboring pixels are scanned counterclockwise sequentially from the pixel above the current contour pixel to find a pixel having the region label currently being processed. The pixel found in the next is defined as the next contour pixel. If the current contour pixel is the first contour pixel, there is no previous contour pixel, and therefore, the pixel adjacent to the left of the first contour pixel is regarded as the previous contour pixel and processed.

Whether the next contour pixel has the same coordinates as the first contour pixel is checked. If the next contour pixel is different, the next contour pixel is set as the current contour pixel, and the process returns to step S503. If the same, the process proceeds to step S505 (step S504). The contour of the region is output and the process proceeds to step S501 (step S505). When all the regions have been processed, the operation is terminated, and the extracted contour of each region is used as an input to the selection unit 104. An example of the contour extraction result is shown in FIG. The number of each pixel in FIG. 3 is a label indicating the area to which it belongs.

Next, the selection unit 104 in FIG. 1 will be described with reference to FIG. FIG. 6 is a block diagram illustrating a detailed configuration of the selection unit 104.
The selection unit 104 includes a first selection unit 601 and a second selection unit 602.

The first selection unit 601 calculates a value obtained by inverting the average value of the curvature of each contour, and selects a contour whose value is equal to or greater than a threshold value. As a result, it is possible to select only a contour made of a straight line or a gentle curve, which is peculiar to the shape of the artificial structure.

The second selection unit 602 calculates the concavo-convex ratio of each contour, and selects a contour having a concavo-convex ratio of a threshold value or less. Thereby, a distorted contour can be removed.

Note that the block diagram of FIG. 6 is merely an example, and the

selection units

601 and 602 included in the selection unit 104 are not necessarily all required, but include only a part of the processing, or the order is It may be replaced.

Next, the first selection unit 601 in FIG. 6 will be described. An example of a method for calculating the average value of curvature will be described with reference to FIG.
The first selection unit 601 receives the contour information 651 extracted by the extraction unit 103, and selects only contours having a value obtained by reversing the average value of the curvature from the positive value to the threshold value. A value obtained by inverting the average value of the curvature between positive and negative corresponds to an example of a polygonal curvature. For each pixel in the contour, a curvature score indicating the size of the curvature of the contour is calculated based on the relationship with the adjacent two pixels. The curvature score becomes smaller as the contour is closer to straight. When the target pixel and two adjacent pixels are on a straight line, the curvature score = -1 ((1) in FIG. 7), and when the adjacent two pixels are oblique to the straight line, the curvature score = -0.5 ((( 2)), when the adjacent two pixels are in a right angle relationship, the curvature score = 0 ((3) in FIG. 7), and when the adjacent two pixels are diagonally close to folding, the curvature score = 0.5 ((4 in FIG. 7). )), When the pixel of interest is folded back and the two adjacent pixels are the same, the curvature score is 1 ((5) in FIG. 7). Finally, an average value of curvature scores of all the pixels in the contour is calculated, and a contour whose value obtained by inverting the average curvature score is equal to or greater than a threshold value is selected as the contour of the artificial structure. This threshold is determined experimentally, but here it is 0.7.

This makes it possible to select only contours consisting of straight lines or gentle curves. In the example of FIG. 7, the clean rectangle on the left is the value obtained by inverting the average curvature score = 0.88, and the graphic including the gentle curve in the middle is the value obtained by inverting the score is 0.73, Since an irregular figure has a value obtained by reversing the score from positive to negative = 0.48, only the left and middle figures can be selected by threshold processing.

The method for assigning the score will be described in detail. As shown in FIG. 8, the eight adjacent pixels of the target pixel 801 are (1) to (8) in order from the upper left.
The curvature score of the pixel of interest 801 is set to −1 because the groups of two pixels before and after the pixel of interest 801 are (1) (8), (2) (7), (3) (6), (4) (5) ) Is one of the cases.
The curvature score is set to −0.5 because the set of two pixels before and after the target pixel 801 is (1) (7), (1) (5), (2) (6), (2) (8), (3) (4), (3) (7), (4) (8), (5) (6).
The curvature score is set to 0 because the group of two pixels before and after the pixel of interest 801 is (1) (6), (1) (3), (2) (4), (2) (5), (3) (8), (4) (7), (5) (7), (6), or (8).
The curvature score is set to 0.5 because the groups of two pixels before and after the pixel of interest 801 are (1) (2), (2) (3), (3) (5), (5) (8), ( 8) Any of (7), (7), (6), (6), (4), (4), and (1).
The curvature score is set to 1 because the group of two pixels before and after the pixel of interest 801 is (1) (1), (2) (2), (3) (3), (4) (4), (5) (5), (6), (6), (7), (7), (8), or (8).

The above example is an example of a contour curvature score calculation method, and another method may be used. For example, there is a method of calculating a radius of curvature and a curvature using a quadratic differentiation of each point of the contour by approximating the shape of the contour as a function.

In addition to this, as an example of the polygon ratio, there is “area included in outline / area of rectangle circumscribing outline”.

The selection unit 104 may remove short contours that are equal to or less than a threshold before performing the processing of the first selection unit 601. The threshold value of the contour length to be deleted is experimentally determined according to the resolution of the image. Here, the contour length = the number of pixels belonging to the contour.

The first selection unit 601 may calculate a value obtained by inverting the average curvature score from the portion excluding the pixels at the screen edge in the contour. Since the image is generally rectangular, the outline of the region including the screen edge is linear. The shape of the contour of the true object can be evaluated by removing the pixels at the screen edge. As another method, the selection unit 104 may not select an outline including pixels at a certain ratio or more at the edge of the screen.

Next, the second selection unit 602 in FIG. 6 will be described. An example of a method for calculating the unevenness ratio indicating the number of unevenness of the contour will be described with reference to FIG.
The second selection unit 602 selects only the contour with less unevenness among the contours in the image selected by the first selection unit 601. The smallest rectangle circumscribing each contour is obtained (dotted line in FIG. 9), and “the length of the contour / the length of the contour of the circumscribed rectangle” is calculated as the unevenness ratio. An example of a method for calculating a circumscribed rectangle will be shown. Among the contour pixels, a pixel having the smallest x coordinate in the image is obtained, and the x coordinate is set to x1. Similarly, the maximum x coordinate is x2, the minimum y coordinate is y1, the maximum y coordinate is y2, and the four vertices (x1, y1), (x1, y2), (x2, y1), (x2, y2) ) Is a circumscribed rectangle. In this calculation method, the unevenness ratio increases as the unevenness increases.

The length of the contour and the length of the contour of the circumscribed rectangle are obtained from the number of pixels belonging to the contour. If the relationship between adjacent pixels is diagonal (near 8), the length is counted as 2, and if not, the length is counted as 1. In FIG. 9, the unevenness ratio = 1 for the left figure, and the unevenness ratio> 1 for the middle and right figures because the contour is distorted. A contour having an unevenness ratio equal to or lower than a threshold is selected as the contour of the artificial structure. This threshold is obtained experimentally, but here it is 1 / 0.75. If the ratio is 1 / 0.75 or less, the natural object rectangle is removed, and the contour of the artificial structure remains.

The above-described example is an example of a method for calculating the contour unevenness rate, and another method may be used. For example, the unevenness ratio = l ² / S may be obtained using the peripheral length l of the contour and the area S of the region included in the contour. Also in this case, the unevenness ratio increases as the unevenness increases.

(Modification of the selection unit 104)
The selection unit 104 may have the following configuration. A modification of the selection unit 104 will be described with reference to FIGS. A modification of the selection unit 104 is assumed to be 104b. FIG. 10 is a block diagram illustrating a detailed configuration of the selection unit 104b.

The selection unit 104b includes a direction classification unit 1001, a corner point calculation unit 1002, an inter-corner curvature calculation unit 1003, and an inter-corner deletion unit 1004.

The selection unit 104b calculates the direction of a line segment composed of contour pixels, and calculates a point where the direction changes as a corner point. Subsequently, a value obtained by reversing the average value of curvature between corner points adjacent to each other along the direction of the contour line segment is calculated. The part where this value is less than or equal to the threshold value is deleted from the contour, and the remaining part (the part where the value obtained by inverting the curvature is greater than the threshold value) is used as the input to the calculation unit 105.

The direction classification unit 1001 classifies the directions between adjacent pixels in the outline into four directions:-(horizontal direction), | (vertical direction), / (right diagonal direction), and \ (left diagonal direction). Subsequently, the number of pixels in each direction for several pixels before and after the contour is obtained for each pixel, and the direction with the largest number of pixels is defined as the average direction of the pixels.

The corner point calculation unit 1002 sequentially scans adjacent pixels on the contour as shown in FIG. 11, and calculates a point where the average direction changes as a corner point. However, a large number of corner points are generated in the portion where the average direction changes frequently. Therefore, if the ratio occupied by the maximum direction of several pixels before and after the change in the average direction is low (the ratio is equal to or less than the threshold), the corner point is not set.

The inter-corner curvature calculation unit 1003 calculates the average curvature of the contour between corners for each pair of corner points adjacent along the direction of the contour. The average curvature score described above with reference to FIG. 7 is used as the average curvature.

The inter-corner deletion unit 1004 deletes a portion in the contour where a value obtained by reversing the average value of the curvature between corners is less than or equal to a threshold value. This threshold is determined experimentally, but here it is 0.7. As a result, the distorted portion of the outline or the bent portion is deleted, and only the straight or gentle curved portion remains. The contour after the processing is partly interrupted, and this contour is input to the calculation unit 105.

Next, the calculation unit 105 of FIG. 1 will be described with reference to FIG. FIG. 12 is a block diagram illustrating a detailed configuration of the calculation unit 105.
The calculation unit 105 includes a direction classification unit 1201, a perpendicularity score calculation unit 1202, and a structural score calculation unit 1203.

The direction classification unit 1201 classifies the direction between adjacent pixels of the contour, and counts the number of pixels in each direction for each contour. The vertical score calculation unit 1202 calculates the ratio of the number of pixels in the directions perpendicular to each other in each direction of the contour as the vertical score. The structural score calculation unit 1203 calculates the contour score of each contour from the contour length and the perpendicularity score, and outputs the sum of the contour scores of all the contours as the structural score of the entire image.

Since the contour score is calculated for each contour, it is possible to appropriately calculate the structure without being influenced by surrounding objects even in a complex image including many objects.

Next, the operation of the direction classification unit 1201 in FIG. 12 will be described. The direction classification unit 1201 classifies the directions between adjacent pixels in the outline into four directions-, |, /, and \, and counts the number of pixels in each direction. The number of pixels in each direction may be obtained by obtaining the number of pixels in each direction in several pixels before and after the contour for each pixel and using the direction with the largest number of pixels as the average direction of the pixels. .

Next, the operation of the perpendicularity score calculation unit 1202 in FIG. 12 will be described.
First, the direction with the maximum number of pixels is obtained from the four directions −, |, /, and \, and the number of pixels is _defined as N _max . Next, the number of pixels obtains the maximum in a direction perpendicular to the direction N _v, and outputs the N v _/ N _max as a vertical score for that contour. When the direction with the maximum number of pixels is |, the direction perpendicular to it is-, and when the direction with the maximum number of pixels is /, the direction perpendicular to it is \.

Since an artificial structure such as a building contains many square parts, there are many line segments in two directions perpendicular to each other. Therefore, by introducing a perpendicularity score, a score reflecting the characteristics of such an artificial structure can be calculated.

When the screen aspect ratio (aspect ratio when displaying on the screen) and the pixel aspect ratio (aspect ratio of the pixels that make up the image) are different, the ratio of the number of pixels and the actual aspect ratio are different, so each direction The verticality score is calculated after correcting the number of pixels. If the direction of the maximum number of pixels among the four directions is − or |, the screen aspect ratio is m: n, and the pixel aspect ratio is p: q, the number of pixels in the − direction is multiplied by m × q / n × p. The vertical score is calculated from When the direction of the maximum number of pixels among the four directions is / or \, the original number of pixels is used as it is for the structural score calculation. This is because the ratio of the length in the \ direction does not change even if the aspect ratio changes.

Next, the operation of the structural score calculation unit 1203 in FIG. 12 will be described.
First, the structural score calculation unit 1203 calculates a contour score indicating the likelihood of an artificial structure of the shape for each contour. Contour score = contour length × verticality score. The contour score is calculated for all the contours selected by the selection unit 104, and the sum of the contour scores is output as the structural score of the image.
When calculating the structural score, the number of pixels excluding the pixels at the edge of the screen in the contour may be used as the contour length of the contour.

Next, the operation of the determination unit 106 in FIG. 1 will be described.
The determination unit 106 determines that an image in which the structural score output by the calculation unit 105 is equal to or greater than a threshold is an artificial structure image. Since the optimum threshold varies depending on the size of the image, it is obtained experimentally.

Note that the configuration of the image processing apparatus may be a form that excludes the calculation unit 105 of FIG. In this case, the determination unit 106 determines that an image in which one or more contours are selected by the selection unit 104 is an artificial structure image.

Finally, the operation flow of the image processing apparatus of FIG. 1 will be described with reference to FIG.
The image processing apparatus performs processing from step S1302 onward until processing of all images is completed (step S1301). The dividing unit 102 divides the image into a plurality of areas, and assigns and outputs a separate label for each area (step S1302). Subsequently, processing from step S1304 is performed until processing of all divided areas is completed (step S1303). The extraction unit 103 extracts the contour of the region by obtaining the boundary point between the region and the region outside the region (step S1304). Next, the selection unit 104 determines whether or not the average curvature score and the unevenness ratio of the extracted contour satisfy a preset threshold value (step S1305). If the condition is satisfied, the process proceeds to step S1306. If not, the contour processing ends and the process proceeds to step S1303. The calculating unit 105 calculates a contour score from the contour length and the perpendicularity score (step S1306). The contour score calculated by the calculation unit 105 is added to the structural score of the entire image, the contour processing is terminated, and the process proceeds to step S1303 (step S1307).

When all areas are processed in step S1303, the process proceeds to step S1308. If the determination unit 106 determines that the structural score of the image is equal to or greater than a preset threshold value, the determination unit 106 determines that the image is an artificial structure image, and if not, the process proceeds to step S1301 (step S1308). When all the images have been processed, the process ends.

According to the image processing apparatus of the present embodiment, the image is divided into regions, and the score is calculated for each outline of the divided regions. Therefore, even in a complex image including many objects, the influence of surrounding objects is affected. Therefore, it is possible to provide an image processing apparatus that can determine a structural object without receiving the image. In addition, since the structural score is calculated for each region in the image, it can be used to grasp where and in what size the building is shown in the image.

The artificial structure image thus obtained can be used for automatic creation of a slide show. By automatically selecting dozens of images from a large number of photos and videos taken by individuals and playing a slideshow, you can appreciate memories of travel and events in a short time. FIG. 14 and FIG. 15 show an example of the image group selected for the slide show. In the slide show, five photos are displayed for several seconds in order from the left.

As a conventional method of image selection used for a slide show, there is a method of selecting a photo with a face up or a photo with many faces as shown in FIG. In this method, photographs with the same person appear often in succession, and the entire slide show has a monotonous impression.
On the other hand, FIG. 15 shows an artificial structure photograph inserted between human photographs by a slide show display apparatus using the image processing apparatus of the present embodiment. The photographs of the artificial structure are the second (house) and the fourth (signboard) from the left. As a result, the slide show can be sharpened and the photograph taken can be grasped. The images to be inserted into the slide show are appropriately selected from images determined as artificial structures. The slide show display device includes a display unit that displays an image of an artifact in a plurality of other images.

When using for a slide show, the configuration of the image processing apparatus may not include the determination unit 106 in the processing unit of FIG. When the determination unit 106 is not present, the top several images having a high structural score output from the calculation unit 105 are selected as images to be inserted into the slide show.

As other uses, it can be used to classify images taken and stored into categories of artificial structures and other categories, and to identify urban and mountainous areas using aerial photographs.

Also, the slide show display device including the image processing device of the present embodiment can be applied to a personal computer, a digital photo frame, a camera, a television, a mobile phone and the like as shown in FIG.

Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

The image processing apparatus and slide show display apparatus according to this embodiment can be applied to a personal computer, a digital photo frame, a camera, a television, a mobile phone, and the like.

DESCRIPTION OF SYMBOLS 101 ... Acquisition part, 102 ... Dividing part, 103 ... Extraction part, 104, 104b ... Selection part, 105 ... Calculation part, 106 ... Determination part, 151 ... Image frame, 152 ... Determination result, 601 ... First selection part, 602 ... second selection unit, 651, 652 ... contour information, 801 ... target pixel, 1001 ... direction classification unit, 1002 ... corner point calculation unit, 1003 ... curvature calculation unit between corners, 1004 ... deletion unit between corners, 1201 ... direction classification , 1202... Vertical score calculator, 1203.

Claims

An acquisition unit for acquiring an image composed of a plurality of pixels;
A dividing unit that divides the image into a plurality of regions from a difference in pixel value between the pixel and a pixel adjacent to the pixel;
An extraction unit for extracting a contour for each region;
Calculating a polygonal rate and a concavo-convex rate of the contour, and selecting a region from which the polygonal rate is equal to or higher than a first threshold value, or the concavo-convex rate is equal to or lower than a second threshold value,
An image processing apparatus comprising: a determination unit that determines an entire image including an area selected by the selection unit as an image of an artifact.
The selection unit further includes a calculation unit that calculates a structural score from the polygon ratio and the unevenness ratio for each contour,
The image processing apparatus according to claim 1, wherein the determination unit determines that an image having a high structural score of the entire image is an image of an artifact.
3. The image processing apparatus according to claim 2, wherein the selection unit selects a region corresponding to a contour in which a value obtained by reversing the average of the curvature of the contour is greater than or equal to a threshold value.
The image processing according to claim 2, wherein the selection unit selects a region corresponding to a contour in which a ratio of a length of the contour to a length of a minimum quadrangle including the region is equal to or less than a threshold value. apparatus.
The selection unit calculates a direction of a line segment composed of pixels of the contour, a corner point calculation unit that calculates a point where the direction changes as a corner point;
A curvature calculating unit that calculates an average value of curvature between corner points adjacent to each other along the direction of the line segment of the contour;
The image processing apparatus according to claim 2, wherein only a portion where the value obtained by reversing the curvature of the curvature is greater than a threshold is used for calculation of the structural score.
A slide show display device comprising: the image processing device according to claim 1, further comprising a display unit that displays the artifact image in a plurality of images different from the image.