JP2007041730A - Electric wire abnormality detection method, device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly perform processing by reducing the labor of an operator in the case of checking electric wire abnormality. <P>SOLUTION: In the segmentation processing of an electric wire section from an original image 1 in a method for detecting the abnormality of an electric wire in image processing from an original image 1 obtained by photographing an electric wire, the inclinations of electric wires are compared when determining the location of an electric wire whose segmentation processing should be executed, and when the scales of those inclinations are different by more than a fixed value, it is judged that the detection of the electric wire has failed, and a retrieval region 3 of the electric wire is extended to retrieve the electric wire again, and the template pattern of the electric wire to be referenced is changed, and the retrieval of the electric wire is operated again Thus, the tracing capability of the electric wire can be improved, and that the quick abnormality detection of the electric wire can be achieved. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing method, apparatus, and program. More specifically, the present invention relates to a method, apparatus, and program for detecting an abnormality in the shape and color of an electric wire by image processing.

  In order to maintain and manage the transmission line, a patrol inspection of the transmission line by a helicopter is conventionally performed. One of the purposes of this inspection is the early detection of abnormal parts of electric wires damaged by lightning strikes. Early detection of abnormalities in wires is essential for maintaining the reliability of power supply. The inspection is performed by photographing the electric wire with a video camera mounted on the helicopter, playing the video image, and visually observing the video image by the operator. The operator confirms whether or not there is a broken wire that has a part of the electric wire cut or an arc mark that may cause a broken wire in the future. In order to reduce oversight of the abnormal part of the electric wire, a check by slow reproduction of the video image or a double check by two workers may be performed.

  A technique has been proposed in which an image of a wire portion is cut out from an image taken of the wire and an abnormality of the wire is detected in the image cut out by image processing (see Patent Document 1). By automatically detecting an abnormality in the shape of the electric wire, the labor of the operator is reduced and the processing speed is increased.

JP 2005-57956 A

  However, the technique of patent document 1 detects an electric wire by performing image processing of the center part of a picked-up image. Therefore, when the electric wire is only photographed at the edge portion of the screen, that is, when the electric wire is not photographed at the center portion of the image, tracking cannot be performed, and there is a problem that the system temporarily stops. . In addition, there is a problem that the electric wire cannot be detected accurately due to the influence of the background of the photographed image. In addition, when tracking becomes impossible, there is no function to determine whether tracking is possible, and when tracking becomes impossible, the operator has to re-set the wire position again, which is complicated. There was a problem.

  In addition, when an image other than the electric wire to be inspected, such as a steel tower or an overhead wire fitting, appears, the electric wire tracking fails, and in each case, there is a problem that correction by operator intervention is necessary.

  As long as the electric wire is photographed from above with a helicopter, the electric wire may not be captured at the center of the image due to the flight status of the helicopter, wind, and the like. Thus, the need for the operator's intervention every time the tracking of the electric wire fails does not contribute to a rapid abnormality detection process. In other words, for rapid abnormality detection processing, it is essential to improve the tracking capability of the wire. If the tracking capability can be improved, the number of times the operator performs setting work can be reduced, and the labor of the operator is reduced. Can be reduced, and quick processing can be performed.

  Needless to say, the shorter the time from the shooting by the helicopter to the completion of the wire inspection, the better. This is because if it is short, a serious problem that needs to be confirmed locally can be dealt with early and the risk can be reduced.

  Therefore, the present invention enables a significant improvement in tracking ability compared to the invention of Patent Document 1, thereby reducing labor and reducing labor and enabling high-speed abnormality detection processing. An object of the present invention is to provide an electric wire abnormality detection method, apparatus, and program.

  In order to achieve such an object, the wire abnormality detection method according to claim 1 includes: a target image cutout process for cutting out a portion where an electric wire is photographed from the original image with respect to an original image obtained by photographing the wire; Including an edge detection process, an ideal outline estimation process, a shape abnormality detection process, or a color abnormality detection process for the target image cut out by the image cut-out process, and the target image In the cut-out process, a search area is set in advance for the original image, an image of the electric wire to be searched is stored in a storage device in advance as a template, and an area most similar to the template is searched for in the search area. And the inclination of the electric wire photographed in the immediately preceding frame image is compared with the search result, and the difference in inclination is not less than a preset threshold value. It is determined that the search fails in the wire when the said expanded search area, so that to estimate the moving position of the wire by searching a region most similar again the template in said search area.

  The wire abnormality detection device according to claim 6 is a target image cut-out processing unit that cuts out a portion of the original image from which the electric wire is photographed with respect to the original image in which the wire is photographed, and the target image cut-out process. A wire abnormality detection processing unit that performs edge detection processing, ideal contour estimation processing, shape abnormality detection processing, or color abnormality detection processing on the target image cut out by the above, and the target image clipping processing unit Sets a search area for the original image in advance, stores an image of the electric wire to be searched as a template in a storage device in advance, searches for an area most similar to the template in the search area, and moves the electric wire Estimate the position, compare the inclination of the electric wire captured in the previous frame image with the search result, and if the difference in inclination is greater than or equal to a preset threshold It is determined that the search fails, the expanded search area, in which to search an area most similar to again the template in the search region for estimating the movement position of the wire.

  Further, the electric wire abnormality detection program according to claim 11 sets a search area in advance for the original image in which the electric wire is photographed, and stores in advance a storage device as an image of the electric wire to be searched as a template, An area most similar to the template is searched in the search area to estimate the moving position of the electric wire, and the inclination between the electric wire photographed in the immediately preceding frame image and the search result is compared. If it is equal to or greater than the set threshold, it is determined that the search for the electric wire has failed, the search area is expanded, and the area most similar to the template is searched again in the search area to estimate the moving position of the electric wire. Then, a target image cutout unit that cuts out a portion of the original image where the electric wire is photographed, and an edge detection is performed on the target image cut out by the target image cutout process. And processing, the ideal contour estimation processing, as well, and so as to be executed as a wire abnormality detection processing means for performing a shape abnormality detection processing or color abnormality detection processing.

  Therefore, when searching for an electric wire in the process of cutting out an electric wire portion from a target image in which the electric wire is photographed, an electric wire search area is set in advance, and the electric wire image to be searched is stored in advance in the main storage area as a template. Then, an area similar to the stored template is searched in the search area to estimate the moving position of the electric wire. In addition, the wire inclination in the previous frame image is compared with the wire inclination estimated in the frame image being processed. By re-expanding the electric wire search area and searching for the electric wire portion again, the tracking ability of the electric wire is improved.

  The wire abnormality detection method according to claim 2 is cut out by a target image cut-out process for cutting out a portion of the original image where the electric wire is shot from the original image where the wire is shot, and a cut-out process of the target image. The target image includes edge detection processing, ideal contour estimation processing, shape abnormality detection processing or color abnormality detection processing, and the target image cut-out processing includes the original image The search area is set in advance, the image of the electric wire to be searched is stored in the storage device in advance as a template, the area most similar to the template is searched in the search area to estimate the moving position of the electric wire, Furthermore, compare the slope between the wire taken in the previous frame image and the search result, and if the difference in slope is greater than or equal to a preset threshold, the wire search fails It is determined that, with reference to the template of another wire previously stored, so that searching for a region that is most similar again the template by the search region.

  The electric wire abnormality detection device according to claim 7 is a target image cut-out processing unit that cuts out a portion of the original image where the electric wire is photographed from the original image obtained by photographing the electric wire, and the target image cut-out process. A wire abnormality detection processing unit that performs edge detection processing, ideal contour estimation processing, shape abnormality detection processing, or color abnormality detection processing on the target image cut out by the above, and the target image clipping processing unit Sets a search area for the original image in advance, stores an image of the electric wire to be searched as a template in a storage device in advance, searches for an area most similar to the template in the search area, and moves the electric wire Estimate the position, compare the inclination of the electric wire captured in the previous frame image with the search result, and if the difference in inclination is greater than or equal to a preset threshold It is determined that the search fails, by referring to the template of another wire previously stored, it is to explore an area most similar to again the template in the search region.

  Further, the electric wire abnormality detection program according to claim 12 sets a search area in advance for an original image in which an electric wire is photographed, and stores in advance a storage device as an image of an electric wire to be searched, An area most similar to the template is searched in the search area to estimate the moving position of the electric wire, and the inclination between the electric wire photographed in the immediately preceding frame image and the search result is compared. If it is equal to or greater than the set threshold value, it is determined that the wire search has failed, the other wire template stored in advance is referenced, and the region most similar to the template is searched again in the search region. Estimating the moving position of the electric wire and extracting the target image from the original image, and the pair extracted by the target image cutting process An edge detection process on the image, the ideal contour estimation processing, as well, and so as to run as a wire abnormality detection processing means for performing a shape abnormality detection processing or color abnormality detection processing.

  Therefore, when searching for an electric wire in the process of cutting out an electric wire part from the target image obtained by photographing the electric wire in the electric wire abnormality detection method, an electric wire search region is set in advance, and an image of the electric wire that is the search target is stored in advance as a template. An area similar to the stored template is searched in the search area, and the moving position of the electric wire is estimated. Furthermore, the wire inclination in the wire template pre-stored in the storage device is compared with the wire inclination estimated from the frame image being processed. With reference to another wire template stored in advance as a failure in detection, an area most similar to the template is searched again in the search area. That is, even if the tracking of the electric wire fails, the tracking ability of the electric wire is improved by changing the template to be referenced and searching for the electric wire again.

  The invention according to claim 3 is the electric wire abnormality detection method according to claim 1 or 2, wherein the template is stored in a main storage device when the target image is cut out, and every time a certain frame passes, The template is automatically updated. The invention according to claim 8 is the electric wire abnormality detection device according to claim 6 or 7, wherein the template is stored in the main storage device when the target image is cut out, and every predetermined frame has elapsed. In addition, the template is automatically updated.

  Therefore, since the wire template is automatically updated every certain number of frames, it is not necessary for the operator to perform the work of adding the wire template. There is no need to perform any operation.

  The invention according to claim 4 is the electric wire abnormality detection method according to claim 1 or 2, wherein the setting of the search area is set in advance for an image of the first frame in the original image in which the electric wire is photographed, From the processing of the image of the next frame, the midpoint of the electric wire search result in the immediately preceding frame is set as the center position of the search area. The invention according to claim 9 is the wire abnormality detection device according to claim 6 or 7, wherein the search area is set in advance for the image of the first frame in the original image in which the wire is photographed. Then, from the processing of the image of the next frame, the midpoint of the electric wire search result in the immediately preceding frame is set as the center position of the search area.

  Therefore, by setting the midpoint of the electric wire in the previous frame as the center of the electric wire search area in the next frame, the electric wire can be searched even when the electric wire position in the captured image is shifted. I am doing so. Even when the shooting position of the electric wire in the image gradually shifts from frame to frame, the electric wire can be searched.

  According to a fifth aspect of the present invention, in the electric wire abnormality detection method according to the first or second aspect, an image irrelevant to the detection of the electric wire is removed from the original image obtained by photographing the electric wire by designating a time code in advance. Then, after removing an image irrelevant to the detection of the electric wire, the target image is cut out. According to a tenth aspect of the present invention, in the electric wire abnormality detection device according to the sixth or seventh aspect, by specifying a time code in advance for an image irrelevant to the detection of the electric wire among the original images taken of the electric wires. The target image is cut out after removing the image unrelated to the detection of the electric wire.

  Therefore, images that are irrelevant to the tracking of the electric wire are excluded by specifying the time code and removing the images that are irrelevant to the detection of the electric wire such as the steel tower portion among the images in which the electric wires are previously captured. Can be removed in advance.

  As described above, according to the wire abnormality detection method according to claim 1, the wire abnormality detection device according to claim 6, and the wire abnormality detection program according to claim 11, there is a sudden movement of the wire in the photographed image, Even when the position of the electric wire in the captured image has moved significantly, the electric wire can be detected accurately. That is, by eliminating the failure in tracking the electric wire and improving the tracking capability of the electric wire, the operator's intervention can be eliminated and the labor of the operator can be reduced.

  According to the wire abnormality detection method according to claim 2, the wire abnormality detection device according to claim 7, and the wire abnormality detection program according to claim 12, the conventional method is affected by the effects of shooting conditions such as sunshine conditions and background images. Even in a shooting environment where it is difficult to search for electric wires, it is possible to accurately detect the wires regardless of the shooting environment by separately storing templates of other electric wires in advance and comparing them. By eliminating the tracking failure of the electric wire and improving the tracking capability of the electric wire, the operator's intervention can be eliminated and the labor of the operator can be reduced.

  According to the wire abnormality detection method according to claim 3 and the wire abnormality detection device according to claim 8, if the initial setting is performed, it is not necessary to perform the operation thereafter, so that the operator's intervention is eliminated and the labor of the operator is reduced. It is possible to reduce the electric wire abnormality detection process.

  According to the wire abnormality detection method according to claim 4 and the wire abnormality detection device according to claim 9, even if the shooting position of the wire in the image is gradually shifted for each frame, the wire is easily Can be searched, and the amount of data processing for searching can be reduced, so that a rapid electric wire abnormality detection process can be performed.

  According to the wire abnormality detection method according to claim 5 and the wire abnormality detection device according to claim 10, since an image irrelevant to tracking of the wire can be removed in advance, the amount of data to be processed can be reduced, A rapid electric wire abnormality detection process can be performed.

  Hereinafter, the configuration of the present invention will be described in detail based on embodiments shown in the drawings.

  1 to 43 show an embodiment of an electric wire abnormality detection method and apparatus and program according to the present invention. For example, as shown in FIG. 11, the wire abnormality detection method includes pre-processing (S1) for obtaining an original image in which a wire is photographed, processing (S2) for cutting out a region of a wire portion from the original image, and a cut-out target image. In contrast, the process includes a process of detecting an abnormality of the electric wire (S3), and a process of outputting an image in which the target image has not been successfully cut out or an abnormality of the electric wire is detected (S4).

  The preprocessing (S1) is a process of reading captured image data obtained by capturing an electric wire from a video camera or a medium on which video is recorded, and recording the image data in an external storage device such as a hard disk. The photographed image data may be recorded in a storage device provided separately from the present apparatus and acquired through the Internet network or the like.

  For example, in the present embodiment, an aerial image of a wire taken by a video camera mounted on a helicopter is used as an image of the wire that is the inspection target. For example, the video camera generates an image of 30 frames per second. Each frame image obtained from the video camera is converted into an RGB color model that can be processed by a computer in the pre-processing, and further converted into an 8-bit grayscale image in order to simplify and speed up the processing. The Each frame image converted as described above is used as an original image in this embodiment. Each pixel constituting the original image has a brightness value (luminance value) of 256 gradations from 0 (black) to 255 (white) as a color information value. The resolution of the original image is, for example, 640 pixels in the horizontal direction and 480 pixels in the vertical direction. However, the original image is not limited to a gray scale image and may be a color image, and the resolution is not limited to the above example.

  Hereinafter, in the present embodiment, the screen horizontal direction is the electric wire longitudinal direction, and the screen vertical direction is the electric wire transverse direction. When the electric wire is photographed in the vertical direction of the screen, the vertical direction of the screen may be the longitudinal direction of the electric wire, and the horizontal direction of the screen may be the horizontal direction of the electric wire. The photographed image does not necessarily have to be photographed from the left end to the right end of the image or from the top end to the bottom end, and may include an image in which the wire is photographed only at the end of the image. An image of a portion other than the electric wire, for example, a steel tower portion may be included.

  By processing the image data obtained by photographing the electric wire recorded in the external storage device or the like by the preprocessing (S1) by the subsequent target image cutout processing (S2) and electric wire abnormality detection processing (S3), The detection is performed. In this embodiment, the target image cut-out process (S2) and the wire abnormality detection process (S3) are performed for each frame of data of the photographed image while reading the photographed image data obtained by photographing the wire. The wire abnormality is detected more.

  In the present embodiment, when photographing an electric wire, in addition to recording an image of the electric wire, time information (hereinafter referred to as a time code) is recorded in association with the image. For example, in this embodiment, when recording an image, a time code is recorded for each frame at the same time, and how many minutes, how many seconds (of 30 frames per second) the image is. Is going to be recorded. The time code recording method is not limited to the above method. For example, a serial number may be assigned for each frame, or the actual photographing time may be recorded.

  An example of the photographed image is shown in FIGS. For example, as shown in FIGS. 35A and 35B, the actual captured image includes a portion other than the electric wire. FIG. 35B is an image of a steel tower portion. Even if the target image cut-out process (S2) is performed on such an image, the electric wire cannot be detected and may be detected erroneously. Further, since wasteful data processing is performed, it does not contribute to quick processing. It is desirable to exclude such images before data processing as much as possible.

  By specifying the time code when reading the captured image of the wire in the preprocessing (S1), an image that is not related to the detection of the abnormality of the wire can be removed. Specifically, a start frame and an end frame of an image irrelevant to the detection of an abnormality in the recorded time code are designated, and the time code is input, so that the image between them is cut out from the target image (S2). It is possible to prevent processing. As a result, images that are not related to the detection of the abnormality of the electric wires can be removed, and the processing can be speeded up. Note that the method of specifying the time code is not limited to the above method. For example, the start code and the end frame of the time code at which the wire portion is photographed are specified and the portion is processed. good.

  In the present embodiment, for example, at the time of video shooting, the time code of the start image and the end image for each span of the wire is stored in the storage device in advance, so that only the frame image between them is to be monitored. Can do. Thereby, an image irrelevant to the abnormality detection of the electric wire, for example, a photographed image of the steel tower portion can be excluded from the inspection target, and the process on the frame where the image irrelevant to the abnormality detection of the electric wire is taken can be omitted. In addition, when the target image cut-out process (S2) is performed on a captured image in which an electric wire such as a steel tower portion is not shot, the electric wire portion cannot be cut out, and the process is stopped. Thus, it is possible to reduce in advance the effort of performing template addition processing. For this reason, the operator's intervention can be reduced and a rapid electric wire abnormality detection process can be performed.

  Next, a target image cut-out process (S2) is performed. This is a process of cutting out the portion where the electric wire was photographed from the original image. Thereby, the data processing amount at the time of performing the process (S3) which detects abnormality of an electric wire can be reduced significantly. That is, since the portion where the elongated electric wire is photographed is a very small part of the original image, even if the edge detection process (S306) is performed on the entire original image, it does not contribute much to the edge detection accuracy. Processing time will be required. For this reason, first, as in the present embodiment, a process (S2) of cutting out the portion where the electric wire was photographed from the original image is performed, the cut out region is stored as a target image in the storage device, and the target image is stored. By performing the edge detection process (S306), the processing time is greatly reduced.

  In the target image cut-out process (S2) of the present embodiment, as shown in FIG. 12, for example, three search areas in which the horizontal range and the vertical range are predetermined are arranged in the horizontal direction, which is the wire longitudinal direction of the original image. In addition to the above distribution, an image of the electric wire to be searched is given in advance as a template (S202), an area most similar to the template is searched in each search area (S206), and a search result for each search area is predetermined. It is determined whether or not regularity is satisfied, and an image including at least a search result satisfying the regularity is cut out from the original image to be a target image (S209).

  The search area in the present embodiment is, for example, vertically long as shown in FIG. 28, the number of pixels X in the horizontal direction range is the same as the number of horizontal pixels in the template, and the number of pixels Y in the vertical range is the vertical direction pixels of the template. The number of pixels is larger than the number and equal to or less than the number of pixels in the vertical direction of the original image. Here, reference numeral 1 in FIGS. 28 and 29 indicates an original image, reference numeral 2 indicates a template, reference numeral 3 indicates a search area, and reference numeral 4 indicates an image of an electric wire. However, the size of the search area is not limited to the above example. In addition, for example, as shown in FIG. 29, the search areas in the present embodiment are distributed at three places with a predetermined distance in the horizontal direction that is the longitudinal direction of the electric wire. However, the number of search areas may be three or more.

  In this embodiment, an original image is captured at a TV rate (30 images / second). That is, a still image is captured every 33 milliseconds. Normally, it is considered that the position of the photographed electric wire in the image does not change significantly within 33 milliseconds, but the photographed image is taken from above the electric wire by a helicopter, so a gust of wind etc. Due to the influence of the above, there is an image in which the electric wire shooting position in the image is greatly moved and the electric wire is imaged only at the edge of the screen. For example, FIG. 30 shows an example of an image in which an electric wire is photographed only at the edge of the screen. Thus, depending on the frame of the image, there may be an image in which the electric wire is not captured at the center of the image. Therefore, for such an image, if the image processing is performed only for the central portion of the screen as in the technique of Patent Document 1, the electric wire cannot be detected. FIG. 31 shows an example of a diagram showing a conventional search area. In order to solve this problem, data processing may be performed on the entire image, but the amount of data processing becomes enormous and the processing cannot be performed quickly.

  Therefore, in the present embodiment, for example, as shown in FIG. 32, the starting point / end point of the electric wire in the longitudinal direction of the electric wire is obtained from the target image of the electric wire cut out from the original image, and then the middle point from the obtained starting point / end point is obtained. A point is obtained, the midpoint is recorded, and the center position of the three points in the search area of the next frame is determined. Specifically, when cutting out the electric wire (S209), a straight line connecting the positions of the three points in the search area is drawn, and two points that are the intersections of the straight line and the end of the image are obtained, and the midpoint is calculated. To do. As a result, even when the position of the electric wire is gradually shifted from the center of the image, the search area moves with the movement of the position of the electric wire, so that the electric wire can be traced. . By moving the electric wire search position for each image to be processed in this way, the electric wire can be tracked even when the electric wire position is gradually shifted from the center of the image.

  For example, the degree of difference due to a square error is used to calculate the similarity between an area having the same size as the template in the search area (hereinafter referred to as a candidate area) and the template. In this case, if the brightness value of the template coordinates (x, y) is v (x, y) and the brightness value of the candidate area coordinates (x, y) is v ′ (x, y), the dissimilarity err is It is calculated by the following formula.

  A candidate area having the smallest difference err is set as an area most similar to the template in the search area, that is, a search result. However, the method of calculating the similarity between the candidate area and the template is not limited to the above example. For example, by using a method such as using cross-correlation for similarity calculation or determining similarity in the frequency domain, it is possible to perform more stable template matching with reduced influence of illuminance change.

  In the present embodiment, as “regularity of search results” that is a criterion for determining success or failure of search, it is determined whether or not the phases of displacement of search results are aligned. The phase here refers to that used in “topology”. For example, if two graphics expressed by line segments have the same shape, it is determined that the two graphics have the same phase, that is, the phases are aligned. In the case of this embodiment, if the three search results are, for example, the first, second, and third search results in order from the left, the slope of the line segment connecting the first search result and the second search result, If the slopes of the line segments connecting the second search result and the third search result can be regarded as the same, it is determined that the phases are aligned. The reason why the success or failure of the search can be determined depending on whether or not the phases of the displacements of the search results are aligned will be described below. That is, the electric wires are not in contact with other objects except for the steel tower portion, and are in a state of hanging according to the laws of physics. Therefore, the electric wire has a curved shape with a very small curvature, and can be regarded as a straight line within a small range within the screen without any problem. Therefore, the displacement of the wire position in the screen is relative to the movement of the helicopter or camera. The helicopter keeps flying at a constant speed along the electric wire and does not change its advancing direction suddenly. In addition, the camera does not rotate, and only pans in the vertical and horizontal directions (moving the lens left and right while leaving the camera position unchanged). As described above, the electric wire can move in the vertical direction in the three search areas with time change, but the movement is usually aligned in the three search areas.

An example of numerically expressing whether or not the phase of displacement of each search result is aligned is shown below. _Assume that y _1i , y _2i , and y _3i are the vertical coordinate positions of predetermined points of each search result, for example, the center point in each search result. Subscripts i = 0,..., P. Here, y ₁₀ , y ₂₀ , y ₃₀ are the coordinate positions in the current original image, y ₁₁ , y ₂₁ , y ₃₁ are the coordinate positions in the original image one frame before, and y _1p , y _2p , y _3p are The coordinate position in the original image before p frames is used. Then, based on the following formula, an average dy12rate of the difference between the distance between y _1i and y _2i and the distance between y _2i and y _3i in a certain period is calculated, and if the average dy12rate is within a predetermined constant value It is determined that the displacement phases of the search results are aligned, and when the predetermined values are exceeded, it is determined that the displacement phases of the search results are not aligned.

However, the determination of whether or not the phase of the displacement of each search result is aligned is not necessarily limited to that based on the above calculation. For example, the following more general calculation method may be used. For example, y1 = (y ₁₀ ,..., Y _1p ), y2 = (y ₂₀ ,..., Y _2p ), y3 = (y ₃₀ , y _3i using the above-described vertical coordinate positions y _1i , y _2i , y _3i . .., Y _3p ), and the cross-correlation of y1, y2, and y3 is calculated. Here, cor12 is a cross-correlation vector between y1 and y2, cor23 is a cross-correlation vector between y2 and y3, and cor31 is a cross-correlation vector between y3 and y1. In each cross correlation vector cor12, cor23, cor31, it is assumed that the maximum value is the j1, j2, j3th element, respectively. When these j1, j2, and j3 are both equal to or less than a certain value, it is determined that the phase of the displacement of each search result is aligned, and when the value exceeds the certain value, the phase of the displacement of each search result is not aligned. Judge. Note that j1, j2, and j3 are 0 when the phases are completely aligned. In the case of this calculation method, it is possible to detect whether or not the phase of displacement of each search result is aligned with higher accuracy. However, the calculation time is longer than the method based on Equation 2.

  FIG. 23 shows an example of a flowchart detailing the moving position estimation process of the electric wire. If the phase of the displacement of each search result is aligned by the above method, wire tracking success / failure processing is performed. Also, if the displacement phases of each search result are not aligned, it is considered that the search has failed, so if the displacement phases are not aligned, the coordinate position of each search result in the immediately preceding frame and Then, a difference (hereinafter referred to as a movement vector) from each coordinate position in this frame is obtained. After correcting the coordinate position of each search area using this movement vector, the tracking success / failure determination process of the electric wire is performed.

  FIG. 24 shows an example of a flowchart detailing the wire tracking success / failure determination process. In the technique of Patent Literature 1, when the phase of displacement of each search result is not aligned or when the similarity does not satisfy the defined standard, a template addition process is newly performed. The operator actually needs to specify the wire portion in the original image on the user interface screen by using a pointing device such as a mouse, and the processing is temporarily stopped at the frame. On the other hand, according to the present invention, the template update process is automatically updated every fixed frame without operator intervention. As a result, if the position of the electric wire is designated first, the electric wire abnormality detection process can be performed without performing the process involving the operator such as re-registration of the template thereafter.

  In the present embodiment, only when the wire tracking is successful, a new wire template pattern is registered every certain number of frames. For example, the template pattern is newly updated every 100 frames. Two types of template patterns are used: an initial registration template registered in advance and an updating template for each fixed frame. Note that the number of registered template patterns is limited to two because there is a limitation on the capacity of the main storage device. However, the number of template patterns is not limited to this, and three or more types may be used.

  In the electric wire tracking success / failure determination processing in the present embodiment, the inclination is about 6 ° (tan θ = 0.1), for example, if the electric wire inclination is equal to or larger than a preset threshold value compared to the image of the immediately preceding frame. ) If it is above, it is determined that the tracking of the wire has failed. The threshold value can be arbitrarily set, and is not limited to the above example. In addition, the inclination of the electric wire is obtained by calculating the inclination of the electric wire from the coordinate position of the starting point and the end point at the same time as obtaining the starting point and the end point of the electric wire in the image.

  Furthermore, in this embodiment, when it is determined that the tracking has failed because the inclination exceeds the threshold, the search area is expanded and the process returns to the template matching process (S206) to search for the wire again. For example, FIGS. 33A to 33C show examples of diagrams in which the search area is gradually expanded. For example, if the initial search area is 3 times the diameter of the wire and the search result within the 3 times search range is compared to the image of the immediately preceding frame, The search area is expanded to 7 times the diameter of the electric wire, and the process returns to the template matching process (S206) to search for the electric wire. Furthermore, if the inclination of the electric wire is tan θ is 0.1 or more, the search area is 15 times the diameter of the electric wire (a normal photographed image) as long as the inclination of the electric wire is tan θ is 0.1 or more compared to the image of the immediately preceding frame within the search range of 7 times. The entire range in the vertical direction) and return to the template matching process (S206) to search for the electric wire. In other words, the search of the electric wire is performed by expanding the search area in three stages of 3 times, 7 times, and 15 times. It should be noted that how much the search area is expanded and how many times the search area is expanded can be arbitrarily set, and is not limited to the above example. In addition to setting a threshold value for the diameter of the electric wire, the threshold value may be set based on another value. For example, a threshold value may be set for a pixel such as 10 pixels or 20 pixels with the number of pixels as a threshold value.

  The reason why the search area is gradually expanded in a plurality of times is to limit the data processing range and perform faster processing. For example, if a threshold value of 15 times is set from the beginning, even if the position of the electric wire has not moved much, data processing in a 15 times range will be performed, resulting in increased processing waste. For this reason, the search area is gradually expanded, and only when the search area is not found, the search area is expanded. Thereby, it is possible to perform quick processing except for useless data processing. It should be noted that the search area may be expanded at least once, not necessarily twice, or three or more times.

  However, even when the search area is expanded in the entire vertical direction of the screen, there is a case where the electric wire cannot be searched. Naturally, this is the case when the wire itself is not photographed in the image, but it has been found that even when the wire is photographed, tracking of the wire may fail. This is due to changes in the brightness of the electric wires caused by the shooting environment such as sunlight, clouds and helicopter positions. In addition, since the video camera shoots the electric wire from above with a helicopter, the background image naturally becomes the ground background. In this case, for example, when the background (for example, a house, a factory, etc.) where the color information value is close to the electric wire is photographed, the detection of the electric wire may fail. That is, there is a case where erroneous detection is caused due to the shadow of the electric wire on the photographed image.

  Therefore, in this embodiment, as described above, by storing the template pattern of the electric wire in the photographed image in the main storage device in advance, the abnormality detection processing of the electric wire regardless of the photographing environment such as a change in sunlight. To be able to do that. If the search of the electric wire fails even if the search area is enlarged, the electric wire is searched by changing the template pattern of the electric wire registered in advance from the image of the immediately preceding frame.

  In this embodiment, the template pattern of the electric wire is input in advance at the start of processing as described above, and the template pattern is stored and updated at a fixed time interval arbitrarily specified during the processing. I am going to go. Specifically, if the search of the electric wire fails even if the above search area is expanded, the template pattern of the electric wire to be referenced is changed, and the normal search area is searched again. Yes. Furthermore, even if the search is performed by changing the template pattern of the electric wire to be referred to, if the search for the electric wire fails, the search area is further expanded to search for the electric wire. The template pattern of the electric wire may be updated at an arbitrary time interval. Alternatively, a plurality of electric wire template patterns may be stored, and the processing may be continued while changing the reference electric wire template pattern until the electric wire can be detected.

  From the results of the experiment, the process of enlarging the search area and determining the inclination of the electric wire, changing the template pattern of the electric wire to be referenced, and expanding the search area again and performing the process of determining the inclination of the electric wire, It has been found that wire detection is successful for all images. In the present embodiment, first, the electric wire is detected while changing the search area in three stages. If the electric wire cannot be detected even in that case, the template pattern of the electric wire to be referenced is changed, and then again. The electric wire is detected while changing the search area in three stages. The number of stages in which the process of expanding the search area and the process of changing the template pattern of the electric wire to be referred to can be arbitrarily set, and is not limited to the above example. Moreover, it is not always necessary to perform both processes, and only one of the processes may be performed. The order in which the two processes are performed is not limited to the above example.

  As a result of the experiment, there was almost no failure in tracking of the wire due to the above-described success / failure tracking processing of the wire, but there is a possibility of failure with a slight probability due to the influence of the background. For this reason, in this embodiment, even if the above-described wire tracking success / failure determination process is performed, if the wire search fails, the image of the frame is recorded as a still image, and the process proceeds to the next frame. I am doing so. By doing in this way, even if it is a case where detection of an electric wire fails, it is supposed that a process will not be stopped. As a result, the operator can specify the position of the electric wire first, and thereafter can perform the process without any intervention until the end of the process, thereby enabling a quick process.

  In the electric wire abnormality detection process (S3), for example, as shown in FIG. 13, the electric wire is sound using the process (S306) for detecting the edge pixel constituting the actual outline of the electric wire and the detected edge pixel. A process for obtaining an ideal contour line (S307), a process for detecting an abnormality in the shape of the electric wire based on information on the edge pixel and the ideal contour line (S308), and a color information value of an area surrounded by the ideal contour line And the process of detecting an abnormality in the color of the electric wire (S309).

  Edge pixels constituting the actual outline of the electric wire are detected from the target image cut out from the original image and stored in the main storage device. Among the target images obtained by cutting out the electric wire portion from the original image, only the electric wire has the edge, that is, the contour information clearly appears, and the electric wire region can be accurately extracted by detecting the edge pixel.

  For example, with respect to an image as shown in FIG. 7, as indicated by an arrow in the figure, the luminance value between two adjacent pixels in one direction in the vertical direction that is the electric wire crossing direction, for example, in the upward direction in the figure. FIG. 8 shows the relationship between the difference value obtained and the vertical coordinate position. Among the points protruding from the difference value 0 appearing in FIG. 8, in other words, the difference value is larger or smaller than both of the difference values on both sides (hereinafter also referred to as a peak), the absolute value is large. The thing has shown the electric wire part. Therefore, if both ends of the peak indicating the electric wire portion can be detected, the electric wire region can be accurately extracted.

  Here, there is a method of searching for a peak where the absolute value of the difference value exceeds the threshold and determining the pixel constituting the peak as an edge pixel. However, in this case, an appropriate threshold fluctuates depending on the sun direction, weather, and the like. Therefore, accurate edge detection cannot be performed under a certain threshold.

  Therefore, in the edge detection processing of the present embodiment, for example, as shown in FIG. 14, for each pixel row in the wire crossing direction in the target image, the difference in color information value between two adjacent pixels facing in one direction in the wire crossing direction. The value is obtained (S306-5). Then, as shown in FIG. 15, a point where the absolute value of the difference value is maximum is obtained (S306-7-1), and is within a predetermined range from the maximum point in both directions in the electric wire crossing direction and A point that is farthest from the maximum point and protrudes from the difference value 0 is obtained, and the pixels constituting each protruding point are determined as edge pixels (S306-7-6, S306-7-11).

  The above principle will be described with reference to FIG. 9 which is an enlarged view of FIG. A point P1 where the absolute value of the difference value is maximum is searched for in FIG. Further, a peak P2 that is within a predetermined range Δj and is farthest from the maximum point P1 around the maximum point P1 is searched. Here, the range Δj for searching for the peak P2 is determined based on the thickness of the electric wire in the target image, for example. For example, it is assumed that the thickness of the electric wire is about 40 pixels, and the outermost peak P2 is searched for in the range Δj of ± 20 pixels around the maximum point P1. And let the pixel which comprises the peak P2 be an edge pixel. Here, each peak P2 is composed of two pixels, and one of the predetermined pixels is an edge pixel. As a result, two edge pixels are obtained. One of the two edge pixels is called a first edge pixel, and the other is called a second edge pixel. In the present embodiment, the upper edge pixel in the target image is the first edge pixel, and the lower edge pixel is the second edge pixel. The above processing is performed for all the vertical pixel rows of the target image.

  Next, based on the edge pixel detected by the above-mentioned process, the process which calculates | requires the ideal outline in case an electric wire is healthy is performed. For example, as shown in FIG. 16, low-reliability pixels that do not satisfy a predetermined criterion are excluded from the first edge pixel group and the second edge pixel group, and the remaining first edge pixel group and second edge pixel group are respectively straight lines. To obtain two ideal contour lines (S307-1 to S307-3).

  For example, in the present embodiment, the following processing is performed on the first edge pixel group and the second edge pixel group, respectively, so that low-reliability pixels that greatly impair the linearity are removed. That is, as shown in FIG. 17, each edge pixel is a pixel of interest, and a vertical range that is a predetermined range in the horizontal direction that is the longitudinal direction of the wire around the pixel of interest, for example, ± 15 pixels is the wire transverse direction of the edge pixel. The average of the coordinate positions in the direction is obtained (S307-1-3, S307-1-5). Then, edge pixels whose vertical coordinate position is significantly different from the average, that is, edge pixels whose absolute value of the difference between the vertical coordinate position and the average is equal to or greater than a predetermined threshold value are excluded as low reliability pixels. (S307-1-4, S307-1-6).

  In addition, an edge pixel in which the distance between the paired first edge pixel and the second edge pixel is extremely narrow or wide, in other words, the vertical direction between the first edge pixel and the second edge pixel having the same horizontal coordinate position. Edge pixels whose direction coordinate position difference is not less than a predetermined upper limit value or less than a lower limit value are excluded as low reliability pixels (S307-1-7, S307-1-8).

  As a method of applying a straight line to the edge pixel group excluding the low reliability pixels, for example, a least square method is used in the present embodiment. However, the method of linear approximation is not limited to this example. The straight lines obtained based on the first edge pixel group and the second edge pixel group excluding the low reliability pixels are referred to as a first ideal contour line and a second ideal contour line, respectively.

  In the shape abnormality detection process using the ideal outline (S308 in FIG. 13), the edge pixels that are separated from the ideal outline by a predetermined distance in the vertical direction that is the electric wire crossing direction are horizontal in the electric wire longitudinal direction. When a predetermined number of lines continue in the direction, it is determined that there is a possibility that an abnormality has occurred in the shape of the electric wire due to a broken wire or the like. For example, when an edge pixel that is 10 pixels or more away from the ideal contour line appears over a length of 20 pixels, it is determined that there is a possibility of a wire abnormality such as a broken wire. This utilizes the fact that when a part of the electric wire is cut off and jumps to the outside, the edge pixels come together to some extent at some distance from the ideal contour line.

  On the other hand, in the color abnormality detection process (S309 in FIG. 13), a representative value in the wire transverse direction is obtained from the color information value of the pixel row in each wire transverse direction sandwiched between the first and second ideal contour lines, and the color information When representative values that deviate from the threshold value continue for a predetermined number in the longitudinal direction of the electric wire, it is determined that there is a possibility that the color of the electric wire may be abnormal due to wire breakage, arc marks, or scratches. .

  For example, in this embodiment, the average value of the luminance values of the vertical pixel columns sandwiched between the first and second ideal contour lines is set as the representative value in the vertical direction. However, the present invention is not limited to this example, and the mode value of the luminance value of each vertical pixel column sandwiched between the first and second ideal contour lines may be used as the representative value in the vertical direction.

  In the present embodiment, the threshold value of the color information to be compared with the representative value is calculated for each target image. For example, as shown in Formula 3, a value obtained by multiplying the standard deviation of the representative value obtained for a certain target image by α is added to the average value of the representative values obtained for the target image, and the value is added to the upper limit of the target image. And a value obtained by subtracting a value obtained by multiplying the standard deviation of the representative value by α from the average value of the representative values is set as the lower limit threshold value. Here, α is an arbitrary coefficient. When α was changed from 0.5 to 1.5, it was possible to accurately detect an abnormal electric wire particularly when α = 0.9. Therefore, in this embodiment, α = 0.9 is set.

<Equation 3>
Threshold = average value of representative values ± α × standard deviation of representative values

  When a representative value that is equal to or higher than the upper limit threshold is continuous for 20 pixels or more in the horizontal direction, or when a representative value that is equal to or lower than the lower limit threshold is continuous for 20 pixels or more in the horizontal direction, It is determined that there is a possibility of wire abnormality. This utilizes the fact that when an electric wire has an arc mark or a flaw, the abnormal portion becomes darker or whitish than a normal portion of the electric wire.

  Here, in the above-described color abnormality detection process, the average value of the luminance values of the pixel rows in the electric wire transverse direction sandwiched between the first edge pixel and the second edge pixel may be used as the representative value in the electric wire transverse direction. It is done. However, in this case, if there is a break in the wire part of the wire is jumping outside, the luminance value of the non-wire part, for example, the luminance value of the background is also included in the average value calculation, and an accurate color abnormality It cannot be detected. As in this embodiment, by calculating the average of the luminance values in the region sandwiched between the first and second ideal contour lines and using it as the representative value, the luminance value of the portion that is not the electric wire is included in the calculation of the average value. This provides an advantage of preventing and performing accurate color abnormality detection.

  Here, the target image of the present embodiment is obtained from an aerial image, and an abnormal portion of the electric wire appears over several frames. Therefore, in the present embodiment, it is determined that an abnormality has occurred in the electric wire when the target image that is determined to have the possibility that the electric wire has an abnormality continues for a predetermined number, for example, two frames or more. I have to. Thereby, a more reliable electric wire abnormality detection can be performed. However, if the flight speed of the helicopter is slow, etc., abnormal parts appear in more continuous frames. Therefore, if more than a number of frames continue, it may be determined that an abnormality has occurred in the electric wire. good.

  When the electric wire abnormality detection process is completed, an abnormal image output process (S4) recorded in the log is performed. In the present embodiment, an original image in which the target image could not be cut out or a continuous image of abnormally detected images is converted into a moving image file (for example, avi file, mpeg2 file, etc.) and can be reproduced on the output device 14 It is supposed to be. Thereby, since it can confirm with the state of a moving image, the confirmation by an operator's visual observation is made easier. The operator can visually determine the presence or absence of a problem by looking at the moving image displayed on the output device 14. For example, in the color abnormality detection process of the present embodiment, there is a possibility that it is determined that there is an abnormality even when there is a stain or the like on the electric wire. The person does it. In addition, you may record as not only a moving image but a still image. For example, still images may be displayed in an album form. Alternatively, both a moving image and a still image may be recorded, and an operator may perform a confirmation operation using both images.

  The wire abnormality detection method described above can be implemented as a wire abnormality detection device using, for example, a known computer. For example, as shown in FIG. 10, the electric wire abnormality detection device 10 includes an input interface 11 to which video images of electric wires are input, and a hard disk as an external storage device 12 in which data such as original images and target images are recorded. A RAM as a main storage device 13 in which temporary work data and the like are recorded, an output device 14 in which an image in which an abnormality is detected is output, a central processing unit (CPU) 15 and the like. . The hardware resources are electrically connected through the bus 16, for example. The input interface 11 converts, for example, a signal input from a video camera or read from a medium such as a DVD on which video is recorded into data that can be processed by a computer, and each frame constituting the video is an image. It has a function of recording in the external storage device 12 as data. As such an input interface 11, for example, an existing NTSC-RGB converter, a frame grabber, an image capturing board for a personal computer, or the like may be used. The output device 14 is a display, for example, and displays an image in which an abnormality is detected, a user interface screen, and the like. Moreover, the electric wire abnormality detection program of the present invention is recorded in the external storage device 12, and the computer functions as the electric wire abnormality detection device 10 when the program is read and executed by the CPU 15. In addition, the wire abnormality detection device 10 detects an edge of a target image cut out by a target image cut-out processing unit that cuts out a portion where the wire is shot from an original image taken of the wire, and a target image cut out by the cut-out processing of the target image. A wire abnormality detection processing unit that performs processing, ideal contour estimation processing, shape abnormality detection processing, or color abnormality detection processing.

  An example of processing executed by the wire abnormality detection device 10 by the wire abnormality detection program of this embodiment will be described along the flowcharts of FIGS. First, pre-processing is performed (S1). FIG. 22 shows a flowchart detailing this preprocessing. First, a time code is designated (S101), and image data corresponding to the time code designated in the external storage device 12 is read from the video camera or the medium on which the video is recorded via the input interface 11 (S102). . Then, the image data recorded in the external storage device 12 is converted into, for example, an 8-bit grayscale image (S103). A Gaussian filter is applied to the original image thus obtained to remove noise components from the original image (S104). In order to avoid such a situation, sudden noise such as impulse noise appears in the image and a large peak is generated in the difference calculation in the edge detection process, causing erroneous detection of the edge. It is. This completes the preprocessing.

  After the preprocessing is completed, the target image is cut out (S2 in FIG. 11). FIG. 12 shows a detailed flowchart of the target image cut-out process. In this process, first, 1 is set to the frame counter n (S201). Then, a template registration process is performed (S202). For example, as shown in FIG. 25, the original image of the nth frame, that is, the first frame is displayed in the window 22. On this screen, the operator selects the upper position 23 of the electric wire displayed in the window 22, and further selects the radio button “upper position of the electric wire” 21 with a pointing device such as a mouse. Next, when the selection is canceled by moving to the lower side of the electric wire while being selected with a mouse or the like (FIG. 26), a rectangle 24 automatically appears (FIG. 27). Finally, the setting completion button is pressed to complete the template registration process (S202).

  When the n-th frame original image exists (S203; Yes), the original image is read (S204). Here, since the wire search is performed for the original image of the first frame when the above template is registered, it is possible to add 1 to the frame counter n in the template registration process (S202). good. Three search areas are arranged with the midpoint of the detection position of the electric wire in the previous frame as the center in the vertical direction of the search position (S205). A template matching process for searching for a region most similar to the template in the search region is performed (S206).

  Next, an electric wire movement position estimation process (S207), an electric wire tracking success / failure process are performed (S208), and an image including at least each search result is cut out from the original image, and is stored in the external storage device 12 as an nth target image. Record and record the coordinate position of the midpoint of the search result for the next placement of the search area (S210). Then, 1 is added to the frame counter n (S211), and the processing from S203 onward is repeated for the original image of the nth frame which is the next frame.

  FIG. 23 shows an example of a flowchart detailing the wire movement position estimation process (S207). The coordinate position in the width direction of each search result is obtained (S207-1). If the phase of the displacement of each coordinate position is aligned (S207-2; Yes), the wire tracking success / failure process (S208) is performed. If the phase of displacement of each coordinate position is not aligned (S207-2; No), a difference (movement vector) from the coordinate position of each search result in the immediately preceding frame is obtained (S207-3), and each search area The absolute value of the difference between the movement vectors is obtained (S207-4). Further, after correcting the coordinate position in the other search area with reference to the movement vector in the search area where the absolute value of the difference between the movement vectors is the smallest (S207-5), the wire tracking success / failure process ( S208) is performed. This completes the wire movement position estimation process.

  FIG. 24 shows an example of a flowchart detailing the wire tracking success / failure determination process. If the inclination of the electric wire is equal to or greater than a preset threshold value (here, tan θ = 0.1) (S208-1; Yes), the search range is expanded from three times to seven times the diameter of the electric wire (S208- 2) Perform template matching processing and wire movement position estimation processing again (S206 to S207). The number of times to expand the search range is not particularly limited, but here the search range is expanded twice. Even if the search range is expanded, if the inclination of the electric wire is not less than the preset threshold value (S208-3; Yes), the search range is expanded from 7 times to 15 times the diameter of the electric wire (S208-4), Template matching processing and wire movement position estimation processing are performed again (S206 to S207). Even if the search range is expanded again, if the inclination of the electric wire is equal to or greater than a preset threshold value (S208-5; Yes), the template pattern of the electric wire to be referenced is changed, and the template matching process and the electric wire moving position are performed again. An estimation process is performed (S206 to S207), and the electric wire tracking success / failure determination process is terminated. Moreover, if the inclination of the electric wire is less than a preset threshold value at any time (S208-1, S208-3, S208-5; No), it is determined that the electric wire has been successfully tracked. The process ends. This completes the wire tracking success / failure determination process.

  When the above processing is performed for all the original images (S203; No), the target image cutout processing is terminated, and the wire abnormality detection processing is performed (S3 in FIG. 11). FIG. 13 shows a detailed flowchart of the electric wire abnormality detection process. In this process, 1 is set in the image counter k (S301), 0 is set in the shape abnormality counter z (S302), and 0 is set in the color abnormality counter q (S303). If the k-th target image exists (S304; Yes), the target image is read (S305), the edge detection process (S306), the ideal contour estimation process (S307), the shape are read for the read target image. An abnormality detection process (S308) and a color abnormality detection process (S309) are performed. After completion of these processes, 1 is added to the image counter k, and the processes from S304 onward are repeated for the kth target image corresponding to the next frame. If the above process is performed about all the target images (S304; No), an electric wire abnormality detection process will be complete | finished.

  FIG. 14 shows a detailed flowchart of the edge detection process. In this process, 1 is set to the pixel number i in the horizontal direction, which is the longitudinal direction of the electric wire (S306-1). Then, it is determined whether the value of the horizontal pixel number i exceeds the horizontal pixel number i_max of the target image (S306-2). If the horizontal pixel number i is equal to or less than i_max (S306-2; Yes), 1 is set to the pixel number j in the vertical direction that is the electric wire crossing direction (S306-3). Then, it is determined whether the value of the vertical pixel number j exceeds the value obtained by subtracting 1 from the number of vertical pixels j_max of the target image (S306-4). If the vertical pixel number j is equal to or smaller than j_max−1 (S306-4; Yes), the difference between the luminance value of the pixel at the coordinate (i, j) and the luminance value of the pixel at the coordinate (i, j + 1) is obtained. The difference value (j) is recorded (S306-5). Then, 1 is added to the vertical pixel number j (S306-6), and the processes after S306-4 are repeated. When the value of the vertical pixel number j exceeds j_max-1 (S306-4; No), all the luminance value difference values between two adjacent pixels in the vertical pixel row at the horizontal pixel number i are obtained. Next, edge pixel search processing is performed (S306-7).

  FIG. 15 shows a detailed flowchart of the edge pixel search process. In this process, the difference value (j ′) having the maximum absolute value is obtained from the recorded difference values, and the vertical pixel number j ′ corresponding to the difference value (j ′) is obtained (S306-7-1). . A value obtained by adding, for example, 20 to Δ ′ as Δj is set as J (S306-7-2). Then, it is determined whether or not the difference value (J) corresponding to the vertical pixel number J constitutes a peak. Specifically, whether difference value (J)> 0, difference value (J)> difference value (J-1), and difference value (J)> difference value (J + 1) (S306-7-) 3) or whether difference value (J) <0 and difference value (J) <difference value (J-1) and difference value (J) <difference value (J + 1) (S306-7-) 4). When the difference value (J) constitutes a peak (S306-7-3; Yes or S306-7-4; Yes), the pixel at the coordinates (i, J) is recorded as the first edge pixel (S306). -7-6). When the difference value (J) does not constitute a peak (S306-7-3; No and S306-7-4; No), 1 is subtracted from the vertical pixel number J (S306-7-5), and J is the center. One pixel is brought closer to the vertical pixel number j ′, and it is examined again whether or not the difference value (J) corresponding to the vertical pixel number J forms a peak. Similarly, a value obtained by subtracting 20 as Δj from j ′ is set as J ′ (S306-7-7), and it is determined whether or not the difference value (J ′) corresponding to the vertical pixel number J ′ constitutes a peak ( S306-7-8, S306-7-9), when forming a peak, the pixel at coordinates (i, J ′) is recorded as the second edge pixel (S306-7-11), and no peak is formed 1 is added to the vertical pixel number J ′ (S306-7-10), and J ′ is brought close to the vertical pixel number j ′ centered by one pixel, and whether or not the difference value (J ′) forms a peak. Judge again. As described above, a pixel that constitutes a peak that is in the range ± Δj determined from the point j ′ where the difference value is maximum and is farthest from the maximum point j ′, that is, an edge pixel is obtained.

  When the first edge pixel and the second edge pixel are obtained for the horizontal pixel number i, the process returns to the process of FIG. 14, and 1 is added to the horizontal pixel number i (S306-8), and S306-2 for the horizontal pixel number i. The following process is repeated. As described above, the first edge pixel and the second edge pixel are obtained for each vertical pixel column constituting the target image, in other words, for each horizontal pixel number. When the value of the horizontal pixel number i exceeds the horizontal pixel number i_max of the target image (S306-2; No), the edge detection process in the target image is terminated.

  When the edge detection process is completed, an ideal contour estimation process (S307) is performed. FIG. 16 shows a detailed flowchart of the ideal contour estimation process. In this process, first, a removal candidate pixel search process is performed (S307-1). Further, in this process, as shown in detail in FIG. 17, 1 is set to the pixel number i in the horizontal direction which is the longitudinal direction of the electric wire (S307-1-1). Then, it is determined whether the value of the horizontal pixel number i exceeds the horizontal pixel number i_max of the target image (S307-1-2). If the horizontal pixel number i is less than or equal to i_max (S307-1-2; Yes), the average value of the coordinate positions in the vertical direction that is the wire crossing direction for the first edge pixels existing in the horizontal pixel numbers i−Δi to i + Δi Is determined as Ave1 (S307-1-3). Δi is set to 15, for example. Then, among the first edge pixels existing in the horizontal pixel numbers i−Δi to i + Δi, the edge pixels whose absolute value of the difference between the vertical coordinate position and the average value Ave1 is equal to or larger than a predetermined threshold value are removed. It records as a candidate pixel (S307-1-4). Similarly, for the second edge pixels existing at the horizontal pixel numbers i−Δi to i + Δi, the average value of the coordinate positions in the vertical direction is obtained as Ave2 (S307-1-5). Edge pixels in which the absolute value of the difference between the vertical coordinate position and the average value Ave2 is equal to or greater than a predetermined threshold are recorded as removal candidate pixels (S307-1-6). Further, the difference between the positions of the first edge pixel and the second edge pixel in the horizontal pixel number i in the vertical direction is obtained and set as ΔD (S307-1-7). When ΔD is greater than or equal to a predetermined upper limit value or less than a lower limit value, the first edge pixel and the second edge pixel at the horizontal pixel number i are recorded as removal candidate pixels (S307-1-8). Then, 1 is added to the horizontal pixel number i (S307-1-9), and the processing from S307-1-2 onward is repeated for the horizontal pixel number i. As described above, an edge pixel greatly deviating from the average of surrounding edge pixels whose vertical coordinate position is within a range of ± 15 pixels, or an edge where the distance between the paired first edge pixel and the second edge pixel is extremely narrow or wide Pixels are recorded as removal candidate pixels. When the value of the horizontal pixel number i exceeds the horizontal pixel number i_max of the target image (S307-1-2; No), the removal candidate pixel search process in the target image ends.

  When the removal candidate pixel is obtained, the process returns to the process of FIG. 16, and the remaining pixels excluding the removal candidate pixel, that is, the low-reliability pixel that impairs the linearity of the edge, from the first edge pixel group are used. A first contour line is created (S307-2). Similarly, a second contour line is created by the least square method using the remaining pixels obtained by removing the removal candidate pixels from the second edge pixel group (S307-3). Thereby, the ideal outline estimation process in the target image is finished, and the shape abnormality detection process (S308) and the color abnormality detection process (S309) are performed on the target image.

  18 and 19 show flowcharts detailing the shape abnormality detection processing. In this processing, 0 is set in the counters w and w '(S308-1 and S308-2), and 1 is set in the horizontal pixel number i which is the longitudinal direction of the electric wire (S308-3). Then, it is determined whether the value of the horizontal pixel number i exceeds the horizontal pixel number i_max of the target image (S308-4). If the horizontal pixel number i is equal to or less than i_max (S308-4; Yes), the vertical coordinate that is the electric wire crossing direction between the first edge pixel at the horizontal pixel number i and the point on the first contour line at the horizontal pixel number i. A difference in position is obtained and set as ΔH1 (S308-5). Similarly, the difference in the vertical coordinate position between the second edge pixel at the horizontal pixel number i and the point on the second contour line at the horizontal pixel number i is obtained, and is set as ΔH2 (S308-6). Then, it is determined whether the absolute value of ΔH1 and the absolute value of ΔH2 are equal to or greater than a predetermined value H_max (S308-7, S308-9). H_max is set to 10, for example. When ΔH1 is greater than or equal to H_max, 1 is added to the counter w (S308-8), and when ΔH2 is greater than or equal to H_max, 1 is added to the counter w ′ (S308-10). Then, it is determined whether the value of the counter w or the counter w ′ is equal to or greater than a predetermined value w_max (S308-11). For example, w_max is 20. If both of the counters w and w ′ do not satisfy w_max (S308-11; No), 1 is added to the horizontal pixel number i (S308-13), and the processing from S308-4 onward is repeated for the horizontal pixel number i. . On the other hand, if both ΔH1 and ΔH2 are less than H_max, 0 is set to the counters w and w ′ (S308-12), 1 is added to the horizontal pixel number i (S308-13), and the horizontal pixel number The process from S308-4 onward is repeated for i.

  When at least one of the counters w and w ′ is equal to or greater than w_max (S308-11; Yes), edge pixels that are separated by 10 pixels or more from the ideal contour line appear continuously, so that the target image Is recorded as a shape abnormality candidate image (S308-14). Then, 1 is added to the shape abnormality counter z (S308-15), and it is determined whether the counter z is equal to or greater than a predetermined value z_max (S308-16). z_max is set to 2, for example. If the shape abnormality counter z does not satisfy z_max (S308-16; No), the shape abnormality detection process for the target image is terminated. On the other hand, when the shape abnormality counter z is equal to or greater than z_max (S308-16; Yes), the shape abnormality candidate images are continuous for two or more frames, and therefore these shape abnormality candidate images are recorded in the log (S308-17). ). Then, the shape abnormality counter z is reset to 0 (S308-18), and the shape abnormality detection process for the target image ends. On the other hand, when the processing is performed until the value of the horizontal pixel number i exceeds the horizontal pixel number i_max of the target image (S308-4; No), 20 edge pixels that are separated by 10 pixels or more from the ideal contour line are consecutive. In this case, the shape abnormality counter z is reset to 0 (S308-18), and the shape abnormality detection process for the target image ends.

  FIG. 4 shows an image obtained by photographing an electric wire that is causing a wire break. FIG. 2 shows the result of performing the above edge detection processing on the image of FIG. Further, the result of obtaining the ideal contour based on the remaining edge pixels excluding the low reliability pixel is identified by the above-described ideal contour estimation process from the edge pixels shown in FIG. As shown in FIG. FIG. 1 shows a result of displaying the edge pixel of FIG. 2 and the ideal outline of FIG. 3 in an overlapping manner. In the figure, reference numeral 40 indicates a first edge pixel, reference numeral 41 indicates a second edge pixel, reference numeral 42 indicates a first ideal contour line, and reference numeral 43 indicates a second ideal contour line. FIG. 1 shows a state in which 20 pixels or more of edge pixels that are 10 pixels or more away from the ideal contour line appear continuously, and the above-described shape abnormality detection processing accurately detects the wire that has been broken. I can confirm that I can do it.

  Next, FIG. 20 and FIG. 21 show flowcharts detailing the color abnormality detection processing. In this processing, 0 is set to the counters u and u '(S309-1, S309-2), and 1 is set to the pixel number i in the horizontal direction which is the longitudinal direction of the wire (S309-3). Then, it is determined whether the value of the horizontal pixel number i exceeds the horizontal pixel number i_max of the target image (S309-4). If the horizontal pixel number i is equal to or less than i_max (S309-4; Yes), the average value of the luminance values of the pixels located between the first contour line and the second contour line at the horizontal pixel number i is obtained, and the average The value is set as a representative value Val (i) (S309-5). Then, 1 is added to the horizontal pixel number i (S309-6), and the processing from S309-4 onward is repeated for the horizontal pixel number i. Thereby, the representative value Val (i) is obtained for each horizontal pixel number i. When the horizontal pixel number i exceeds i_max (S309-4; No), the upper limit threshold value Val ′ and the lower limit threshold value Val ″ are obtained by Expression 3 using the obtained representative values Val (1) to Val (i_max) ( S309-7).

  Then, the horizontal pixel number i is set to 1 again (S309-8), and it is determined whether the horizontal pixel number i is equal to or less than i_max (S309-9). If the horizontal pixel number i is not more than i_max, it is determined whether the representative value Val (i) corresponding to the pixel number i is not less than the upper limit value Val ′ or not more than the lower limit value Val ′ (S309-10, S309). -12). When the representative value Val (i) is greater than or equal to the upper limit value Val ′, 1 is added to the counter u (S309-11), and when the representative value Val (i) is less than or equal to the lower limit value Val ″, 1 is added to the counter u ′. (S309-13) Then, it is determined whether the value of the counter u or the counter u ′ is equal to or larger than a predetermined value u_max (S309-14), where u_max is set to 20, for example. If both do not satisfy u_max (S309-14; No), 1 is added to the horizontal pixel number i (S309-16), and the processing from S309-9 onward is repeated for the horizontal pixel number i. If (i) is not greater than or equal to the upper limit value Val ′ and not less than the lower limit value Val ″, 0 is set in the counters u and u ′ (S309-15), and 1 is added to the horizontal pixel number i (S309-16). ), S3 for the horizontal pixel number i 9-9 repeats the following processing.

  When at least one of the counters u and u ′ is greater than or equal to u_max (S309-14; Yes), a representative value Val (i) that is greater than or equal to the upper limit value Val ′ or less than or equal to the lower limit value Val ″ appears continuously for 20 pixels. Therefore, the target image is recorded as a color abnormality candidate image (S309-17), and 1 is added to the color abnormality counter q (S309-18), and the counter q is equal to or greater than a predetermined value q_max. In step S309-19, q_max is set to 2. If the color abnormality counter q is less than q_max (S309-19; No), the color abnormality detection process for the target image is terminated. If the color abnormality counter q is equal to or greater than q_max (S309-19; Yes), the color abnormality candidate images are continuous for two or more frames, and therefore these color abnormality candidate images are recorded in the log (S309). -20) Then, the color abnormality counter q is reset to 0 (S309-21), and the color abnormality detection process for the target image is terminated, whereas the value of the horizontal pixel number i is the number of pixels in the horizontal direction of the target image. When the processing is performed until i_max is exceeded (S309-9; No), the representative value Val (i) that is greater than or equal to the upper limit value Val ′ or less than or equal to the lower limit value Val ″ does not appear continuously for 20 pixels. Then, the color abnormality counter q is reset to 0 (S309-21), and the color abnormality detection process for the target image ends.

  FIG. 6 shows an image in which an electric wire with an arc mark is photographed. FIG. 5 shows the result of the above color abnormality detection process performed on the image of FIG. In the figure, reference numeral 44 indicates a representative value in each horizontal pixel number, and reference numeral 55 indicates a lower limit threshold. FIG. 5 shows a state in which a representative value equal to or lower than the lower limit threshold appears continuously for 20 pixels or more, and it can be confirmed that an electric wire having an arc mark can be accurately detected by the color abnormality detection process.

  When the wire abnormality detection process for all frames is completed (S3), the abnormal image recorded in the log is output as a moving image file (S4 in FIG. 11). Note that the process of outputting an abnormal image as a moving image file may be performed after completion of the process for all the frames, or may be performed in parallel during the process. Thus, the processing executed by the wire abnormality detection device 10 is completed by the wire abnormality detection program of the present embodiment.

  According to the present invention, it is not necessary for the operator to visually inspect all the aerial images of the electric wires from the beginning, and it is only necessary to check some images and videos presented by the electric wire abnormality detection device 10. It is greatly reduced. Furthermore, by connecting the video camera to the electric wire abnormality detection device 10 and processing the image obtained from the video camera in real time, it becomes possible to detect the abnormal portion of the electric wire at the site where the helicopter patrols, It becomes possible to reduce human resources and equipment resources related to work and to greatly reduce the overall cost.

  The above-described embodiment is an example of a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the gist of the present invention. For example, in the above-described embodiment, each process is performed using a grayscale image as an original image, but each process may be performed using a color image as an original image. For example, when the original image is a grayscale image, the target image may be judged as a search result in which the ridge lines of the mountains are aligned in the phase of displacement in the clipping process of the target image. Such a failure can be avoided by performing template matching using the hue value in addition to the value.

  Further, the edge detection process is not necessarily limited to the example of the above-described embodiment, and a conventional edge detection method may be adopted. In order to further increase the processing speed, a part or all of the processing shown in FIGS. 11 to 24 may be implemented as hardware. In particular, since the edge detection processing part is a general-purpose method, it has good compatibility with a commercially available image processing board and can be easily implemented as hardware. By using hardware, processing at the video rate can also be expected.

  Each parameter such as a threshold, for example, a range Δj for searching for a peak in the edge detection process, a reference H_max of a distance between an edge pixel for determining a shape abnormality and an ideal outline, a threshold coefficient α for determining a color abnormality, a continuous abnormality The criteria w_max and u_max for candidate pixels and the criteria z and q for consecutive abnormal candidate images may be adjusted as appropriate according to the shooting conditions.

  In the above-described embodiment, the wire abnormality detection process is performed on the target image cut out from the original image. However, in some cases, the wire abnormality detection process may be performed using the original image as it is as the target image. Moreover, you may use the target image obtained by the above-mentioned cutting-out process for the image processing for electric wire inspections other than a shape abnormality detection process and a color abnormality detection process.

  In addition, by using the color information value in addition to the luminance value, for example, the detection accuracy of the electric wire with a red line in which a red line for twist confirmation such as OPGW (optical fiber composite ground wire) is drawn You may make it improve. In addition, by correcting interlaced scanning (the capture time is different between the odd and even lines of the moving image to be processed), the sharpness of the image is improved and the abnormal part is sharpened to detect the abnormality. The accuracy may be improved.

Example 1
Using the image in which the electric wire is actually captured by the video camera, the search area is expanded to the technique described in Patent Document 1 and the technique described in Patent Document 1 that do not have the variable search range and brightness pattern variable functions. Comparison of wire tracking performance in a program that adds a function (hereinafter referred to as a variable search range version) and a program that adds a function that refers to a brightness pattern (hereinafter referred to as a variable search range + variable reference brightness pattern version) An experiment was conducted.

  Table 1 shows the experimental results. In Table 1, there is a difference in the total number of processed images because, in the technique described in Patent Document 1, the tracking process is canceled due to the inability to track, and the total number of processed images is reduced in the search range variable version. Here, the tracking failure rate refers to the rate at which the position of the wire could not be specified, and the retracking success rate refers to the rate at which the wire tracking succeeded again after the tracking failure. Here, for the image where the background of the image is green such as forest and roads, rivers, private houses, factories, etc. are not reflected, there was no difference in the tracking performance between the technique described in Patent Document 1 and the two improved versions. . However, when there is a background including a gray portion (road, private house, factory, etc.) similar to an electric wire, the tracking failure rate is high in the technique described in Patent Document 1 or the search range variable version. In this case, the retracking rate was also low. On the other hand, the search range + reference brightness pattern variable version succeeded 100% in re-tracking in addition to the tracking failure rate of 1% or less per span. That is, it was found that the wire tracking capability could be greatly improved by the program with the search range variable function and the reference brightness pattern function added.

  From this experiment, it was found that the failure rate of 0.53% and 1% or less can be suppressed in the variable search range + variable reference brightness pattern version. In addition, when the background of a gray portion similar to an electric wire (private house, factory, etc.) exists, tracking is impossible with the technique described in Patent Document 1 and the search range variable version, and retracking sometimes fails. It was found that 100% re-tracking is possible with the variable + reference brightness pattern variable version. That is, since the tracking failure rate is less than 1% and the retracking success rate is 100%, it was found that the system can be monitored with almost no stoppage. As a result, the operator's intervention was reduced.

  Next, the image unrelated to the wire abnormality detection was removed using the time code. FIG. 34 shows an example of a time code input screen. FIGS. 35A to 35C show an example of removal of an image unrelated to electric wire abnormality detection. The upper part shows the time code, and the lower part shows the number of images processed so far. For example, FIG. 35A shows that the number of processed images is 28, and the time code is 22 which is 48 minutes and 30 seconds. That is, since 30 images are taken per second, this indicates the 22nd image after 48 minutes and 30 seconds. Further, FIG. 35 (c) shows that the time code is 48 minutes 49 seconds. That is, the seventh image of 48 minutes 49 seconds later is processed. Although the time code is displayed on the actual screen, the time code is shown again below each image in order to help understanding.

  In this experiment, in order to remove the image irrelevant to the wire abnormality detection, the time code of the start and end of each span is stored in the storage device at the time of inspection video shooting, and the shot image only during the designated period is stored. The detection process was performed for the electric wire or more. Specifically, the start and end time codes for each span are input on the input screen shown in FIG. 34, so that only the image between the specified time codes can be processed. I made it. For example, as shown in FIG. 35B, an image with a time code of 48 minutes 31 seconds to 48 minutes 48 seconds was a captured image of a steel tower portion. Abnormality detection processing was omitted. As a result, the amount of data to be processed is reduced, so that quick processing can be performed.

(Example 2)
Next, an application experiment was performed on an electric wire with a red line in which a red line for twist confirmation such as OPGW (optical fiber composite ground wire) was drawn. Since there is a big difference in the color distribution between the red line and the arc mark, this color information is used to reduce misjudgments. Specifically, an experiment was conducted on a technique for discriminating the OPGW twist confirmation red line and the arc mark. FIG. 36 shows the appearance frequency distribution of the luminance values of the red (Red), green (Green), and blue (Blue) components when there is a red line, and FIG. 37 shows the luminance values of the RGB components when there is an arc mark. The appearance frequency distribution is shown. Focusing on the R component of the color information value, the red line and the arc mark were distinguished.

  FIG. 38 shows an example of an image including a red line, and FIG. 39 shows an example of an image including an arc mark. First, paying attention to the R component of the electric wire, it was applied to an image in which an electric wire with a red line was photographed. Here, it is assumed that an arc mark is detected, and a case where the luminance value difference between the R component and the R component is 6 or more is erroneously detected by the red line. This value was experimentally obtained by applying to the electric wire with a red line in advance. When the color information of the electric wire is not used, the false detection rate for erroneously determining the red line as an arc mark was 35.0%. However, the false detection rate when the R component is used is 18.7%. It was found that it was effective to use the color information value in addition to the luminance value. FIG. 40A shows a case where determination is performed using only luminance values without using color information, and FIG. 40B shows an example of determination results when color information is used. If color information is not used, it is erroneously determined that there is an arc mark as shown in FIG. 40 (a), but it is correctly determined that there is a red line as shown in (b) by using the color information value for determination. I found out that This experiment reduces the false detection rate by applying color information values when detecting abnormalities in red-line electric wires with twisted red lines such as OPGW (optical fiber composite ground wire). As a result, it was found that an effective wire abnormality detection can be performed.

(Example 3)
Next, an experiment was conducted to evaluate the performance of this system. In the field of pattern identification, evaluation is usually made with the ability to detect abnormal points, but in the work of detecting electric wire abnormalities, it is necessary to reliably detect abnormalities. Therefore, the evaluation of this system is not an evaluation method adopted in normal pattern identification, but parameters are set so that 100% abnormality can be detected, and normal ones that are considered to increase with this are determined as abnormal. We decided to evaluate the ratio of misjudgment (false detection rate) as an index. By adopting this evaluation method, if the false detection rate is small, the time for final confirmation of abnormality detection can be reduced, so that the work efficiency can be evaluated.

  The resolution of the image developed on the system that can be configured with the electric wire abnormality detection method and low price is lower than that of Hi-Vision. Therefore, Table 2 shows the result of comparison with the inspection record in which the abnormality was examined with the image resolution targeted by this system.

  In the electric wire abnormality detection system of the technology described in Patent Document 1, only arc marks and wire breakage are targeted for detection, but in actual work, since dust and discoloration are also inspected, these are also detected. The parameters were adjusted. The false detection rate at which the person (worker) in Table 2 determined to be normal and this system determined to be abnormal was 43.5%. The technique described in Patent Document 1 is improved by about 26% compared with the error detection rate of about 70%. The reason for the improvement is considered to be that the tracking capability of the electric wire has been improved by this system and the electric wire can be recognized more accurately.

  Next, Table 3 shows a comparison result based on an inspection record with an image by Hi-Vision used in an actual electric wire inspection operation.

  There is a difference of 8 times between the target image resolution and the high-definition resolution in this system, and even if you cannot confirm dust and discoloration in the low-resolution image, you can confirm it with high-definition. Although the false positive rate is expected to increase when the parameter is set to, a comparative experiment was conducted to investigate the applicability of this system to inspection work. From the results of Table 3, it was found that the false detection rate only increased by about 30% due to the difference in resolution. As a result, it was found that there is a possibility that further improvement in work efficiency can be achieved by first selecting candidates for abnormal points in the preliminary inspection of this system and finally confirming only the problematic points with the HDTV system. .

  Also, one of the major problems at present is that minute dust and discoloration are also targeted for detection, so that it is difficult to distinguish them from shadows caused by twisting of the wires. FIG. 41 shows an example of an image in which it is difficult to distinguish between a twisted portion and an abnormal portion. Therefore, it is conceivable to recognize a shadow portion due to twisting, excluding the inspection portion, and referring to the wire template for the remaining portion.

  Here, the moving images processed by this system have different capture times for odd and even rows of the image (this is called interlaced scanning) and are captured at 1/60 second intervals. An example of interlaced scanning is shown in FIGS. For example, as shown in FIG. 42 (a), if a black line enters the screen for 1/30 second, it will be as shown in FIG. 42 (b) if the odd lines and even lines are captured at the same timing. Thus, an image close to the original straight line can be constructed. However, since the interlace scanning is actually performed, the data of the part that should be originally as shown in FIG. 42C is lost. In this case, if an image is formed in 1/30 seconds, an image is formed by a portion with data in odd rows and even rows, so the shape and color change from the original image as shown in FIG. End up.

  Therefore, at the stage of taking in the computer, as shown in FIG. 41, the image becomes a blurred image. However, in order to identify minute dust and discoloration, it is necessary to correct this, improve the sharpness of the abnormal part, and add processing for clarifying the difference between the abnormal part and the normal part. FIG. 43 shows an example of an image obtained as a result of correction excluding the influence of interlaced scanning. Thereby, it turned out that the twist of an electric wire becomes clear and can improve a detection accuracy.

It is a graph explaining the principle of the electric wire abnormality detection method of this invention, and displays the edge pixel of an electric wire and an ideal outline so as to overlap. The vertical axis of the graph indicates the vertical pixel number, and the horizontal axis indicates the horizontal pixel number. It is the graph which displayed the edge pixel of the electric wire, a vertical axis shows a vertical pixel number, and a horizontal axis shows a horizontal pixel number. It is the graph which displayed the ideal outline of the electric wire, a vertical axis shows a vertical pixel number, and a horizontal axis shows a horizontal pixel number. The image which becomes the origin of Drawing 1-Drawing 3 is shown, and the image of the electric wire which has raise | disconnected the strand is shown. It is a graph explaining the principle of the electric wire abnormality detection method of this invention, and represents the representative value and threshold value of the color information value of an electric wire. The vertical axis of the graph indicates the luminance value, and the horizontal axis indicates the horizontal pixel number. FIG. 5 shows the original image of FIG. 5 and shows an image of an electric wire with arc marks. The image of the electric wire for demonstrating the principle of edge detection is shown. It is a graph which shows the result of having calculated | required the difference value of the luminance value between two adjacent pixels above the perpendicular direction shown by the arrow of FIG. The vertical axis of the graph indicates the vertical pixel number of the lower pixel for which the difference value is obtained, and the horizontal axis indicates the magnitude of the difference value. FIG. 9 is an enlarged view of a part of FIG. It is a schematic block diagram which shows one Embodiment of the electric wire abnormality detection apparatus of this invention. It is a flowchart which shows an example of the process performed by execution of the electric wire abnormality detection method and apparatus of this invention, and a program. 12 is a flowchart illustrating an example of processing in which the target image cutout processing in FIG. 11 is detailed. It is a flowchart which shows an example of the process which detailed the electric wire abnormality detection process of FIG. It is a flowchart which shows an example of the process which detailed the edge detection process of FIG. It is a flowchart which shows an example of the process which detailed the search process of the edge pixel of FIG. It is a flowchart which shows an example of the process which detailed the ideal outline estimation process of FIG. FIG. 17 is a flowchart illustrating an example of a process in which the removal candidate pixel search process of FIG. 16 is detailed. It is a flowchart which shows an example of the process which detailed the shape abnormality detection process of FIG. 13, and shows the first half part of a process. It is a flowchart which shows the second half part of FIG. It is a flowchart which shows an example of the process which detailed the color abnormality detection process of FIG. 13, and shows the first half part of a process. It is a flowchart which shows the second half part of FIG. It is a flowchart which shows an example of the process which detailed the pre-process of FIG. It is a flowchart which shows an example of the movement estimation process of the electric wire of this invention. It is a flowchart which shows an example of the tracking success / failure determination process of the electric wire of this invention. It is a figure which shows the example of the screen which the electric wire abnormality detection apparatus of this invention displays. It is a figure which shows the other example of the screen which the electric wire abnormality detection apparatus of this invention displays. It is a figure which shows the other example of the screen which the electric wire abnormality detection apparatus of this invention displays. It is a conceptual diagram explaining the principle which searches an electric wire in the electric wire abnormality detection method of this invention. It is a conceptual diagram explaining the principle which searches an electric wire in the electric wire abnormality detection method of this invention. The electric wire has shown the image image | photographed by the edge part of the picked-up image. It is a figure which shows the search area | region of the electric wire of a prior art. It is a figure which shows the process which determines a search area | region from the midpoint of the electric wire of this invention. It is a figure which shows an example of the tracking success / failure determination process of the electric wire of this invention, (a) is an example of the figure which shows the case where the search range is expanded 3 times, (b) is 7 times the search range. It is an example of the figure which shows the case where it expanded, (c) is an example of the figure which shows the case where the search range is expanded 15 times. It is an example of the time code input screen of this invention. It is an image which shows the screen which the electric wire abnormality detection apparatus of this invention displays, (a) shows the image in time code 48 minutes 31 seconds 22 time, (b) shows the image unrelated to electric wire detection, (c ) Shows an image at the time code 48 minutes 49 seconds 7. It is an example of the graph which shows the luminance distribution of the electric wire with a red line. It is an example of the graph which shows the luminance distribution of the electric wire with an arc mark. It is an example of the image of the electric wire containing a red line. It is an example of the image of the electric wire containing an arc mark. (A) is a graph which shows an example of the result of having identified the red line only with the luminance value. (B) is a graph which shows an example of the result of having identified the red line using the color information value. It is an example of an image in which it is difficult to distinguish between a twisted portion and an abnormal portion when detecting an abnormality of an electric wire. It is a figure which shows an interlace scanning, (a) shows the figure when a black line enters in 1/30 second, (b) shows the taking-in method when not performing interlace scanning, (c) shows interlace scanning. A capturing method in the case of performing scanning is shown, and (d) shows a result of performing interlaced scanning. FIG. 42 is an example of an image after correcting interlaced scanning for the image of FIG. 41. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Original image 2 Template 3 Search area 10 Electric wire abnormality detection apparatus

Claims (12)

  A target image cutout process for cutting out a portion where the electric wire is shot from the original image with respect to the original image where the electric wire is shot, and an edge detection process for the target image cut out by the cutout process of the target image; An ideal outline estimation process, a wire abnormality detection process that performs a shape abnormality detection process or a color abnormality detection process, and the target image cut-out process sets a search area in advance for the original image, The image of the electric wire is stored in advance in a storage device as a template, the region most similar to the template is searched in the search region to estimate the moving position of the electric wire, and the electric wire photographed in the immediately preceding frame image Compare the slope with the search result, and if the difference in slope is greater than or equal to a preset threshold value, determine that the wire search has failed and expand the search area , Wire abnormality detection method characterized by estimating the movement position of the wire by searching a region most similar again the template in said search area.   A target image cutout process for cutting out a portion where the electric wire is shot from the original image with respect to the original image where the electric wire is shot, and an edge detection process for the target image cut out by the cutout process of the target image; An ideal outline estimation process, a wire abnormality detection process that performs a shape abnormality detection process or a color abnormality detection process, and the target image cut-out process sets a search area in advance for the original image, The image of the electric wire is stored in advance in a storage device as a template, the region most similar to the template is searched in the search region to estimate the moving position of the electric wire, and the electric wire photographed in the immediately preceding frame image The inclination of the search result is compared, and if the difference in inclination is equal to or greater than a preset threshold value, it is determined that the electric wire search has failed, and another electric power stored in advance is determined. Wire abnormality detection method characterized by referring to the template, to explore the region most similar to again the template by the search region.   The wire abnormality detection according to claim 1, wherein the template is stored in a main storage device when the target image is cut out, and the template is automatically updated every certain frame. Method.   The search area is set in advance for the image of the first frame of the original image in which the wire is photographed. From the processing of the image of the next frame, the midpoint of the search result of the wire in the immediately preceding frame is set. The wire abnormality detection method according to claim 1 or 2, wherein is a center position of the search area.   An image irrelevant to the detection of the electric wire is removed by designating a time code in advance from the original image taken of the electric wire, and the target image is cut out after removing the image irrelevant to the detection of the electric wire. The electric wire abnormality detection method according to claim 1 or 2.   A target image cut-out processing unit that cuts out a portion where the electric wire is shot from the original image, and an edge detection process for the target image cut out by the target image cut-out process A wire abnormality detection processing unit that performs ideal contour estimation processing, shape abnormality detection processing, or color abnormality detection processing, and the target image cut-out processing unit sets a search area for the original image in advance. The image of the electric wire to be searched is previously stored in the storage device as a template, the region most similar to the template is searched in the search region, the moving position of the electric wire is estimated, and the frame image immediately before is captured. The search area is compared, and when the difference in inclination is equal to or greater than a preset threshold, it is determined that the search for the wire has failed, and the search area Expanded, wire abnormality detecting apparatus is intended to estimate the moving position of the wire by searching a region most similar again the template in said search area.   A target image cut-out processing unit that cuts out a portion where the electric wire is shot from the original image, and an edge detection process for the target image cut out by the target image cut-out process A wire abnormality detection processing unit that performs ideal contour estimation processing, shape abnormality detection processing, or color abnormality detection processing, and the target image cut-out processing unit sets a search area for the original image in advance. The image of the electric wire to be searched is previously stored in the storage device as a template, the region most similar to the template is searched in the search region, the moving position of the electric wire is estimated, and the frame image immediately before is captured. The inclination of the electric wire and the search result was compared, and when the difference in inclination was equal to or greater than a preset threshold, it was determined that the electric wire search failed and stored in advance. Of with reference to the wires of the template, the wire abnormality detecting apparatus is intended to explore the area that is most similar again the template by the search region.   The wire abnormality detection device according to claim 6 or 7, wherein the template is stored in a main storage device when the target image is cut out, and the template is automatically updated every certain frame.   The search area is set in advance for the image of the first frame of the original image in which the wire is photographed. From the processing of the image of the next frame, the midpoint of the search result of the wire in the immediately preceding frame is set. The wire abnormality detection device according to claim 6 or 7, wherein is a center position of the search region.   An image irrelevant to the detection of the electric wire is removed by designating a time code in advance from the original image taken of the electric wire, and the target image is cut out after removing the image irrelevant to the detection of the electric wire. The wire abnormality detection device according to claim 6 or 7, wherein the wire abnormality detection device is a device.   The computer sets a search area in advance for the original image in which the electric wire is photographed, stores the electric wire image to be searched as a template in a storage device in advance, and an area most similar to the template is stored in the search area. Search and estimate the moving position of the electric wire, compare the inclination of the electric wire captured in the previous frame image with the search result, and search for the electric wire when the difference in inclination is greater than or equal to a preset threshold value The part where the electric wire is photographed from the original image by determining that it has failed, enlarging the search area, searching again the area most similar to the template in the search area and estimating the moving position of the electric wire A target image cut-out means for cutting out the target image, edge detection processing, ideal contour estimation processing, and shape abnormality detection for the target image cut out by the target image cut-out processing. Wire abnormality detection program for causing run as wire abnormality detection processing means for performing the processing or color abnormality detection processing. The computer sets a search area in advance for the original image in which the electric wire is photographed, stores the electric wire image to be searched as a template in a storage device in advance, and an area most similar to the template is stored in the search area. Search and estimate the moving position of the electric wire, compare the inclination of the electric wire captured in the previous frame image with the search result, and search for the electric wire when the difference in inclination is greater than or equal to a preset threshold value It judges that it failed, refers to the template of the other electric wire memorized beforehand, searches the region most similar to the template again in the search region, estimates the movement position of the electric wire, and uses the electric wire from the original image A target image cutout unit that cuts out a portion where the image is captured, an edge detection process and an ideal contour estimation process for the target image cut out by the target image cutout process , As well as wire abnormality detection program for causing run as wire abnormality detection processing means for performing a shape abnormality detection processing or color abnormality detection processing.