JP7264428B2 - Road sign recognition device and its program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Law (1) Proceedings of the 2018 Tohoku Section Joint Conference of the Institutes of Electrical and Related Engineers, Tohoku Section Joint Conference of the Institutes of Electrical and Related Engineers, September 6, 2018

Application of Article 30, Paragraph 2 of the Patent Law (2) Lecture on September 6, 2018 at the 2018 Tohoku Section Joint Conference of the Institutes of Electrical and Related Engineers, Ueda Campus, Iwate University

Application of Article 30, Paragraph 2 of the Patent Law (3) The 6th IIAE International Conference on Intelligent Systems and Image Processing 2018 Paper submission system (website), https://www2. ia-engineers. org/conference/index. php/icisip/icisip2018/, posted on September 10, 2018

Application of Article 30, Paragraph 2 of the Patent Act (4) The 6th IIAE International Conference on Intelligent Systems and Image Processing 2018, Shimane Prefectural Industrial Exchange Hall (Kunibiki Messe), September 13, 2018

Application of Article 30, Paragraph 2 of the Patent Law (5) Japan Society of Applied Engineering, National Conference 2018 Paper submission system (website), https://www2. ia-engineers. org/conference/index. php/iaeac/iiae2018, posted on September 13, 2018

Application of Article 30, Paragraph 2 of the Patent Law (6) The Japan Society for Industrial Applied Engineering National Convention 2018, Shimane Prefectural Industrial Exchange Hall (Kunibiki Messe), September 13, 2018 Lecture

Application of Article 30, Paragraph 2 of the Patent Law (7) 2018 Information Processing Society of Japan Tohoku Branch Research Meeting (Akita University), Open University of Japan Akita Learning Center, November 29, 2018 Lecture

The present invention relates to a road sign recognition device and the like for recognizing road signs. In particular, the present invention is suitable for recognizing road signs at night and road signs that have faded over time.

A road sign is a display board that is installed on the side or above a road and provides information necessary for road traffic to vehicle drivers, pedestrians, and the like. An example of a road sign is a circular speed limit sign. Since the maximum speed of a vehicle (also referred to as speed limit or speed limit) varies depending on the road on which it is traveling, the vehicle driver needs to drive the vehicle while constantly checking the maximum speed. If a vehicle driver misses a speed limit sign, there is a risk of an accident resulting from overspeeding. Therefore, in recent years, driving support systems that automatically recognize speed signs and present the recognition result to the vehicle driver have been developed and put on the market.

However, some commercially available driving support systems do not operate normally in recognizing road signs at night or road signs that have faded due to age deterioration. Especially at night, there is a problem that the rate of fatal accidents due to speeding violations is extremely high, and there is a demand for a driving support system that can robustly recognize road signs at night.

Patent Literature 1 discloses a road sign recognition device for vehicles intended to reliably recognize the type and content of road signs that have deteriorated over time in a dark environment such as at night or in a tunnel. The vehicle road sign recognition device described in Patent Document 1 has an imaging unit that has sensitivity in the visible light region and the near infrared region, and a near infrared light irradiation unit that irradiates near infrared light. Determines whether or not a road sign is captured based on the luminance distribution of the captured image. When it is determined that a road sign is captured, the contents of the road sign are recognized. As a result, even when the brightness of the road sign has decreased due to deterioration over time, an image having recognizable contrast can be captured.

JP 2016-196233 A

Mikio Takagi, Haruhisa Shimoda (supervision), "New Image Analysis Handbook", University of Tokyo Press, 2004. Shinichi Murakami, "Image Processing Engineering", Tokyo Denki University Press, 1996.

However, the vehicle road sign recognition device described in Patent Document 1 requires special components such as an imaging unit that has sensitivity in the near-infrared region and a near-infrared light irradiation unit that irradiates near-infrared light. . In addition, since it is necessary to repeatedly pick up images in a dark environment, it is presumed that there may be a delay in recognizing road signs.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems. , to provide a road sign recognition device or the like capable of robust recognition.

A first invention for achieving the above object is a road sign recognition device for recognizing a road sign, comprising an image input unit for inputting a photographed image, detecting the road sign included in the photographed image, an extracted image generation unit that extracts a part of the photographed image and generates an extracted image; a binarized image generation unit that binarizes the extracted image and generates a binarized image; a content recognition unit that recognizes the content of the road sign, wherein the binarized image generation unit determines a binarization threshold value based on a histogram of brightness values of pixels included in the central portion of the extracted image. and the road sign is a speed limit sign indicating a maximum speed, the number of the speed limit sign is composed of a characteristic number characterizing the speed limit sign and a common number common to the speed limit signs, and the content recognition is performed. The unit determines division positions based on the peripheral distribution in the vertical direction of the binarized image, and divides the binarized image at the division positions to obtain characteristic numeral images including the characteristic numerals and the common numerals. and the content recognition unit executes template matching processing between a plurality of common numeral template images having different image sizes and the common numeral image, and the position of the common numeral in the common numeral image and size, and the content recognition unit estimates a characteristic numeral presence region in which the characteristic numeral exists in the characteristic numeral image based on the position and size of the common numeral, and identifies a plurality of characteristics having different numerals. The road sign recognition apparatus is characterized by executing a template matching process between a number template image and the characteristic number existing area . According to the first invention, even a road sign whose color has faded due to deterioration over time can be robustly recognized without requiring special components. In addition, it is possible to accurately divide the characteristic numerals and the common numerals, to accurately specify the position and size of the common numerals, to accurately specify the characteristic numerals, and to accurately recognize the maximum speed sign. can be done.

A second aspect of the invention is a road sign recognition device for recognizing road signs, comprising an image input unit for inputting a captured image, detecting the road sign included in the captured image, and extracting a part of the captured image. an extracted image generation unit for generating an extracted image; a binarized image generation unit for binarizing the extracted image to generate a binarized image; and recognizing the content of the road sign from the binarized image. a content recognition unit, wherein the binarized image generation unit determines a binarization threshold value based on a histogram of brightness values of pixels included in the central portion of the extracted image; A road in which the photographed images of a plurality of frames are input in chronological order, and the position of the road sign is tracked in subsequent photographed images of frames subsequent to the photographed image in which the content of the road sign is recognized by the content recognition unit. a sign tracker, wherein the road sign tracker tracks the position of the road sign by applying a particle filter in the subsequent captured image, each particle corresponding to a symbol of the road sign. Calculate three likelihoods of a first region, a second region corresponding to the background of the road sign, and a third region corresponding to a portion excluding the first region and the second region, and weight the likelihoods A road sign recognition device characterized by estimating the center coordinates of a road sign based on . According to the second invention, even a road sign whose color has faded due to deterioration over time can be robustly recognized without requiring special components. Also, road signs can be accurately tracked.

A third aspect of the invention is a program for causing a computer to function as a road sign recognition device for recognizing road signs, comprising: an image input unit for inputting a photographed image; a binarized image generation unit for generating an extracted image by extracting the portion including the a content recognition unit for recognizing content, wherein the binarized image generation unit determines a binarization threshold value based on a histogram of brightness values of pixels included in a central portion of the extracted image, and The sign is a speed limit sign indicating the maximum speed, the number of the speed limit sign is composed of a characteristic number characterizing the speed limit sign and a common number common to the speed limit sign, and the content recognition unit is configured to: dividing positions are determined based on the peripheral distribution in the vertical direction of the binarized image, and by dividing the binarized image at the dividing positions, the characteristic numeral image including the characteristic numeral and the common numeral including the common numeral generates an image, and the content recognition unit executes template matching processing between a plurality of common number template images having different image sizes and the common number image, and determines the position and size of the common number in the common number image. The content recognizing unit estimates a characteristic numeral existence region in which the characteristic numeral exists in the characteristic numeral image based on the position and size of the common numeral, and a plurality of characteristic numeral template images having different numerals. and a program for functioning as a road sign recognition device that executes template matching processing for the characteristic numeral existing area. By installing the third invention in a general-purpose computer, the road sign recognition device of the first invention can be obtained.

A fourth aspect of the invention is a program for causing a computer to function as a road sign recognition device for recognizing road signs, comprising: an image input unit for inputting a photographed image; a binarized image generation unit for generating an extracted image by extracting the portion including the a content recognition unit for recognizing content, wherein the binarized image generation unit determines a threshold for binarization based on a histogram of brightness values of pixels included in a central portion of the extracted image, and The input unit inputs the photographed images of a plurality of frames in chronological order, and the position of the road sign in subsequent photographed images of frames subsequent to the photographed image in which the content of the road sign is recognized by the content recognition unit. , wherein the road sign tracking unit tracks the position of the road sign by applying a particle filter in the subsequent captured image, wherein for each particle the road sign Calculating three likelihoods of a first region corresponding to a symbol, a second region corresponding to the background of the road sign, and a third region corresponding to a portion excluding the first region and the second region, A program for functioning as a road sign recognition device for estimating central coordinates of road signs based on likelihood weighting. By installing the fourth invention in a general-purpose computer, the road sign recognition device of the second invention can be obtained.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide a road sign recognition device or the like capable of robustly recognizing even a road sign at night or a road sign that has faded due to deterioration over time without requiring special components.

Diagram showing the outline of the road sign recognition device Flowchart showing the flow of road sign recognition processing for speed limit signs An example of a speed limit sign, an example of a photographed image, and a drawing substitute photograph showing an example of execution of preprocessing Drawing substitute photo showing an execution example of edge extraction processing Drawing-substituting photograph showing an execution example of road sign detection processing Drawing substitute photograph showing an example of an extracted image and a binarized image Drawing substitute photo for explaining the binarized image generator Drawing-substituting photograph for explaining the binarized image segmentation process A drawing substitute photograph explaining the template matching process of common numbers A drawing substitute photograph explaining the template matching process of characteristic numbers Flowchart showing the flow of road sign tracking processing Drawing substitute photo showing an example of particle scattering Drawing-substituting photograph for explaining extraction processing of the first region and the second region Diagram for explaining likelihood calculation processing Flowchart showing the flow of road sign recognition processing for no vehicle entry signs Drawing-substituting photograph showing an execution example of road sign recognition processing for no-vehicle entry signs

FIG. 1 is a diagram showing an outline of a road sign recognition device. The road sign recognition device 1 is a device mounted on the vehicle 100, and configured as part of a driving support system or a car navigation system of the vehicle 100, for example. The road sign recognition device 1 is connected to a photographing device 2 whose photographing direction is in front of the traveling direction of the vehicle 100, and an output device 3 such as a display and a speaker. ) 4.

The road sign recognition device 1 includes a CPU (abbreviation for "Central Processing Unit") as a control unit, a memory as a main storage unit, and hardware such as an HDD (abbreviation for "Hard Disk Drive") and flash memory as an auxiliary storage unit. have clothing. The auxiliary storage stores an OS (abbreviation for “Operating System”), application programs, data necessary for processing, and the like. The control unit of the road sign recognition device 1 reads out the OS and application programs from the auxiliary storage unit, stores them in the main storage unit, controls other devices while accessing the main storage unit, and executes processing described later.

The road sign recognition device 1 includes an image input unit 11 that inputs a photographed image from the photographing device 2, and an extracted image generator that detects a road sign included in the photographed image, extracts a part of the photographed image, and generates an extracted image. a binarized image generation unit 13 that binarizes the extracted image to generate a binarized image; a content recognition unit 14 that recognizes the content of the road sign from the binarized image; and a road sign tracking unit 15 for tracking the position of the road sign in a subsequent captured image of a frame after the captured image in which the content of the road sign is recognized. The road sign recognition device 1 outputs the recognition result by the content recognition unit 14 and the tracking result by the road sign tracking unit 15 to the output device 3, the ECU 4, and the like.

The photographing device 2 photographs moving image data whose photographing direction is in front of the traveling direction of the vehicle 100 and outputs the moving image data to the road sign recognition device 1 . A captured image is a still image of a frame included in moving image data. The image input unit 11 of the road sign recognition device 1 inputs the photographed images of a plurality of frames in chronological order, and executes road sign recognition processing and road sign tracking processing, which will be described later.

FIG. 2 is a flow chart showing the flow of road sign recognition processing for speed limit signs. In the processing shown in FIG. 2, the road sign recognition device 1 processes a single captured image and recognizes the content of the speed limit sign included in the captured image.

As shown in FIG. 2, when the image input unit 11 of the road sign recognition device 1 inputs a photographed image from the photographing device 2 (step S1), the extraction image generation unit 12 sets the target range of the processing described later (step S2), a predetermined preprocessing is executed (step S3).

FIG. 3 shows an example of a speed limit sign, an example of a photographed image, and a drawing substitute photograph showing an example of execution of preprocessing. FIG. 3(a) shows a speed limit sign 20, FIG. 3(b) shows a photographed image 21, and FIG. 3(c) shows an image 22 after preprocessing.

As shown in FIG. 3A, the speed limit sign 20 has a background 20a colored white, a donut-shaped rim 20b colored red, and positioned inside the rim 20b and colored blue. and numerals 20c and 20d indicating the maximum speed of the road at the installation location. In the process shown in FIG. 2, the maximum speed indicator 20 indicates the maximum speed as a two-digit number and a multiple of ten. In this case, the number 20d of the first digit is commonly "0" in all the speed limit signs 20, and is defined as a common number. The second digit number 20c is a number that characterizes the speed limit sign 20 and is defined as a characteristic number. Characteristic numerals are, for example, "3", "4", "5", "8" and the like. In other words, in the present embodiment, the numbers on the speed limit sign 20 are composed of characteristic numbers that characterize the speed limit sign 20 and common numbers that are common to the speed limit signs 20 .

As shown in FIG. 3B, the photographed image 21 has a left-right direction substantially parallel to the horizontal direction and an up-down direction substantially parallel to the vertical direction. The number of pixels of the captured image 21 is 640 pixels in the horizontal direction and 480 pixels in the vertical direction.

In order to shorten the processing time, the extracted image generation unit 12 uses a straight line extending in the left-right direction as a dividing line, sets the upper 75% as the target range for the processing described below, and excludes the lower 25% for the processing described later. . Therefore, the number of pixels in the target range is 640 pixels in the horizontal direction and 360 pixels in the vertical direction.

Next, the extracted image generation unit 12 sets a 2×2 size filter to smooth the image, and executes coarse-graining processing in which the average value of this filter is used as a new pixel. As a result, the number of pixels in the target range is 320 pixels in the horizontal direction and 180 pixels in the vertical direction.

Next, the extraction image generation unit 12 executes gamma correction processing (see Non-Patent Document 1) in order to brighten the luminance of the entire image. This improves the accuracy of road sign recognition in dark environments.

Next, the extraction image generation unit 12 performs smoothing processing (see Non-Patent Document 1) to remove fine noise, and uniforms the color information of the sky. The result is the preprocessed image 22 shown in FIG. 3(c). Compared with the photographed image 21, in the post-preprocessing image 22, the road sign is visible in the lower left, and there is little fine noise.

Returning to the description of FIG. Next, the extraction image generation unit 12 executes edge extraction processing to generate an edge extraction image (step S4). In the present embodiment, the extracted image generator 12 applies a plurality of edge detection filters to the preprocessed image 22 and calculates the logical sum to generate an edge extracted image.

FIG. 4 is a drawing substitute photograph showing an execution example of edge extraction processing. 4(a) is an image 23 after applying the Laplacian filter to the preprocessed image 22, FIG. 4(b) is an image 24 after applying the Canny edge filter to the preprocessed image 22, and FIG. 4(c) is an image 25 obtained by synthesizing the image 23 and the image 24 by logical sum, and FIG.

As described above, the peripheral edge 20b of the speed limit sign 20 is colored red, but the color information is greatly lost in a dark environment. Therefore, the extraction image generation unit 12 combines the result of applying a Laplacian filter (see Non-Patent Document 1) that strongly reacts to red in a bright environment and the Canny edge filter (Non-Patent Document 1) that has less undetected contours in a dark environment. ) are combined by disjunction. This improves the accuracy of road sign recognition in dark environments. However, in the image 24 shown in FIG. 4B, the threshold value of the Canny edge filter is set low in order to improve the accuracy of road sign recognition in a dark environment. are also often detected as edges. Similarly, a large amount of noise remains in the image 25 shown in FIG. 4(c).

The extraction image generation unit 12 performs noise removal processing based on features of objects and scenery other than the road signs in order to remove noise of edges other than the contours of the road signs. Specifically, the extracted image generation unit 12 determines (first edge noise determination condition) pixels having a brightness value of less than 10 seen in an area such as the sky, (second edge noise determination condition) utility poles, buildings, Pixels satisfying any of the following three conditions: linear pixels with a connection number of 25 or more, and (third edge noise determination condition) single-pixel or two-pixel-connected isolated point pixels seen on signboards, etc., are removed as noise. . The result is the image 26 shown in FIG. 4(d). In the image 26, the road signs to be extracted appear as circles in the lower left part. Below, the image 26 is assumed to be an edge-extracted image.

Returning to the description of FIG. Next, the extraction image generator 12 detects road signs from the edge extraction image and generates an extraction image (step S5). In the present embodiment, the extracted image generation unit 12 removes noise from the edge extracted image based on the outline feature of the road sign (=the circular shape for the maximum speed sign) to detect the road sign. It should be noted that the road sign detection processing in the present embodiment is applicable not only to speed limit signs but also to circular road signs.

FIG. 5 is a drawing-substituting photograph showing an execution example of road sign detection processing. FIG. 5(a) is an image 27 showing the circle detection result by the generalized Hough transform, and FIG. 5(b) is an image 28 after removing noise from the circle detection result shown in the image 27. FIG.

As shown in FIG. 3(a), since the contour of the speed limit sign is circular, the extracted image generation unit 12 detects the speed limit sign candidate using the generalized Hough transform useful for circle detection. The generalized Hough transform is a method that transforms the coordinates in the image into another parameter space and detects a figure in that space. , is useful for detecting contours of speed limit signs (see Non-Patent Document 1 for details). The extracted image generation unit 12 detects circles composed of 6 to 22 pixels with the number of votes by the generalized Hough transform as candidates for the maximum speed sign. The result is the image 27 shown in FIG. 5(a).

Areas marked with A to K in FIG. 5(a) are detected as circles. A and B are circular road signs. However, as in C to K, edges extracted by the influence of signboards, street lights, vehicle headlights, etc., besides circular road signs, become noise and are detected as circles. In the present embodiment, the extracted image generation unit 12 focuses on the characteristics of circular road signs, regards circles that satisfy any of the following determination conditions as noise, and removes them from the maximum speed sign candidates.

(First Circle Noise Judgment Condition) The extracted image generating section 12 judges that a portion of a circle passing outside the image is noise and removes it. It should be noted that there may be a case where the circle constituting the road sign passes outside the image, but it is excluded in that frame and detection is attempted in the next frame and thereafter.

(Second condition for judging circular noise) A circular road sign appears as a small circle at the bottom of the image. Therefore, among the circles passing through the lower third of the image, the extracted image generation unit 12 determines that the circles having a radius of 10 pixels or more are noise and removes them.

(Third Circle Noise Judgment Condition) A circular road sign appears as a large circle in the upper part of the image. Therefore, among the circles passing through the upper third of the image, the extracted image generation unit 12 determines that circles with a radius of 10 pixels or less are noise and removes them.

(Fourth Circle Noise Judgment Condition) A circular road sign is characterized by a high density of the number of votes forming the circle. However, since the estimated circles are sorted by the number of votes, even if the density of the number of votes that make up the circle is low, noise with a large radius is detected as a circle with a higher rank compared to circular road signs. Therefore, the extracted image generation unit 12 determines that a circle with the number of votes of 60% or less of the circumference is noise and removes it.

(Fifth Circle Noise Judgment Condition) Since the generalized Hough transform responds to white pixels, countless circles are detected when a white region is input. On the other hand, in the contour detection image, there are few white pixels inside the circular road sign. Therefore, the extracted image generation unit 12 determines that a circle in which 70% or more of the pixels in the circle are white pixels is noise, and removes the circle.

(Sixth Circular Noise Judgment Condition) Circular road signs are mostly included in the top 15 ranks when sorted by the number of votes. Therefore, the extracted image generation unit 12 determines that circles ranked 16th and below are noise and removes them.

The result after removing the noise is the image 28 shown in FIG. 5(b). In comparison to image 27, in image 28 only circles A and B remain as possible speed limit signs. Circle A is a speed limit sign, while circle B is a circular road sign that is different from the speed limit sign. At this stage, it is sufficient if circular road signs can be detected without being limited to speed limit signs. The extracted image generation unit 12 uses the circles A and B in the image 28 as candidates for the maximum speed sign, projects a rectangular area including each circle onto the captured image 21, extracts the area, and generates an extracted image.

Returning to the description of FIG. Next, the binarized image generator 13 binarizes the extracted image to generate a binarized image (step S6). The processing in step S6 targets all extracted images. In the present embodiment, the binarized image generation unit 13 determines the threshold for binarization based on a histogram of brightness values of pixels included in the central portion of the extracted image.

FIG. 6 is a drawing substitute photograph showing an example of an extracted image and a binarized image. 6(a) is an extracted image 29 after preprocessing, and FIG. 6(b) is a binarized image 30 obtained by binarizing the extracted image 29. FIG. The binarized image generation unit 13 executes gamma correction processing (see Non-Patent Document 1) with a gamma value of 2.0 in order to brighten the luminance of the entire image. Next, the binarized image generator 13 expands the image size to 100 pixels in height while maintaining the aspect ratio using the bilinear interpolation method (see Non-Patent Document 1). FIG. 6(a) is an extracted image 29 after performing these preprocessings.

Next, the binarized image generation unit 13 divides the extracted image 29 into 5×5, and divides the center portion 29a (see FIG. 6A), which is a square in the center in both the left-right direction and the up-down direction. A threshold for binarization is determined using Otsu's discriminant analysis method (see Non-Patent Document 1) for only pixels, and the entire extracted image 29 is binarized. Specifically, the binarized image generator 13 binarizes the extracted image 29 using a threshold that maximizes the degree of separation of the histogram of the brightness values of the central portion 29a. A binarized image 30 in FIG. 6B is the result of binarizing the extracted image 29 by executing this process. In the binarized image 30, the two numbers "4" and "0" indicating the maximum speed can be clearly recognized as a set of white pixels.

Here, the central portion of the extracted image is a region having a certain area that includes at least the central point of the extracted image, includes a portion of the number indicating the maximum speed, and is a donut-shaped area colored red. A region that does not include the perimeter is desirable. For example, the central portion of the extracted image is not limited to the central square when divided into 5×5 as described above, but may be the central square when divided into 3×3, 3×5, 5×7, or the like. can be Further, for example, the shape of the central portion of the extracted image is not limited to a quadrangle, and may be a polygon other than a quadrangle, a circle, or an ellipse.

FIG. 7 is a drawing substitute photograph for explaining the binarized image generation unit. With reference to FIG. 7, the reason why the binarized image generation unit 13 according to the present embodiment limits the pixels used for determining the binarization threshold to the central portion of the image will be described. FIG. 7(a) is an extracted image 31 of a speed limit sign that has not faded due to aging, FIG. 7(b) is an extracted image 32 of a speed limit sign that has faded due to aging, and FIG. 7(c) is a binarized image. FIG. 7D shows a binarized image 33 in which the pixels used for determining the threshold value are the entire extracted image 32, and FIG. It is a binarized image 34 .

In the extracted image 31, the brightness value of the red donut-shaped periphery pixel 31a is "47", and the brightness value of the background pixel 31b inside the periphery is "84". On the other hand, in the extracted image 32, the brightness value of the peripheral pixel 32a is "99", and the brightness value of the background pixel 32b inside the peripheral portion is "82". In this way, the magnitude relationship between the brightness values of the peripheral portion and the background is reversed depending on the degree of deterioration over time. Assuming that the pixels used to determine the binarization threshold for the extracted image 32 are the entire image, as shown in FIG. distorted image 33 is generated. On the other hand, if the pixels used for determining the binarization threshold for the extracted image 32 are limited to the central portion of the image as in the present embodiment, as shown in FIG. A binarized image 34 whose digits are recognizable as a set of white pixels is generated. In this way, by limiting the pixels used for determining the binarization threshold to the central portion of the image, it is possible to recognize even the speed limit sign, which has faded due to deterioration over time.

Returning to the description of FIG. Next, the content recognition unit 14 divides the binarized image into left and right to generate a characteristic numeral image containing characteristic numerals and a common numeral image containing common numerals (step S7). The processing in step S7 targets all binarized images. In the present embodiment, the content recognition unit 14 determines the division position based on the vertical marginal distribution of the binarized image, and divides the binarized image into left and right at the division position to obtain the characteristic numeral image and the Generate a common digit image.

FIG. 8 is a drawing substitute photograph for explaining the binarized image division processing. 8(a) is a binarized image 30 to be processed, FIG. 8(b) is a mask image 35, and FIG. 8(c) is a binarized image 36 obtained by synthesizing the binarized image 30 and mask image 35, and The marginal distribution graph 37 and FIG. 8(d) are the characteristic numeral image 42 and the common numeral image 43 generated by dividing the binarized image 36. FIG.

In step S5, since a rectangular area including a circle is generated as an extracted image, the area outside the circle is included. The area outside the circle becomes noise in the processing described later. Therefore, in order to eliminate the influence of noise, the content recognition unit 14 performs mask processing (see Non-Patent Document 1) on the binarized image 30 using a mask image 35 to generate a binarized image 36. do. The mask image 35 has white pixels inside and black pixels outside the outline of the speed limit sign.

Next, the content recognition unit 14 determines the division position based on the peripheral distribution of the binarized image 36 in the vertical direction, and divides the binarized image 36 into two parts, left and right, at the division position to obtain a characteristic numeral image. and generate a common digit image. Here, the peripheral distribution of the binarized image 36 in the vertical direction is the number obtained by counting one pixel value (=black pixel or white pixel) of the binarized image 36 along the vertical direction at each coordinate in the horizontal direction. is the distribution of (see Non-Patent Document 2). In the marginal distribution graph 37 of FIG. 8(c), the black portion corresponds to the number of black pixels, and the white portion corresponds to the number of white pixels. In the present embodiment, the content recognition unit 14 detects black pixels (corresponding to the background of the speed limit sign) within the range of the central portion when the binarized image 36 is divided into three equal parts by lines extending in the vertical direction. The division position is the position where the number of pixels is the maximum.

In FIG. 8(c), a dotted line 38 indicates a line that divides the binarized image 36 into two halves, and a dotted line 39 indicates a line that passes through the dividing position. If the binarized image 36 is simply divided into two halves, it may be divided at positions passing through the characteristic numerals or common numerals, as indicated by the dotted line 38 . On the other hand, in this embodiment, division can be made at positions passing through only the background located between the characteristic number and the common number. Therefore, as shown in FIG. 8(d), in the present embodiment, white pixels corresponding to the characteristic numeral 40 exist only in the characteristic numeral image 42, and white pixels corresponding to the common numeral 41 exist only in the common numeral image 43. , and the characteristic numeral 40 and the common numeral 41 can be accurately divided even if the sign is not photographed from the front.

Returning to the description of FIG. Next, the content recognition unit 14 executes template matching processing of the common numeral image to identify the position and size of the common numeral (step S8). The processing in step S8 targets all common numeral images. In the present embodiment, the content recognition unit 14 executes template matching processing between a plurality of common number template images having different image sizes and the common number image to specify the position and size of the common number. This makes it possible to accurately specify the position and size of the common numeral regardless of the size of the road sign included in the captured image.

FIG. 9 is a drawing-substituting photograph for explaining the template matching process for common numerals. FIG. 9(a) relates to explanation of the template matching process, and FIG. 9(b) relates to explanation of specifying the position and size of the common number. The template matching process (see Non-Patent Document 1) is a process of moving a template image within a search range to search for the best matching location, that is, the location where the degree of similarity is maximized. In the present embodiment, the content recognition unit 14 uses the template matching process of the image correlation method (see Non-Patent Document 1) in which the correlation coefficient is used as the degree of similarity. However, the present invention is not limited to the template matching processing of the image correlation method, and can be appropriately selected from known template matching processing.

As shown in FIG. 9A, the content recognition unit 14 stores, for example, a common numeral template image 44 having an image size of 100 in height in advance, and uses the bilinear interpolation method (see Non-Patent Document 1). is used to reduce the height from 100 to 24 in steps of 2, and template matching processing is performed for each image size. As a result, the template matching process is executed 39 times, and the same number of locations on the common numeral image 43 where the degree of similarity is maximized are obtained. In FIG. 9A, there is only one common numeral image 43, but if there are multiple maximum speed sign candidates as in the example of FIG. will do. The content recognition unit 14 performs template matching processing on all common numeral images 43 . For example, if there are two common numeral images 43, the template matching process is executed 2×39=78 times, and the same number of locations on the common numeral image 43 with the maximum degree of similarity are obtained. If the maximum degree of similarity among these is equal to or greater than the threshold, the content recognition unit 14 determines that the original photographed image includes a maximum speed sign, and executes processing described later.

In FIG. 9A, the 27×46 common numeral template image 44a has the maximum similarity of 0.92. As shown in FIG. 9B, the content recognition unit 14 determines the image size of the common number template image 44a, that is, the width w (=horizontal length) and height h (=vertical length). get. The content recognition unit 14 also acquires the coordinates of the location on the common numeral image 43 where the degree of similarity is maximized. Since the width w and height h of the common number are known, the content recognition unit 14 may obtain the coordinates of one vertex of the rectangle surrounding the common number, for example, the coordinates 45 of the upper left vertex. The content recognition unit 14 identifies the coordinates 45 , width w and height h as the position and size of the common numeral included in the common numeral image 43 .

Returning to the description of FIG. Next, the content recognition unit 14 executes template matching processing of the characteristic number image to identify the characteristic number (step S9). Then, the content recognition unit 14 recognizes the content of the road sign (step S10). In the present embodiment, the content recognition unit 14 estimates a characteristic numeral presence region in which the characteristic numeral exists in the characteristic numeral image based on the position and size of the common numeral, and extracts a plurality of characteristic numeral template images having different numerals. and the template matching processing of the characteristic numeral existing area. This makes it possible to accurately specify the characteristic number.

FIG. 10 is a drawing-substituting photograph for explaining the template matching processing of characteristic numerals. FIG. 10(a) relates to a description of specifying the position and size of characteristic numerals, and FIG. 10(b) relates to a description of template matching processing of characteristic numerals. As shown in FIG. 10A, the content recognition unit 14 calculates the distance d from the coordinates 45 to the left end of the common numeric image 43 . Next, the content recognizing unit 14 identifies coordinates 46 having the same vertical position as the coordinates 45 in the right end portion of the characteristic numeral image 42 . Next, the content recognition unit 14 extracts a rectangular feature number presence area 47 having a coordinate 46 as the upper right vertex, a width of d+w, and a height of h. Next, the content recognition unit 14 converts the image size of a plurality of pre-stored template images with different numbers using a bilinear interpolation method so that the height is h, and uses them as characteristic number template images. Then, the content recognition unit 14 executes template matching processing between a plurality of characteristic number template images having different numbers and the characteristic number existence area 47 .

In the example shown in FIG. 10B, a characteristic number template image 48a with the number "3", a characteristic number template image 48b with the number "4", and a characteristic number template image 48c with the number "5" are shown. It is For example, when information about whether the road on which the vehicle 100 is traveling is a general road or an expressway is obtained from a car navigation system or the like, the content recognition unit 14 uses only a part of the characteristic number template image 48 to , the template matching process of the characteristic numbers may be executed. For example, the content recognition unit 14 recognizes the characteristic number template images 48 having numbers such as "3", "4" and "5" for general roads, and the numbers "7" and "8" for highways. You may use only the template image 48 for characteristic numerals, such as . The content recognition unit 14 recognizes the content of the maximum speed sign based on the number of the characteristic number template image 48 with the highest degree of similarity. In the example shown in FIG. 10(b), since the characteristic number template image 48b with the number "4" has the maximum similarity of 0.76, the content recognition unit 14 determines that the maximum speed indicated by the maximum speed sign is "40 km/h". /h”.

As described above, in this embodiment, the content recognition unit 14 accurately divides the common numeral image and the characteristic numeral image, and recognizes the characteristic numeral image based on the position and size of the common numeral included in the common numeral image. Since the template matching process is executed, the speed limit sign can be recognized with high accuracy. In particular, in the process of the extracted image generation unit 12, even if the elements of the edges that make up the extracted circle change depending on the sign, or if the noise cannot be completely removed by the noise removal process, the maximum speed sign is robust. can recognize.

The common number template image 44 and characteristic number template image 48 described above are preferably extracted from moving image data obtained by photographing a speed limit sign actually installed on the road. This improves the accuracy of template matching processing.

In the above description, the maximum speed sign 20 is a two-digit number indicating the maximum speed that is a multiple of 10. And even if the maximum speed is a multiple of 10, it is applicable. In this case, the number of the first digit is commonly "0" in all the speed limit signs 20, and is defined as a common number. The second and third digits are numbers that characterize the maximum speed and are defined as characteristic numbers. The content recognition unit 14 determines whether the number is a two-digit number or a three-digit number based on the vertical marginal distribution of the binarized image, and based on the vertical marginal distribution of the binarized image. 1 or 2 division positions are determined, and the binarized image is horizontally divided into two or three divisions at the division positions of 1 or 2, thereby generating 1 or 2 characteristic numeral images and 1 common numeral images. do.

FIG. 11 is a flow chart showing the flow of road sign tracking processing. The road sign tracking unit 15 tracks the position of the road sign by applying a particle filter to subsequent captured images of frames after the captured image in which the content of the road sign has been recognized by the content recognition unit 14 . A particle filter is a state estimation algorithm for non-linear, non-Gaussian state-space models based on Bayes' theorem. For each particle, the road sign tracking unit 15 divides each particle into a first region corresponding to the symbol of the road sign, a second region corresponding to the background of the road sign, and a third region corresponding to the portion excluding the first region and the second region. Calculate the three likelihoods of the region and estimate the central coordinates of the road sign by weighting the likelihoods. Here, a symbol is a figure or a picture to which a character or meaning is attached, and indicates the content of a road sign. In the following description, as in the case described above, a maximum speed sign is used as an example of a road sign.

As shown in FIG. 11, the road sign tracking unit 15 inputs the recognition result of the road sign recognition processing (step S21). The recognition result includes not only the content of the road sign but also various images generated in the road sign recognition process, such as a photographed image, an extracted image, a binary image, a common numeral image, a characteristic numeral image, and various images specified. including coordinates of various positions

Next, the road sign tracking unit 15 scatters particles around the road signs recognized by the road sign recognition processing in the captured image of the current frame (step S22). For example, the scatter range is a square area centered on the center point of the circular sign to be recognized, and the length of one side is 2.5 times the radius of the circular sign, and the number of scatters is 3 of the area in the scatter range. Divide. Assuming that the radius of the recognized circular road sign is R, the length of one side to which particles are scattered is L, and the number of particles to be scattered is N, then L=2.5R and N=0.3×L squared. .

FIG. 12 is a drawing-substituting photograph showing an example of particle scattering. FIG. 12(a) is an example of the scatter range, and FIG. 12(b) is an example of the scatter result. The example shown in FIG. 12 is a case where a circular road sign with a radius of 50 pixels is recognized by road sign recognition processing. As shown in FIG. 12A, the length L of one side for scattering particles is L=2.5×50 pixels=125 pixels. Also, the number N of particles to be scattered is N=0.3×125 pixels×125 pixels≈4,688 pixels. The road sign tracking unit 15 scatters particles evenly with random numbers using the target frame and coordinates as seed values.

Returning to the description of FIG. Next, the road sign tracking unit 15 detects the first area (=area corresponding to the symbol of the road sign) and the second area (=corresponding to the background of the road sign) in the binarized image generated by the road sign recognition processing. area) is extracted (step S23), and the center of gravity of the color information (=RGB values) of the first area and the second area is calculated (step S24).

FIG. 13 is a drawing-substituting photograph for explaining the extraction processing of the first area and the second area. FIG. 13(a) is an example of an extracted image, FIG. 13(b) is an example of a binarized image, and FIG. 13(c) is an example of an inverted image. An extracted image 51 shown in FIG. 13(a) is an image including a speed limit sign. A binarized image 52 shown in FIG. 13B is an image generated from the extracted image 51 . In the following processing, the rectangular area image 53 surrounding the common number is targeted. It should be noted that the rectangular area surrounding the common numeral can be specified based on the result of template matching processing of the common numeral image. An inverted image 54 shown in FIG. 13C is an image obtained by inverting the pixel values of the rectangular area image 53 . The road sign tracking unit 15 extracts the black pixels of the reversed image 54 as the first area and the white pixels of the reversed image 54 as the second area.

Next, the road sign tracking unit 15 acquires color information of pixels forming the first region and the second region from an image having color information (=RGB values), such as the extracted image 51, and formula (1) and The center of gravity is calculated for each of the RGB values according to equation (2), and the color information _C1 representing the first area and the color information _C2 representing the second area are obtained.

Returning to the description of FIG. Next, the road sign tracking unit 15 calculates the likelihood of each particle, and each particle corresponds to a first area, a second area, and a third area (=areas excluding the first area and the second area). ) to which it belongs (step S25). In the present embodiment, the road sign tracking unit 15 uses the color information C ₁ representing the first area, the color information C ₂ representing the second area, and the color information C _p of the particle of interest as three-dimensional coordinates. Calculate the distance and calculate the degree of membership δ ₁ to δ ₄ for each region according to equations (3) to (6). Here, _δ1 is the degree of belonging close to the first region, _δ2 is the degree of belonging far from the first region, _δ3 is the degree of belonging close to the second region, and _δ4 is the degree of belonging far from the second region. Furthermore, the road sign tracking unit 15 calculates the likelihoods μ ₁ to μ ₄ for each area according to equations (7) to (10). where _μ2 is the likelihood of the first region, _μ3 is the likelihood of the second region, and _μ1 and _μ4 are the likelihoods of the third region. Then, the road sign tracking unit 15 takes the maximum likelihood among μ ₁ to μ ₄ as the estimation result of the particle of interest.

FIG. 14 is a diagram explaining the likelihood calculation process. 14(a) is a likelihood estimation table, FIG. 14(b) is a graph of the degree of belonging, FIG. 14(c) is a likelihood estimation table of the first calculation example, and FIG. 14(d) is a likelihood estimation table of the second calculation example. degree estimation table. The likelihood estimation table shown in FIG. 14(a) shows the correspondence between the membership degrees δ ₁ to δ ₄ and the likelihoods μ ₁ to μ ₄ . FIG. 14(b) is a graph of membership degrees _δ1 and _δ3 .

Here, a first calculation example of likelihood will be described. In the first calculation example, the color information _C1 representing the first area is (R, G, B)=(78, 51, 47), and the color information _C2 representing the second area is (R, G, B )=(104, 87, 84), and the color information _Cp of the target particle is (R, G, B)=(79, 54, 39). At this time, the Euclidean distance d ₁₂ between C ₁ and C ₂ is 57.8, the Euclidean distance d _1p between C ₁ and C _p is 8.6, and the Euclidean distance d _2p between C ₂ and C _p is 61.1. When the degrees of belonging δ ₁ to δ ₄ and the likelihoods μ ₁ to μ ₄ are calculated according to equations (3) to (10), the likelihood estimation table of the first calculation example shown in FIG. 14(c) is obtained. Since the maximum likelihood μ ₁ to μ ₄ is the likelihood μ ₂ of 0.94, the road sign tracking unit 15 determines that the particle of interest corresponds to the first region, that is, the road sign symbol. presumed to belong to the region.

Next, a second calculation example of likelihood will be described. In the second calculation example, the color information _C1 representing the first area is (R, G, B)=(78, 51, 47), and the color information _C2 representing the second area is (R, G, B )=(104, 87, 84), and the color information _Cp of the target particle is (R, G, B)=(110, 93, 93). At this time, the Euclidean distance d ₁₂ between C ₁ and C ₂ is 57.8, the Euclidean distance d _1p between C ₁ and C _p is 70.0, and the Euclidean distance d _2p between C ₂ and C _p is 12.4. When the degrees of belonging δ ₁ to δ ₄ and the likelihoods μ ₁ to μ ₄ are calculated according to equations (3) to (10), the likelihood estimation table of the second calculation example shown in FIG. 14(d) is obtained. Since the maximum likelihood μ ₁ to μ ₄ is the likelihood μ ₃ of 0.91, the road sign tracking unit 15 determines that the particle of interest corresponds to the second region, that is, the background of the road sign. presumed to belong to the region.

Returning to the description of FIG. Next, the road sign tracking unit 15 estimates the central coordinates of the road sign in the captured image of the current frame (step S26). In this embodiment, the road sign tracking unit 15 estimates the central coordinates of the road sign based on the likelihood weighting of the first, second, and third regions calculated in step S25. Specifically, the road sign tracking unit 15 calculates the center coordinates O _{(x, y)} of the road sign according to Equation (11). This makes it possible to accurately estimate the central coordinates of the road sign and accurately track the road sign.

Next, the road sign tracking unit 15 confirms whether particles exist in the captured image of the current frame (step S27). If the determination in step S27 is Yes, the road sign tracking unit 15 maintains the recognition result of the road sign recognition processing (step S28), scatters particles in the captured image of the next frame (step S29), and performs the processing of step S25. repeat from The particle scattering range in step S29 is determined in the same manner as in step S22, centering on the center coordinates of the road sign estimated in step S26. On the other hand, if the determination in step S27 is No, the road sign tracking unit 15 terminates the road sign tracking process.

FIG. 15 is a flow chart showing the flow of road sign recognition processing for a no-vehicle entry sign. In the process shown in FIG. 15, the road sign recognition apparatus 1 processes a single captured image and recognizes the content of the vehicle entry prohibition sign included in the captured image. Since the processing of steps S31 to S36 is the same as the processing of steps S1 to S6 shown in FIG. 2, the description thereof is omitted.

FIG. 16 is a drawing-substituting photograph showing an execution example of road sign recognition processing for a no-vehicle entry sign. 16(a) is a vehicle entry prohibition sign 60, FIG. 16(b) is an extracted image 61, and FIG. 16(c) is a binarized image 62. FIG. As shown in FIG. 16(a), the vehicle entry prohibition sign 60 is composed of a background 60a colored in red and a rectangular symbol 60b colored in white and having a longitudinal direction. In the example shown in FIG. 16(b), the binarized image generation unit 13 divides the extracted image 61 into five in the horizontal direction and into three in the vertical direction, and the grid is the center in both the horizontal direction and the vertical direction. A binarization threshold value is determined using Otsu's discriminant analysis method for only the pixels in the central portion 61a, and the entire extracted image 61 is binarized. As a result, a binarized image 62 shown in FIG. 16(c) is generated.

Next, the content recognition unit 14 performs labeling processing (see Non-Patent Document 1) on the binarized image, extracts characteristic portions of the road sign (step S37), and recognizes the content of the road sign (step S38). ). In the present embodiment, the content recognition unit 14 extracts a region having the largest area as a characteristic portion of the road sign as a result of the labeling process. As a result, the symbol 60b of the vehicle entry prohibition sign 60 can be accurately extracted, and the vehicle entry prohibition sign 60 can be accurately recognized. Then, the content recognition unit 14 determines that the value obtained by dividing the width (=horizontal length) of the characteristic portion of the road sign by the height (=vertical length) is within the range of 2.5 to 4.0. If there is, it is recognized as a vehicle entry prohibition sign.

As described above, the road sign recognition device 1 according to the present embodiment can robustly recognize even a road sign at night or a road sign that has faded due to deterioration over time without requiring special components. . In particular, the extracted image generator 12 can apply similar processing to various circular road signs. Moreover, the binarized image generation unit 13 and the road sign tracking unit 15 can apply similar processing to road signs of various shapes, not limited to circular ones. On the other hand, the content recognition unit 14 performs different processing depending on the type of road sign, but in this embodiment, it can recognize the maximum speed sign and no entry sign, which are particularly important at night.

An embodiment of road sign recognition processing for a speed limit sign by the road sign recognition device 1 of the present invention will be described below. The moving image data was captured by a vehicle traveling on general roads and highways in Akita Prefecture at night. An α7S manufactured by Sony Corporation was used as an imaging device. The shooting conditions are 640 x 480 pixels, 256 gradations for each of RGB, 30 fps, ISO sensitivity of 20,000 to 51,200, shutter speed of 1/2000 to 1/3200 seconds, aperture value of F2.8, autofocus and bottom. The conditions for the target speed limit sign are (1) that the number of the speed limit sign can be visually confirmed, (2) that a part of the tree or building does not overlap the speed limit sign and that it can be visually confirmed, ( 3) It was set to be the target lane. For the purpose of solving the problem in the existing technology, the light of the vehicle 100 was always set to low beam and the photographing was performed.

Data set A of video data includes 312 speed limit signs (30 km/h limit: 120, 40 km/h limit: 103, 50 km/h limit: 89) installed on general roads. It was Data set B also included 62 speed limit signs (80 km/h limit) placed on highways. Successful extraction was determined when the maximum speed sign was successfully extracted in one or more frames, and successful recognition was determined when the numbers in the speed limit sign matched the output results of recognition processing in one or more frames.

For dataset A, out of 312 top speed signs, 307 (98.4%) were successfully extracted and 303 (97.1%) were successfully recognized. The road sign recognizing device 1 of the present invention recognizes any of a speed limit sign discolored due to aged deterioration, a speed limit sign installed in a dark environment, and a speed limit sign installed in a bright environment such as an urban area. was also able to be recognized well.

For dataset B, out of 62 highest speed signs, 60 (96.8%) were successfully extracted and 59 (95.2%) were successfully recognized. Since it was raining when the data set B was captured, the wipers of the vehicle 100 were operated and the data set was acquired in a state in which no noticeable water droplets adhered to the windshield. I was able to recognize the signs well.

As described above, the road sign recognition apparatus 1 of the present invention can detect road signs at night, road signs that have been difficult to recognize with the conventional technology, road signs that have faded due to deterioration over time, and road signs in an environment where the vehicle 100 is traveling at high speed. Road signs and road signs during rainfall were also well recognized. That is, the road sign recognition device 1 of the present invention was able to robustly recognize road signs under various environments.

Although the preferred embodiments of the road sign recognition device and the like according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to such examples. It is obvious that a person skilled in the art can conceive of various modifications or modifications within the scope of the technical ideas disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

1……Road sign recognition device 2……Photographing device 3……Output device 4……ECU
REFERENCE SIGNS LIST 11: Image input unit 12: Extracted image generation unit 13: Binary image generation unit 14: Content recognition unit 15: Road sign tracking unit 100: Vehicle

Claims (4)

A road sign recognition device for recognizing road signs,
an image input unit for inputting a captured image;
an extracted image generating unit that detects the road sign included in the captured image, extracts a part of the captured image, and generates an extracted image;
a binarized image generation unit that binarizes the extracted image to generate a binarized image;
a content recognition unit that recognizes the content of the road sign from the binarized image;
with
The binarized image generation unit determines a binarization threshold based on a histogram of brightness values of pixels included in the central portion of the extracted image,
The road sign is a speed limit sign indicating the maximum speed,
The number of the speed limit sign is composed of a characteristic number characterizing the speed limit sign and a common number common to the speed limit sign,
The content recognition unit determines a division position based on a peripheral distribution in the vertical direction of the binarized image, and divides the binarized image at the division position to obtain a characteristic numeral image including the characteristic numeral and a generating a common numeral image containing the common numeral;
The content recognition unit executes template matching processing between a plurality of common number template images having different image sizes and the common number image, and identifies the position and size of the common number in the common number image,
The content recognition unit estimates a characteristic numeral existence region in which the characteristic numeral exists in the characteristic numeral image based on the position and size of the common numeral, and extracts a plurality of characteristic numeral template images having different numerals and the characteristics. Perform template matching processing for digit presence area
A road sign recognition device characterized by: A road sign recognition device for recognizing road signs,
an image input unit for inputting a captured image;
an extracted image generating unit that detects the road sign included in the captured image, extracts a part of the captured image, and generates an extracted image;
a binarized image generation unit that binarizes the extracted image to generate a binarized image;
a content recognition unit that recognizes the content of the road sign from the binarized image;
with
The binarized image generation unit determines a binarization threshold based on a histogram of brightness values of pixels included in the central portion of the extracted image,
The image input unit inputs the captured images of a plurality of frames in chronological order,
a road sign tracking unit that tracks the position of the road sign in a subsequent captured image of a frame after the captured image in which the content of the road sign is recognized by the content recognition unit;
The road sign tracking unit tracks the position of the road sign by applying a particle filter to the subsequently captured image, and for each particle, a first region corresponding to the symbol of the road sign, Three likelihoods of a second area corresponding to the background and a third area corresponding to a portion excluding the first area and the second area are calculated, and the center coordinates of the road sign are calculated based on the weighting of the likelihoods. to estimate
A road sign recognition device characterized by: A program for causing a computer to function as a road sign recognition device for recognizing road signs,
said computer,
an image input unit for inputting a captured image;
an extracted image generation unit that extracts a portion including the road sign from the captured image and generates an extracted image;
a binarized image generation unit that binarizes the extracted image to generate a binarized image;
a content recognition unit that recognizes the content of the road sign from the binarized image;
with
The binarized image generation unit determines a binarization threshold based on a histogram of brightness values of pixels included in the central portion of the extracted image,
The road sign is a speed limit sign indicating the maximum speed,
The number of the speed limit sign is composed of a characteristic number characterizing the speed limit sign and a common number common to the speed limit sign,
The content recognition unit determines a division position based on a peripheral distribution in the vertical direction of the binarized image, and divides the binarized image at the division position to obtain a characteristic numeral image including the characteristic numeral and a generating a common numeral image containing the common numeral;
The content recognition unit executes template matching processing between a plurality of common number template images having different image sizes and the common number image, and identifies the position and size of the common number in the common number image,
The content recognition unit estimates a characteristic numeral existence region in which the characteristic numeral exists in the characteristic numeral image based on the position and size of the common numeral, and extracts a plurality of characteristic numeral template images having different numerals and the characteristics. Perform template matching processing for digit presence area
A program for functioning as a road sign recognition device. A program for causing a computer to function as a road sign recognition device for recognizing road signs,
said computer,
an image input unit for inputting a captured image;
an extracted image generation unit that extracts a portion including the road sign from the captured image and generates an extracted image;
a binarized image generation unit that binarizes the extracted image to generate a binarized image;
a content recognition unit that recognizes the content of the road sign from the binarized image;
with
The binarized image generation unit determines a binarization threshold based on a histogram of brightness values of pixels included in the central portion of the extracted image,
The image input unit inputs the captured images of a plurality of frames in chronological order,
a road sign tracking unit that tracks the position of the road sign in a subsequent captured image of a frame after the captured image in which the content of the road sign is recognized by the content recognition unit;
The road sign tracking unit tracks the position of the road sign by applying a particle filter to the subsequently captured image, and for each particle, a first region corresponding to the symbol of the road sign, Three likelihoods of a second area corresponding to the background and a third area corresponding to a portion excluding the first area and the second area are calculated, and the central coordinates of the road sign are calculated based on the weighting of the likelihoods. to estimate
A program for functioning as a road sign recognition device.

Images