CN108241874B - Video text area localization method based on BP neural network and spectrum analysis - Google Patents

Abstract

The video text area localization method based on BP neural network and spectrum analysis, by constructing a BP neural network to classify the pixels in the video frame, and processing the pixels classified into the text class by a distance-based clustering algorithm to obtain candidates In the text area, after converting the candidate area to the frequency domain through fast Fourier transform, the neural network is constructed again, the candidate area is classified based on the spectrogram, and the candidate area classified as the false positive class is filtered out, thereby filtering out the false positive. The invention has the beneficial effects that the positioning accuracy is high and the practicability is more comprehensive.

Description

Video text area localization method based on BP neural network and spectrum analysis

technical field

The invention relates to the technical field of image processing, in particular to a video text area positioning method based on BP neural network and spectrum analysis.

Background technique

With the explosive growth of multimedia data, various forms of multimedia information such as text, images, voice, and video will be transmitted to the Internet, and people are rapidly entering the information society. The application of multimedia information in network and communication is also more and more extensive, especially represented by video data, which has become an important resource for people to share information. Video data has become the most important information carrier with its rich, intuitive and specific information expression, conveying a lot of information and knowledge to people. Among them, news video, as a representative medium of video data, occupies an important proportion of video resources. Compared with text news, video news is vivid, intuitive, easy to understand, and has a large amount of information, which has attracted people's attention. Due to the particularity of news videos, most of the high-level semantics are in text subtitles, and audio and image features are almost included in text features, so the location and extraction of subtitle regions in news videos is particularly important.

Since the color, size, font and position of the text is variable, it is difficult to find a general way to separate it from the background. Text location methods can be divided into two broad categories: region-based methods and texture-based methods. These methods have their own advantages and disadvantages, and the appropriate method can be used for different situations to have good results. However, most methods contain many false positive regions after localization, which reduces the accuracy of text localization. Therefore, the present invention proposes a new method, which is based on BP neural network and spectrum analysis to effectively locate the subtitle region of news video and filter out the located false positive region.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide a video text area localization method based on BP neural network and spectrum analysis, to solve the problem of false positives in the current existing methods, and to improve the accuracy of algorithm localization.

The technical scheme adopted by the present invention to solve the above-mentioned technical problems is: a method for locating video text region based on BP neural network and spectrum analysis, comprising the following steps:

Step 1. Extract news video frames, and convert the extracted news video frames into grayscale images;

Step 2. Build a BP neural network as a classifier, classify all pixels in each image, and obtain pixels classified as text;

Step 3. Perform distance-based clustering processing on the pixels classified as text classes obtained in step 2 to obtain candidate text regions;

Step 4, perform fast Fourier transform on the candidate text area obtained in step 3 to obtain a spectrogram;

Step 5: Build the BP neural network again as a classifier, classify the candidate text regions, and filter out false positive regions.

The step 2 of the present invention constructs a BP neural network as a classifier, and the specific method for classifying all the pixels in each image is as follows:

Step 2.1. Perform corner detection on all pixels in the image, and assign a feature value of 1 to a pixel determined to be a corner, and a feature value of 0 to non-corner points;

Step 2.2. Take each pixel as the center pixel in turn, and take its M*M-sized neighborhood window as the feature window;

Step 2.3, build a neural network, take the gray value of all pixels in the selected window and the corner point judgment value as the input of the neural network, set the number m of nodes in the input layer to M*M*2, and set the number of nodes in the output layer as M*M*2. The number of points n is set to 2;

Step 2.4. Set the hidden layer nodes. The setting of the number N of hidden layer nodes is calculated by formula (1) or (2):

Among them, N represents the number of hidden layer nodes set, m and n represent the number of input layer and output layer nodes, respectively, a is a constant;

Step 2.5. The two nodes in the output layer represent text classes and non-text classes respectively, and the output is a vector containing two floating-point values. When calibrating the sample, the pixels belonging to the text class are demarcated as (1,0), and the Non-text pixels are marked as (0,1);

Step 2.6. Train and test the neural network. In the output vector of the pixel of the test sample, if the first value is greater than the second value, the pixel is classified as a text class, if the second value is greater than the first value. value, then the pixel is classified as a non-text class, and finally all pixels that are determined to be a text class are marked.

The step 3 of the present invention performs distance-based clustering processing on the pixels classified as text, and the specific method for obtaining candidate text regions is as follows:

Step 3.1. Set the distance threshold d ₁ . Among all the pixels classified as text, randomly select a pixel P ₁ as the basic pixel, and calculate the distance between P ₁ and other pixels classified as text. Euclidean distance, and add the pixels whose Euclidean distance is less than d ₁ to the set G ₁ of P ₁ until all the pixels that meet the conditions are found, and then take other pixels in G ₁ except P ₁ as the basic Pixels perform the same operation until no new pixels are added to the set, then the set G ₁ will be classified as K ₁ ;

Step 3.2. Repeat the above operation for all other pixels classified as text class except K ₁ class until all text class pixels are classified, and obtain all classes K _t , t≥1;

Step 3.3. Clear all classes that contain too few pixels;

Step 3.4, make the minimum circumscribed rectangle of each class, that is, obtain the candidate text area.

The specific method for obtaining a spectrogram by performing fast Fourier transform on the obtained candidate text region in step 4 of the present invention is:

Step 4.1, perform image binarization on the candidate text area;

Step 4.2, perform grayscale projection on the binarized image in the vertical direction;

Step 4.3: Perform fast Fourier transform on the projected function, convert the time domain to the frequency domain, and obtain a spectrogram.

Step 5 of the present invention classifies candidate text regions, and the specific method for filtering out false positive regions is as follows: Step 5.1. In the spectrogram, select a feature window with a size of 2 to 3 times the average text width, and this window does not contain frequency 1 ;

Step 5.2, build a BP neural network, take the amplitude corresponding to the frequency in the selected window range and the frequency value of the highest amplitude in the frequency near the average text width as the input value of the neural network, and set the number of nodes in the output layer to 2 ;

Step 5.3. Use formula (1) or formula (2) to select hidden layer nodes;

Step 5.4, the number of 2 output layer nodes respectively represent the true positive area (candidate text area containing text) and false positive area (candidate text area without text), the output is a vector containing two floating-point values, and the sample is calibrated When , the true positive text area is marked as (1,0), and the false positive area is marked as (0,1);

Step 5.5. Train and test the neural network. In the output vector of the candidate text area of the test sample, if the first value is greater than the second value, the candidate area will be classified as a true positive class. If the second value is greater than the first value, then the candidate region will be classified as a false positive class and filtered out;

Step 5.6: The true positive candidate text region remaining after filtering out the false positive region is the final text positioning region.

The method for corner detection in step 2.1 of the present invention is the Harris corner detection method.

The value of the constant a in step 2.4 of the present invention is 1-10.

The number of pixel points of the class to be cleared in step 3.3 of the present invention is less than 20.

The beneficial effects of the present invention are: the method provided by the present invention classifies the pixel points in the video frame by constructing a BP neural network, and performs a distance-based clustering algorithm on the pixel points classified into the text category to obtain candidates. In the text area, after the candidate area is converted to the frequency domain by fast Fourier, the neural network is constructed again, the candidate area is classified based on the spectrogram, and the candidate area classified as false positive class is filtered out, so as to filter out the false positive and improve the algorithm. The accuracy rate of the video text area is more accurate and the practicability is more comprehensive.

Description of drawings

1 is a schematic flowchart of a positioning method according to the present invention;

Fig. 2 is the BP neural network model constructed by the present invention;

3 is a diagram of a distance-based breadth-first clustering algorithm of the present invention;

Fig. 4 is the candidate text area positioning example diagram of the present invention;

FIG. 5 is an example diagram of the location of candidate text regions containing false positives according to the present invention;

Fig. 6 is a true positive example diagram in the candidate text area of the present invention;

Fig. 7 is a false positive example diagram in the candidate text area of the present invention;

Fig. 8 is the spectrogram of the true positive example graph of Fig. 6;

Fig. 9 is the spectrogram of the false positive example graph of Fig. 7;

Fig. 10 is an effect diagram after filtering out false positives in Fig. 5;

FIG. 11 is a comparison diagram of the positioning results of the present invention and the existing method under different conditions.

Detailed ways

The specific embodiments (embodiments) of the present invention will be described below with reference to the accompanying drawings, so that those skilled in the art can better understand the present invention.

Figure 1 is a flow chart of the method of the present invention. The video text area localization method based on BP neural network and spectrum analysis is divided into the following steps:

Step 1: Build the BP neural network model as shown in Figure 2 as the classifier, select the 5*5 size neighborhood window as the feature window, set the number of input layer nodes to 50, the number of hidden layer nodes to 17, and output The number of layer nodes is 2, and the video frame pixels in the dataset are manually marked, and the neural network is trained and tested.

As shown in Figure 3, the pixels classified as text are marked in blue for easy display.

Step 2: Perform distance-based breadth-first clustering on the pixels classified as text categories obtained after the processing in Step 1. FIG. 3 is a schematic diagram of the algorithm.

Figure 4 shows the localization effect of candidate text regions in the example image after clustering.

Step 3: Since step 1 cannot completely and correctly classify text and non-text pixels, false positive regions will be generated in some cases. Figure 5 is a localization result map with multiple false positive candidate text regions .

After binarizing all the obtained candidate text regions, perform grayscale projection in the vertical direction, and perform Fourier transform to obtain a spectrogram. Figures 6 and 7 are example diagrams of true positive regions and false positive regions. Figure 8 is the spectrogram of the true positive example, and Figure 9 is the spectrogram of the false positive example of Figure 7. Comparing the two, it is found that the spectrograms of the true positive area and the false positive area are significantly different.

Build the BP neural network as the classifier again, select the frequency 2-frequency 60 window as the feature window in the spectrogram of the candidate area, and set the amplitude corresponding to the insufficient frequency to 0 for the candidate text area with a width of less than 60. The number of nodes in the input layer is 60, the number of nodes in the hidden layer is 45, and the number of nodes in the output layer is 2. All generated candidate text regions are manually annotated, and the neural network is trained and tested.

Step 4: All candidate text regions classified as false positives are filtered out, and the remaining true positive candidate text regions are the final text positioning regions. Figure 10 is the final positioning result of the example image in Figure 5 after false positive filtering.

Step 5: Figure 11 compares the localization results of this method with the existing corner-based methods and edge-based methods in some different situations. It is found that this method is more comprehensive in applicability, and has a variety of different localization situations. Good positioning effect.

The present invention uses the Harris corner detection method to perform corner detection on all pixels in the picture, but is not limited to the Harris algorithm, and other corner detection algorithms can also be used. Because the Harris algorithm can detect interest points in the case of grayscale changes, rotation and interference noise in the image, it has good anti-noise ability, and takes into account the requirements of both efficiency and accuracy, the false detection rate is low, and the corner extraction The reliability is high, so the present invention uses this method for corner detection.

The above is a method for locating the text region of news video based on BP neural network and spectrum analysis provided by the present invention. However, the protection scope of the present invention is not limited to this, and any person skilled in the art will make changes based on the idea of the present invention. To sum up, the content of this specification should not be construed as a limitation of the present invention.

Claims (6)

1. the video text area localization method based on BP neural network and spectrum analysis, is characterized in that: may further comprise the steps: Step 1. Extract news video frames, and convert the extracted news video frames into grayscale images; Step 2. Build a BP neural network as a classifier, classify all pixels in each image, and obtain pixels classified as text; Step 3. Perform distance-based clustering processing on the pixels classified as text classes obtained in step 2 to obtain candidate text regions; Step 4, perform fast Fourier transform on the candidate text area obtained in step 3 to obtain a spectrogram; Step 5. Build the BP neural network again as a classifier, classify the candidate text regions, and filter out the false positive regions. The specific method is as follows: Step 5.1. In the spectrogram, select a feature window with a size of 2 to 3 times the average text width, and this window does not contain frequency 1; Step 5.2, build a BP neural network, take the amplitude corresponding to the frequency in the selected window range and the frequency value of the highest amplitude in the frequency near the average text width as the input value of the BP neural network, and set the number of output layer nodes as 2; Step 5.3. Use formula (1) or formula (2) to select hidden layer nodes:

Among them, N represents the number of hidden layer nodes set, m and n represent the number of input layer and output layer nodes, respectively, a is a constant; In step 5.4, the two output layer nodes represent the true positive area and the false positive area respectively, and the output is a vector containing two floating-point values. The positive area is marked as (0,1); Step 5.5. Train and test the BP neural network. In the output vector of the candidate text area of the test sample, if the first value is greater than the second value, then the candidate text area will be classified as a true positive class. If the value is greater than the first value, the candidate text area will be classified as false positive and filtered out; Step 5.6: The true positive candidate text region remaining after filtering out the false positive region is the final text positioning region. 2. the video text area localization method based on BP neural network and spectrum analysis according to claim 1, is characterized in that: described step 2 builds BP neural network as classifier, and all pixel points in each image are classified The specific method is: Step 2.1. Perform corner detection on all pixels in the image, and assign a feature value of 1 to a pixel determined to be a corner, and a feature value of 0 to non-corner points; Step 2.2. Take each pixel as the center pixel in turn, and take its M*M-sized neighborhood window as the feature window; Step 2.3. Build a BP neural network, take the gray value of all pixels in the window and the judgment value of the corner points as the input of the BP neural network, set the number m of nodes in the input layer to M*M*2, and output The number of layer nodes n is set to 2; Step 2.4. Set the hidden layer nodes. The setting of the number N of hidden layer nodes is calculated by formula (1) or (2):

Among them, N represents the number of hidden layer nodes set, m and n represent the number of input layer and output layer nodes, respectively, a is a constant; Step 2.5. The two nodes in the output layer represent text classes and non-text classes respectively, and the output is a vector containing two floating-point values. When calibrating the sample, the pixels belonging to the text class are demarcated as (1,0), and the Non-text pixels are marked as (0,1); Step 2.6. Train and test the BP neural network. In the output vector of the pixel point of the test sample, if the first value is greater than the second value, then the pixel is classified as a text class. If the second value is greater than the first value A value, then the pixel is classified as a non-text class, and finally all pixels that are determined to be a text class are marked. 3. the video text area positioning method based on BP neural network and spectrum analysis according to claim 1, is characterized in that: described step 3 carries out distance-based clustering processing to the pixel points classified as text class, obtains candidate text The specific methods of the area are: Step 3.1. Set the distance threshold d ₁ . Among all the pixels classified as text, randomly select a pixel P ₁ as the basic pixel, and calculate the distance between P ₁ and other pixels classified as text. Euclidean distance, and add the pixels whose Euclidean distance is less than d ₁ to the set G ₁ of P ₁ until all the pixels that meet the conditions are found, and then take other pixels in G ₁ except P ₁ as the basic Pixels perform the same operation until no new pixels are added to the set, then the set G ₁ will be classified as K ₁ ; Step 3.2. Repeat the above operation for all other pixels classified as text classes except K ₁ class until all text class pixels are classified, and get all classes K ₁ , K ₂ , ..., K _t , t ≥1; Step 3.3. Clear all classes containing less than 20 pixels; Step 3.4, make the minimum circumscribed rectangle of each class, that is, obtain the candidate text area. 4. the video character area positioning method based on BP neural network and spectrum analysis according to claim 1, is characterized in that: the concrete method that described step 4 carries out fast Fourier transform to the candidate character area that obtains and obtains spectrogram is: : Step 4.1, perform image binarization on the candidate text area; Step 4.2, perform grayscale projection on the binarized image in the vertical direction; Step 4.3: Perform fast Fourier transform on the projected function, convert the time domain to the frequency domain, and obtain a spectrogram. 5. The video text region localization method based on BP neural network and spectrum analysis according to claim 2, wherein the method for corner detection in the step 2.1 is Harris corner detection method. 6 . The method for locating video text regions based on BP neural network and spectrum analysis according to claim 2 , wherein the value of the constant a in the step 2.4 is 1-10. 7 .

Images