CN108614849B - A Webpage Advertisement Detection Method Based on Dynamic Insertion and Static Multi-Script Page Feature Extraction - Google Patents

Abstract

The invention discloses a novel detection method for webpage advertisements, which uses a program analysis method combining dynamic analysis and static analysis to identify and detect advertisement codes contained in a webpage. Firstly, acquiring a possible advertisement position in a webpage by using a dynamic analysis method, then recording and tracking the advertisement on the position, and acquiring function call path information generated by the advertisement so as to acquire a generated script file set; classifying the file set by using the static characteristics, and extracting the static characteristics on the advertisement generation path; and regulating and controlling the types and the quantity of the contained static features according to the test set. On the basis of reducing the omission factor, the detection precision of the webpage dynamic advertisement is improved.

Description

A Webpage Advertisement Detection Based on Dynamic Insertion and Static Multi-Script Page Feature Extraction method

technical field

The invention relates to a webpage advertisement detection method based on dynamic stub insertion and static multi-script page feature extraction, belonging to the field of Internet and software engineering.

Background technique

With the popularization and development of the Internet, online advertising has become an important prerequisite for the profitability and sustainable development of free websites; and with the development of Web2. , webpage advertisements may appear in the form of pop-up windows, or occupy part of the webpage to attract users to click. The presentation of such advertisements has promoted the business development of online sellers mainly based on e-commerce, and has also improved the quality of services provided by free websites to users.

The rapid development of online advertising has provided a lot of convenience for the network business model, but it has also provided convenience for the spread of many malicious websites: some malicious websites load their own malicious scripts on normal pages through advertising alliances, and induce users to click on these websites disguised as Malicious links of advertisements; some webpage advertisements occupy a large amount of space on the webpage, seriously affecting the reading experience of users when browsing the webpage; some webpage advertisements seriously interfere with the normal visit of users, and collect and violate the privacy information of users.

In recent years, detection methods for online advertisements mainly focus on methods such as static pattern matching and static feature matching.

There are two static pattern matching methods: by collecting the domain names of all advertising service companies to generate a blacklist; by using selector pattern matching to identify advertising elements in the browser.

The detection method of static feature matching extracts features from webpages containing webpage advertisements, and analyzes webpages by acquiring features such as native function calls, eval function usage, code length, whether they contain specific strings, and whether obfuscation techniques are used. Ads are identified and detected.

The static pattern matching method in the prior art cannot correctly detect obfuscated domain names and selectors; in addition, the static feature matching method only uses a single page as a data set, and the detection accuracy is not high.

Since static advertisements on webpages often insert a picture containing a link in the original webpage, sometimes even a simple link label, which is not substantially different from the picture links of a large number of external websites on the portal website, so static advertisements are not included. within our focus.

Regarding the dynamic nature of advertisements, we focus on dynamic advertisements rather than simple image links. For the analysis of dynamic advertisements, we mainly consider the advertisements spread through the advertising network. This type of advertisement usually only requires the publisher of the web page to insert a specified tag when coding the page for positioning and placement of the advertisement. The advertising network recognizes the This tag dynamically generates the advertisement content to be displayed according to information such as the cookie of the user browsing the page. The display of advertisements is to insert some JavaScript script files into the page, these script files are often executed automatically, and after a series of function calls, different advertisements are finally displayed on the page.

Contents of the invention

Technical problem: the purpose of the present invention is to overcome the deficiencies of the prior art, according to the propagation path of the advertisement in the webpage, dynamically acquire all the script files on the advertisement generation path, and use it as a feature to identify the webpage advertisement.

To achieve the above object, the present invention first dynamically obtains the function call path information of the advertisement generation path in the webpage by executing the page that includes the webpage advertisement, and obtains all JavaScript script files needed for webpage advertisement generation by calling the path information; on this basis, Static analysis is performed on all script files, and web page advertisements are identified through features.

The invention obtains the advertisement in the web page and the call path generated by the advertisement by using dynamic insertion, which overcomes the inability to detect the confusing domain name and the selector in the static pattern matching method; The script is used to extract features, which is highly targeted and overcomes the defect of large data noise in the static feature matching method.

The inventive method specifically comprises the steps:

Step 1: Analyze the ad, get its dynamic characteristics, and position the ad in the web page

Through the analysis of the dynamic advertisements in the webpage, the dynamic characteristics of the webpage advertisements can be obtained. For this purpose, it is necessary to analyze the complete generation process of the dynamic advertisements, and compare the differences between them and ordinary webpage elements to locate specific webpage advertisements.

Step 2: Track the calling path of the webpage advertisement

Through step 1, the specific location of the webpage advertisement in the page is located, and then the complete call path tracking of the webpage advertisement is carried out, including the function call path during the advertisement generation process and the specific execution script code and other information, through the acquisition of the function call path You can get all the JavaScript script files on the advertisement generation path. In this way, the analysis of advertisements may not be limited to feature analysis of elements on a single page.

Step 3: Extract features for multiple script files obtained

According to the method in step 2, obtain JavaScript script files related to advertisements, collect a large number of advertisement-related and irrelevant script files, and perform feature extraction to obtain static features related to advertisement generation, including HTML DOM element features, JavaScript script features, and CSS features etc., train this with a classifier to generate an ad code detection model.

Step 4: Feedback on the results

Use the ad code detection model to run test data, compare its results with actual ads, adjust thresholds in the classification process, and detect and identify actual web ads.

Compared with prior art, the present invention has following advantage:

Through dynamic detection, the interference of obfuscated code can be overcome, and the missed detection rate of the page can be reduced; by obtaining multiple JavaScript script files on the advertisement call path, multi-level feature extraction is performed, which is more comprehensive and reliable.

Description of drawings

Fig. 1 is a flow chart of acquiring the starting position of an advertisement;

Fig. 2 is a flow chart of obtaining an advertisement function call path;

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings.

Step 1: Analyze the ad to get its dynamic characteristics and position the ad in the web page

Through the analysis of the dynamic advertisements in the webpage, the dynamic characteristics of the webpage advertisements are obtained. These dynamic characteristics need to analyze the complete generation process of the dynamic advertisements, and compare the differences between them and the common webpage elements to locate the specific webpage advertisement codes.

Regarding the dynamic nature of advertisements, we focus on dynamic advertisements rather than simple image links. For the analysis of dynamic advertisements, we mainly consider the advertisements spread through the advertising network. This type of advertisement usually only requires the publisher of the web page to insert a specified tag for positioning and placement of the advertisement when coding the page. The advertising network recognizes the This tag dynamically generates the advertisement content to be displayed according to information such as the cookie of the user browsing the page. The display of advertisements is to insert some JavaScript script files into the page, these script files are often executed automatically, and after a series of function calls, different advertisements are finally displayed on the page.

It can be known from the process of advertisement generation that the generation of dynamic advertisements must call a third-party script library, that is, the advertising network, to generate advertisements. Therefore, the dynamic feature of advertisement generation we use comes from the automatic execution of the JavaScript code of the third-party script library.

Figure 1 inserts web page elements according to this feature, records the functions that are actively executed, and judges whether they come from a third-party script library, and marks the execution that meets the conditions.

Step 2: Track the calling path of the webpage advertisement

The specific location of the webpage advertisement in the page is located through step 1, and then a complete call path tracking is performed on the webpage advertisement.

The generation of web advertisements has undergone multiple function calls, and the calling path has actually experienced multiple jumps, and multiple script files have also been called. The script files in the calling path of web advertisement generation are the datasets we need to use to obtain features. Therefore, it is considered that for the possible advertising slots that have been determined in step 1, it is necessary to track and record their calling paths.

For function calls in JavaScript, you cannot get which function is called when the function is executed, but you can get the function that calls itself. According to this feature, as shown in Figure 2, the caller of each function is obtained during the operation of the web page, and it is judged whether the caller has been marked. If the caller is marked, path information is added to the function itself to track the path and save; if not marked, do nothing.

Using the dynamic instrumentation method, you can add custom attributes containing call information to the functions in JavaScript. By displaying the call information, you can obtain the specific set of script files on the path. In this way, the characteristics of all script files on the advertisement generation path can be analyzed.

Step 3: Extract features for multiple script files obtained

Use the multi-script file obtained in step 2 as a data set to extract the static features of the advertisement. For a large number of websites containing advertisements, use the dynamic insertion method in step 2 to obtain the JavaScript script files related to the advertisements, and use the batch processing method to save the corresponding files as the data set for extracting the static characteristics of the advertisements. The same method will be used to save the JavaScript script files irrelevant to advertisements as a control data set irrelevant to advertisements.

Static features include the depth of the function call path during the advertisement generation process, the number of string splicing in the script file, the number of dynamic code execution, the type and number of native functions used, the type and number of JavaScript event processing functions used, etc. Feature extraction is performed on the advertisement file according to these features, and it is determined that the script file satisfying certain features is an advertisement.

Step 4: Feedback on the results

Compare the result of the classifier running the test data with the actual advertisement, adjust the threshold in the classification process, and detect and identify the webpage advertisement.

Divide the experimental data into two categories. The first category is training data, which is used to classify the features used in step 3 into advertisement-related features and advertisement-independent features; the second category is test data, which is used to perform training on the trained model. Tests to evaluate the accuracy of the model.

The present invention is not limited to the above-mentioned examples, and all equivalent replacements or technical solutions formed by equivalent replacements fall within the protection scope of the present invention.

Claims (5)

1. A webpage advertisement detection method based on dynamic stub insertion and static multi-script page feature extraction, it is characterized in that, first by executing the page that contains webpage advertisement, dynamically obtain the function call path information of the advertisement generation path in the webpage, by calling the path Information acquisition of all JavaScript script files required for generating web page advertisements; on this basis, static analysis is performed on all script files, and web page advertisements are identified through feature classification models; Include the following steps: 1) Analyze the advertisement, obtain its dynamic characteristics, and locate the advertisement in the webpage, and obtain the dynamic characteristics of the webpage advertisement through the analysis of the dynamic advertisement in the webpage. For this reason, it is necessary to analyze the complete generation process of the dynamic advertisement, and compare The difference between it and ordinary web page elements is to locate specific web page advertising codes; 2) Use the method of 1) to locate the specific location of the webpage advertisement on the page, and then trace the complete calling path of the webpage advertisement, including the function calling path during the advertisement generation process and the specific execution script code information, through calling the function Obtaining the path obtains all JavaScript script files on the advertisement generation path; 3) extract features for the multiple script files obtained, that is, according to the JavaScript script files obtained in 2), extract features for multiple JS files, and obtain static features generated by advertisements, including HTMLDOM element features, JavaScript script features, CSS features, using The classifier is trained on this to generate an ad code detection model; 4) Feedback on the results, run the test data using the advertising code detection model, compare the results with the actual advertisements, adjust the thresholds in the classification process, and detect and identify the actual web advertisements. 2. the web page advertisement detection method based on dynamic insertion and static multi-script page feature extraction according to claim 1, is characterized in that in step 1), the advertisement is analyzed to obtain its dynamic characteristics and locate the advertisement in the webpage, specifically for: Through the analysis of the dynamic advertisements in the webpage, the dynamic characteristics of the webpage advertisements are obtained. For this reason, it is necessary to analyze the complete generation process of the dynamic advertisements, and compare the differences between them and the elements of ordinary webpages, and locate the specific webpage advertisement codes; For the analysis of dynamic advertisements, it is necessary to consider the advertisements spread through the advertising network. The generation of such advertisements calls the script library of the third party, namely the advertising network, to generate advertisements. The dynamic characteristics of the advertisement generation come from the third-party script library. JavaScript code is executed automatically. 3. the webpage advertisement detection method based on dynamic instrumentation and static multi-script page feature extraction according to claim 1, is characterized in that in step 2), the webpage advertisement is carried out the tracking of calling path: After locating the specific position of the webpage advertisement on the page, the complete calling path of the webpage advertisement is tracked; the generation of the webpage advertisement has undergone multiple function calls, and the calling path includes multiple jumps, that is, calling multiple script files , these script files are the data sets needed for subsequent acquisition of advertising features, therefore, for the possible advertising positions that have been determined, it is necessary to track and record their calling paths; Obtain the caller of each function during webpage operation, and judge whether the caller has been marked. If the caller is marked, add path information to the function itself, track and save the path; if not marked, do not any operation; Use the dynamic instrumentation method to add custom attributes containing call information to the functions in JavaScript, and obtain the specific script file collection on the path by displaying the call information, so as to analyze the characteristics of all script files on the advertisement generation path . 4. the web page advertisement detection method based on dynamic insertion and static multi-script page feature extraction according to claim 1, is characterized in that in step 3), extracts feature for a plurality of script files obtained: Use the dynamic insertion method for a large number of websites that contain advertisements to obtain advertisement-related JavaScript script files; use the batch method to save the corresponding files as a data set for extracting static features of advertisements; save JavaScript script files that have nothing to do with advertisements Use the same method to save and use it as a control data set that has nothing to do with advertisements; extract static features of JavaScript script files related to advertisements, including the depth of function call paths during advertisement generation, the number of string splicing in script files, and dynamic execution Code times, types and times of using native functions, types and times of using JavaScript event processing functions, based on which feature extraction is performed on advertisement files to determine whether script files satisfying certain characteristics are advertisements. 5. the web page advertisement detection method based on dynamic insertion and static multi-script page feature extraction according to claim 1, is characterized in that in step 4), the result is fed back, and the result of the classifier operation test data is compared with the actual Advertisements are compared, the threshold in the classification process is adjusted, and web advertisements are detected and identified; the training data in the experimental data is used to classify the used features into advertisement-related features and advertisement-independent features; the test data is used to train good The model is tested to evaluate the accuracy of the model.

Images