WO2022227207A1

WO2022227207A1 - Text classification method, apparatus, computer device, and storage medium

Info

Publication number: WO2022227207A1
Application number: PCT/CN2021/097195
Authority: WO
Inventors: 刘翔; 谷坤
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-04-30
Filing date: 2021-05-31
Publication date: 2022-11-03
Also published as: CN113011533A; CN113011533B

Abstract

A text classification method, an apparatus, a computer device, and a storage medium. The method comprises: extracting target text data to be analyzed from an original text (301); performing pre-processing on the target text data and obtaining a word segmentation result for the target text data (302); obtaining a target character vector, a target word vector, and a target position vector corresponding to the target text data on the basis of the word segmentation result (303); inputting the target character vector, the target word vector, and the target position vector into a pre-trained text classification model, and obtaining a target classification label output by the text classification model, wherein the text classification model is an ALBERT model that has undergone fine-tuning (304). The present method utilizes an ALBERT model to perform processing on text data, and text classification efficiency and accuracy are effectively increased.

Description

Text classification method, apparatus, computer equipment and storage medium

This application claims the priority of the Chinese patent application with the application number 202110482695.1 and the title of the invention "Text Classification Method, Apparatus, Computer Equipment and Storage Medium", which was submitted to the State Intellectual Property Office on April 30, 2021, the entire contents of which are by reference Incorporated in this application.

technical field

The present application relates to the technical field of natural language processing, and in particular, to text classification methods, apparatuses, computer equipment and storage media.

Background technique

With the rapid development of network technology, massive information resources exist in the form of text. The inventor realized that how to effectively classify these texts and mine effective information from massive texts quickly, accurately and comprehensively has become one of the hotspots in the field of natural language processing research. A text classification method refers to determining a category for each document in a document collection according to a predefined topic category. The text classification method technology has a wide range of applications in daily life, for example, the technical division of patent texts and so on.

Compared with general texts, patent texts have the characteristics of special structure, strong professionalism, and more vocabulary in the field, so a more targeted classification method needs to be adopted. The patent text classification method belongs to the field of natural language processing, and generally includes steps such as data preprocessing, text feature representation, classifier selection and effect evaluation. Among them, text feature representation and classifier selection are the most important, which will directly affect the accuracy of classification results.

In the prior art, text classification methods based on traditional machine learning, such as the TF-IDF text classification method, only measure the importance of words by "word frequency", and the subsequent feature value sequences of documents are independent of each other and cannot reflect the sequence. information; easily affected by the skewness of the dataset, such as too many documents in a certain category, it will lead to underestimation of the IDF; the processing method is to increase the category weight. The intra- and inter-class distribution biases (when used for feature selection) are not considered. Text classification methods based on deep learning, such as Facebook's open source FastText text classification method, Text-CNN text classification method, Text-RNN text classification method, etc. TextCNN can perform well in many tasks, but the biggest problem of CNN is the fixed filter_size field of view. On the one hand, it cannot model longer sequence information, and on the other hand, the hyperparameter adjustment of filter_size is also very cumbersome. The essence of CNN is to do the feature expression work of text, and the recurrent neural network (RNN, Recurrent Neural Network) is more commonly used in natural language processing, which can better express context information. Although CNN and RNN are used in the task of text classification methods, they have the disadvantage that they are not intuitive enough, and the interpretability is not good, especially when analyzing badcase.

SUMMARY OF THE INVENTION

The present application provides a text classification method, apparatus, computer equipment and storage medium.

A first aspect provides a text classification method, the method comprising:

Extract the target text data to be analyzed from the original text;

Preprocessing the target text data to obtain a word segmentation result of the target text data;

The word segmentation result is input into the trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result and based on the target word vector. vector, the target word vector and the target position vector to obtain the target classification label of the target text data; wherein, the text classification model is a trained alber model.

A second aspect provides a text classification device, including:

The target text acquisition module is used to extract the target text data to be analyzed from the original text;

A word segmentation module for preprocessing the target text data to obtain a word segmentation result of the target text data;

The classification module is used to input the word segmentation result into a trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result and The target classification label of the target text data is obtained based on the target word vector, the target word vector and the target position vector; wherein, the text classification model is a trained alber model.

A third aspect provides a computer device, including a memory and a processor, where computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, cause the processor to perform the following steps:

Extract the target text data to be analyzed from the original text;

The word segmentation result is input into the trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result and based on the target word vector. vector, the target word vector and the target position vector to obtain the target classification label of the target text data; wherein, the text classification model is a trained alber model. A fourth aspect provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps:

Extract the target text data to be analyzed from the original text;

The above text classification method, device, computer equipment and storage medium, the above text classification method, first, extract the target text data to be analyzed from the original text; secondly, preprocess the target text data to obtain the target text data. word segmentation result; then, the target word vector, target word vector and target position vector corresponding to the target text data are obtained based on the word segmentation result; finally, the word segmentation result is input into the trained text classification model, the text The classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result, and obtains the target text based on the target word vector, the target word vector and the target position vector The target classification label for the data. Therefore, the albert model is used to process the text data, and the obtained word vector sequence contains the text information and context information of the text data, so it integrates the full text semantic information, contains more comprehensive text information, and is more conducive to subsequent text classification , which helps to improve the accuracy of text classification and improve the classification effect.

Description of drawings

Fig. 1 is the implementation environment diagram of the text classification method provided in one embodiment;

2 is a block diagram of the internal structure of a computer device in one embodiment;

3 is a flowchart of a text classification method in one embodiment;

FIG. 4 is a structural block diagram of a text classification apparatus in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element.

FIG. 1 is an implementation environment diagram of the text classification method provided in one embodiment. As shown in FIG. 1 , the implementation environment includes a computer device 110 and a terminal 120 .

The computer device 110 is a text classification server, the terminal 120 is a text acquisition device to be classified, and has a text classification result output interface. When text classification is required, the text to be classified is obtained through the terminal 120, and the text to be classified is processed through the computer device 110. Classification.

It should be noted that the terminal 120 and the computer device 110 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but are not limited thereto. The computer device 110 and the terminal 110 may be connected through Bluetooth, USB (Universal Serial Bus, Universal Serial Bus) or other communication connection methods, which are not limited in this application.

FIG. 2 is a schematic diagram of the internal structure of a computer device in one embodiment. As shown in FIG. 2, the computer device includes a processor, a storage medium, a memory, and a network API interface connected through a system bus. Wherein, the storage medium of the computer device stores an operating system, a database and computer-readable instructions, the database can store a control information sequence, and when the computer-readable instructions are executed by the processor, the processor can be made to implement a text classification method . The processor of the computer device is used to provide computing and control capabilities and support the operation of the entire computer device. The computer device may have computer readable instructions stored in the memory that, when executed by the processor, cause the processor to perform a text classification method. The network API interface of the computer device is used to communicate with the terminal connection. Those skilled in the art can understand that the structure shown in FIG. 2 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

For the convenience of understanding, the terms involved in the embodiments of the present application are first described below.

albert model: A language model released by Google in 2018 that trains deep bidirectional representations by uniting bidirectional transformers in all layers. The albert model combines the advantages of many natural language processing models, and has achieved better results in many natural language processing tasks. In the related art, the model input vector of the albert model is the sum of the vectors of a word vector (TokenEmbedding), a position vector (PositionEmbedding) and a sentence vector (SegmentEmbedding). Among them, the word vector is the vectorized representation of the text, the position vector is used to represent the position of the word in the text, and the sentence vector is used to represent the sequence of sentences in the text.

Pre-training: A process in which the neural network model learns common features in the dataset by training the neural network model with a large dataset. The purpose of pre-training is to provide high-quality model parameters for subsequent neural network model training on a specific dataset. The pre-training in this embodiment of the present application refers to the process of training the albert model by using unlabeled training text.

Fine-tuning: A process of further training a pretrained neural network model using a specific dataset. Usually, the data volume of the data set used in the fine-tuning stage is smaller than that of the data set used in the pre-training stage, and the fine-tuning stage adopts a supervised learning method, that is, the training samples in the data set used in the fine-tuning stage contain annotation information. The fine-tuning stage in the embodiment of the present application refers to training the albert model by using the training text containing the classification labels.

Natural language processing is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between humans and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, the language that people use on a daily basis, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

As shown in FIG. 3, in one embodiment, a text classification method is proposed, and the text classification method can be applied to the above-mentioned computer device 110, and may specifically include the following steps:

Step 101, extracting target text data to be analyzed from the original text;

Among them, the original text can be a patent text, and the patent text has the characteristics of special structure, strong professionalism, and many field vocabulary, and needs to adopt a more targeted classification method. Patent text classification belongs to the field of natural language processing, and generally includes steps such as data preprocessing, text feature representation, classifier selection, and effect evaluation. Among them, text feature representation and classifier selection are the most important, which will directly affect the accuracy of classification results.

In this embodiment, the text data of the description abstract, the claims and the description title part in the patent text are extracted as the target text data.

Step 102, preprocessing the target text data to obtain the word segmentation result of the target text data;

In this embodiment, the purpose of preprocessing the target text data is to extract useful data in the original text data, or to delete the noise data in the original text, so that the text data irrelevant to the extraction purpose in the original text data can be deleted. .

In some embodiments, the above step 102 may include: performing one of stop word removal and duplication removal on the target text data to obtain second text data, and performing a word segmentation operation on the second text data to obtain a word segmentation result.

Among them, when deleting the noise data, duplicate data in the original text data is removed by deduplication; noise data in the original text data is removed by means of deletion, so that the noise data in the original text data can be removed.

Stop words refer to certain words or words that are automatically filtered out before or after processing natural language texts in order to save storage space and improve search efficiency in information retrieval. These words or words are called stop words (Stop words). Words).

In this embodiment, removing stop words can remove words in the natural language text that do not contribute to text features, such as punctuation, tone, person, meaningless garbled characters and spaces. The selected method for removing stop words is stop word list filtering. Stop word list filtering can be performed by matching the words in the text data one by one through the constructed stop word list. If the match is successful, then the word is Stop word, which needs to be deleted.

In order to obtain the target text data in the form of vectors, it is necessary to firstly perform word segmentation on the second text data. Word segmentation is a basic task in lexical analysis, and word segmentation algorithms are mainly divided into two categories according to their core ideas: one is dictionary-based word segmentation, which first divides text data into words according to the dictionary, and then finds the best combination of words; the other The first is word segmentation based on words, that is, words are formed from words. First, the sentence is divided into words, and then the words are combined into words to find the optimal segmentation strategy. At the same time, it can also be transformed into a sequence labeling problem. The word segmentation algorithm used when performing word segmentation in this embodiment may include: a rule-based word segmentation method, an understanding-based word segmentation method, or a statistics-based word segmentation method.

Among them, the rule-based word segmentation method (such as the word segmentation method based on string matching) is to match the Chinese character string to be analyzed with the entry in a "sufficiently large" dictionary according to a certain strategy. string, the match is successful (a word is recognized). Commonly used rule-based word segmentation methods include: forward maximum matching method (from left to right direction); reverse maximum matching method (from right to left direction); minimum segmentation (minimize the number of words cut out in each sentence). ). The forward maximum matching method is to separate a string, where the length of the separation is limited, and then match the separated substring with the words in the dictionary. If the match is successful, the next round of matching is performed until all strings are processed. Completed, otherwise remove a word from the end of the substring, and then match, and so on. The reverse maximum matching rule is similar to this forward maximum matching method.

The word segmentation method based on comprehension is to achieve the effect of word recognition by letting the computer simulate the human understanding of the sentence. The basic idea of comprehension-based word segmentation is to perform syntactic and semantic analysis at the same time as word segmentation, and to use syntactic and semantic information to deal with ambiguity. Statistical word segmentation method: From a formal point of view, words are stable combinations of words, so in the context, the more times adjacent words appear at the same time, the more likely it is to form a word. Therefore, the frequency or probability of adjacent co-occurrence of words can better reflect the credibility of the words. By calculating the frequency of the combinations of adjacent co-occurring words in the text data, their mutual occurrence information is calculated. The mutual information reflects the closeness of the combination relationship between Chinese characters. When the closeness is higher than a certain threshold, it can be considered that this group of characters may constitute a word. In practical applications, the statistical word segmentation system can use a basic word segmentation dictionary to perform string matching word segmentation, and at the same time use statistical methods to identify some new words, that is, combine string frequency statistics and string matching, which not only gives full play to the speed and efficiency of matching word segmentation It also takes advantage of the advantages of dictionary-free word segmentation combined with context recognition of new words and automatic disambiguation.

After the above word segmentation processing, the original text data is represented by a series of keywords, but the data in this text form cannot be directly processed by the subsequent classification algorithm, but should be converted into a numerical form, so these keywords need to be word vectorized. Form conversion to obtain the text data to be classified, which is in the form of text vectors.

Step 103, input the word segmentation result into the trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result, and based on the target word vector, target word vector and target The position vector obtains the target classification label of the target text data; wherein, the text classification model is a trained alber model.

Among them, the word segmentation results are input into the pre-trained text classification model. The text classification model is a pre-trained albert model. In order to enable the alber model to learn the contextual relationship of the text, it can also learn the mapping between pinyin and text. relationship, in the embodiment of the present application, when training the albert model, vectors of three degrees are used, that is, a word vector, a word vector, and a position vector.

Optionally, the word vector is obtained by converting the text by using a word vector (wordtovector, word2vec) model.

In this embodiment, step 103 may include the following steps:

Step 1031: Acquire a word vector corresponding to the text data according to the part-of-speech and location information of the text data. In this embodiment, position information is added to the text data using position coding, and the text data to which the position information is added is represented by an initial word vector; the part of speech of the text data is obtained, and the part of speech is converted into a part of speech vector; the initial word vector and the part of speech are combined The vectors are added to obtain the word vector corresponding to the text data.

Step 1032: Input the word vector into the albert model for data processing to obtain a word matrix of the text data.

Step 1033: Acquire a word vector sequence of the text data according to the word matrix. In this embodiment, the word matrix is used to predict whether the two sentences in the text data are upper and lower sentences, the masked words in the two sentences and the part-of-speech features of the masked words, and the part-of-speech features are normalized to obtain the word vector of the text data sequence.

It should be understood that the albert model used in this embodiment is a pre-trained model, so when processing text data, it is only necessary to input the text data into the pre-trained albert model to obtain its corresponding Sequence of word vectors.

Among them, in order to enable the alber model to realize text classification, a classifier needs to be set in the alber model. Optionally, the classification category and quantity of the classifier are related to the classification task to be implemented by the text classification model, and the classifier may be a multi-class classifier (such as a softmax classifier). The embodiment of the present application does not limit the specific type of the classifier.

In some embodiments, the above-mentioned text classification method, before extracting the text data to be classified from the original text, further includes:

Step 100a, extracting keywords in the original text, and forming a keyword set;

Step 100b, determine the word frequency-inverse document frequency of the keyword set in the corpus of each category based on the TF-IDF model;

Specifically, the text features matching the keywords in the text features of the corpus of this category are determined, and the word frequency-inverse document frequency of the matched text features is used as the word frequency-inverse document frequency of the keyword. Among them, according to punctuation marks such as periods, question marks, exclamation marks, and semicolons, the text in a corpus of a certain category is divided into several sentences, and the text features in each sentence are extracted. According to the extracted text features, a text feature library is established for each category. Under each category, count the frequency of each text feature. The inverse document frequency of each text feature is counted, that is, the natural logarithm of the quotient of the total number of categories and the number of categories containing the text feature, and under each category, the word frequency-inverse document frequency of each text feature is calculated separately.

Step 100c, based on the word frequency-inverse document frequency of the keyword set of the original text in the corpus of each category, determine the confidence that the original text belongs to each category;

Specifically, for each category, respectively perform the following operations: determine the number of times the keyword appears in the corpus of this category; The number of occurrences, determine the class conditional probability of the original text relative to the category; according to the class conditional probability of the original text relative to the category, determine the confidence that the text to be classified belongs to the category.

Step 100d, according to the confidence that the original text belongs to each category, determine the first-level classification label of the original text;

Specifically, among the confidence levels of the original text belonging to each category, the category with the highest confidence level is used as the first-level classification label of the text to be classified.

Step 100e: Match the first-level classification label with preset first-level classification label information, and determine whether to use a text classification model to perform text classification on the original text according to the matching result.

It can be understood that the target classification label obtained in the above steps 101 to 104 is the lowest classification label of the patent text. For example, the patent text has a third-level classification label, the first-level classification label has only one classification, and the second-level classification label has At least two, and at least two tertiary classification labels. Therefore, in this step, firstly use the TF-IDF model to perform the first-level classification label according to the keywords of the original file. If the preset first-level classification label of the first-level classification label of the patent file does not match, the original file does not need to be labeled. Classified, the initial classification label is a label that is artificially set at a higher level than the underlying classification label.

In some embodiments, the above-mentioned text classification method further includes: a pre-trained text classification model, and a pre-trained text classification model, including:

Step 1001, obtaining a first training sample set, the first training sample set includes a first training text, and the first training text includes a corresponding first classification label;

Optionally, the first training sample set is a specific data set related to text classification, wherein the training text includes a corresponding classification label, the classification label can be manually labeled, and the classification label belongs to the classification result of the text classification model. In an illustrative example, when a text classification model is used to classify patent texts, the classification labels include specific technical fields, such as cloud computing, image processing, and the like. The embodiment of the present application does not limit the specific content of the classification label.

Step 1002, based on the first training sample set, pre-training the albert model with the first classification label as the classification target to obtain an initial text classification model;

The above step 1002 may include:

Divide the first training sample set into training data and verification data according to a preset ratio;

Input the training data into the initial text classification model to be trained for model training;

The trained initial text classification model is verified based on the verification data, and the optimized initial text classification model is obtained according to the verification results.

In this step, the first training sample set is divided according to the ratio of 9:1, 90% is used as the training set, and 10% is used as the validation set. Prediction, and appropriately tune the model parameters according to the results to obtain the initial text classification model.

Step 1003, judging whether the accuracy of the classification result of the initial text classification model is greater than a preset threshold,

Step 1004, if it is greater than, then take the initial text classification model as the final text classification model;

Step 1005, if not greater than, correct the classification label corresponding to the first training text, and iterate the initial text classification model based on the error-corrected first training sample set, until the classification result of the initial text classification model is accurate. rate is greater than the preset threshold.

It can be understood that in step 1005, the initial text classification model is iterated based on the error-corrected first training sample set, that is, the initial text model can be optimized based on all or part of the error-corrected first training sample set. For the specific number of iterations, it is necessary to judge whether the accuracy of the classification result of the fine-tuned initial text classification model is greater than the preset threshold.

In the above step 1003, judging whether the accuracy rate of the classification result of the initial text classification model is greater than a preset threshold value may include:

1003a. Obtain a second training sample set, where the second training sample set includes a second training text;

1003b, based on the initial text classification model, obtain the predicted classification label corresponding to the second training text in the second training sample set;

1003c, according to the predicted classification label and the second classification label corresponding to the second training text, determine whether the accuracy rate of the classification result of the initial classification model is greater than a preset threshold, wherein the second classification label is the second classification label manually marked by the user .

In this embodiment, a second training sample set different from the first training sample set is used as the verification data for verifying the accuracy of the classification result of the initial text classification model. First, the training data of the initial classification model is expanded; The problem of low accuracy of the initial text classification model due to errors in the original classification labels of the first training sample set.

In the above step 1005, error correction is performed on the classification label corresponding to the first training text, which may include:

1005a. Review the prediction results, and obtain the first training text that is correctly predicted and the first training text that is wrongly predicted;

1005b. Manually label the first training text with the prediction error, so as to correctly label the label of the first training text with the prediction error.

In this embodiment, in view of the inaccuracy of the initial prediction of the initial text classification model, this embodiment iterates the model, so that the model prediction is more accurate.

In some embodiments, the computer device uses a gradient descent or backpropagation algorithm to adjust the network parameters of the albert model according to the error between the prediction result and the classification label, until the error satisfies the convergence condition.

In a possible implementation manner, since the pre-trained albert model has learned the context of the text, the data volume of the second training sample set used for fine-tuning is much smaller than the data volume of the first training sample set.

Similar to the pre-training process, in order to enable the text classification model to learn the mapping relationship between text classification and text pinyin, except that the word vector, position vector and sentence vector of the text in the second training text are used as input, the albert model is processed. Fine tune.

In a possible implementation, during the fine-tuning process, the computer device uses the second word vector, the second target word vector and the second target position vector of the second training sample set as the input vector of the albert model, and obtains the output of the albert model. Text classification prediction results, and then use the classification labels corresponding to the second training text as supervision, fine-tune the albert model, and finally train to obtain a text classification model.

As shown in Figure 4, in one embodiment, a text classification device is provided, and the text classification device can be integrated in the above-mentioned computer equipment 110, and can specifically include

The target text acquisition module 411 is used to extract the target text data to be analyzed from the original text;

The word segmentation module 412 is used to preprocess the target text data to obtain the word segmentation result of the target text data;

The vector obtaining module 413 is used to obtain the target word vector, target position vector and target sentence vector corresponding to the text in the target classification text;

The classification module 414 is used to input the target word vector, the target word vector and the target position vector into the text classification model, to obtain the target classification label output by the text classification model, and the target classification label is a text classification model using any one of claims 1 to 4. The text classification model trained by the training method.

In one embodiment, a computer device is proposed, the computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the following steps when executing the computer program: from the original text Extract the target text data to be analyzed; preprocess the target text data to obtain the word segmentation result of the target text data; obtain the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result; The word vector and the target position vector are input into the pre-trained text classification model, and the target classification label output by the text classification model is obtained, wherein the text classification model is a fine-tuned alber model.

In one embodiment, before extracting the text data to be classified from the original text, the method further includes: extracting keywords from the original text based on the TF-IDF model, and forming a keyword set; First-class classification label; the step of matching the first-class classification label with the preset first-class classification label information, and determining whether to use the text classification model to classify the original text according to the matching result.

In one embodiment, the original text is patent text data, and extracting the text data to be classified from the original text includes: extracting the text data of the description abstract, claims and description title parts in the patent text as the text data to be classified .

In one embodiment, inputting the word segmentation result into a pre-trained albert model to obtain a sequence of word vectors corresponding to the text data, including: obtaining word vectors corresponding to the text data according to part of speech and position information of the text data; inputting the word vectors to Data processing is performed in the albert model to obtain the word matrix of the text data; according to the word matrix, the word vector sequence of the text data is obtained.

In one embodiment, judging whether the accuracy rate of the classification result of the initial text classification model is greater than a preset threshold includes: acquiring a second training sample set, where the second training sample set includes the second training text; based on the initial text classification model, obtaining The predicted classification label corresponding to the second training text in the second training sample set; according to the predicted classification label and the second classification label corresponding to the second training text, determine whether the accuracy of the classification result of the initial classification model is greater than the preset threshold, wherein, The second classification label is the second classification label manually annotated by the user.

In one embodiment, pre-training the albert model with the first classification label as the classification target based on the first training sample set to obtain an initial text classification model includes: dividing the first training sample set into training data and Verify data; input the training data into the initial text classification model to be trained for model training; verify the trained initial text classification model based on the verification data, and obtain an optimized initial text classification model according to the verification results.

In one embodiment, performing error correction on the classification label corresponding to the first training text includes:

Review the prediction results, and obtain the first training text that is correctly predicted and the first training text that is wrongly predicted;

Manually label the first training text with the prediction error, so as to correctly label the label of the first training text with the prediction error.

In one embodiment, a storage medium is provided that stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: Extract the target text data to be analyzed; preprocess the target text data to obtain the word segmentation result of the target text data; obtain the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result; The word vector and the target position vector are input into the pre-trained text classification model, and the target classification label output by the text classification model is obtained, wherein the text classification model is a fine-tuned alber model.

Those of ordinary skill in the art can understand that the realization of all or part of the processes in the methods of the above embodiments can be accomplished by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the program is During execution, it may include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disc, a read-only memory (Read-Only Memory, ROM), or the like, or a volatile memory, that is, a random access memory (Random Access Memory, RAM). )Wait.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims

A text classification method, wherein the method comprises:

Extract the target text data to be analyzed from the original text;

Preprocessing the target text data to obtain a word segmentation result of the target text data;

The word segmentation result is input into the trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result and based on the target word vector. vector, the target word vector and the target position vector to obtain the target classification label of the target text data; wherein, the text classification model is a trained alber model.
The text classification method according to claim 1, wherein before extracting the text data to be classified from the original text, it further comprises:

Extracting keywords in the original text to be described, and forming a keyword set;

Determine the word frequency-inverse document frequency of the keyword set in the corpus of each category based on the TF-IDF model;

Determine the confidence that the original text belongs to each category based on the word frequency-inverse document frequency of the keyword set of the original text in the corpus of each category;

Determine the first-level classification label of the original text according to the confidence that the original text belongs to each category;

Matching the first-level classification label with preset first-level classification label information, and determining whether to use the text classification model to perform text classification on the original text according to the matching result.
The text classification method according to claim 1, wherein the preprocessing of the text data to obtain a word segmentation result comprises:

One of removing stop words and removing duplicates is performed on the target text data to obtain second text data, and a word segmentation operation is performed on the second text data to obtain a word segmentation result.
The text classification method according to claim 1, wherein the method further comprises: training the text classification model, and the training the text classification model comprises:

obtaining a first training sample set, the first training sample set includes a first training text, and the first training text includes a corresponding first classification label;

Based on the first training sample set, the albert model is pre-trained with the first classification label as a classification target to obtain an initial text classification model;

judging whether the accuracy of the classification result of the initial text classification model is greater than a preset threshold,

If it is greater than the preset threshold, the initial text classification model is used as the final text classification model;

If it is not greater than the preset threshold, perform error correction on the classification label corresponding to the first training text, and iterate the initial text classification model based on the error-corrected first training sample set until the initial The accuracy of the classification result of the text classification model is greater than the preset threshold.
The method for training a text classification model according to claim 4, wherein the judging whether the accuracy of the classification result of the initial text classification model is greater than a preset threshold comprises:

obtaining a second training sample set, where the second training sample set includes a second training text;

Based on the initial text classification model, obtain the predicted classification label corresponding to the second training text in the second training sample set;

According to the predicted classification label and the second classification label corresponding to the second training text, it is judged whether the accuracy of the classification result of the initial classification model is greater than a preset threshold, wherein the second classification label is manually Annotated second category label.
The method for training a text classification model according to claim 4, wherein, based on the first training sample set, the first classification label is used as a classification target to pre-train an albert model to obtain an initial text classification model, comprising:

Dividing the first training sample set into training data and verification data according to a preset ratio;

Inputting the training data into the initial text classification model to be trained for model training;

The trained initial text classification model is verified based on the verification data, and an optimized initial text classification model is obtained according to the verification result.
The method for training a text classification model according to claim 4, wherein the performing error correction on the classification label corresponding to the first training text comprises:

Reviewing the prediction results, and obtaining the first training text that is correctly predicted and the first training text that is wrongly predicted;

Manually label the first training text with the prediction error, so as to correctly label the label of the first training text with the prediction error.
A text classification device, comprising:

The target text acquisition module is used to extract the target text data to be analyzed from the original text;

A word segmentation module for preprocessing the target text data to obtain a word segmentation result of the target text data;

The classification module is used to input the word segmentation result into a trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result and The target classification label of the target text data is obtained based on the target word vector, the target word vector and the target position vector; wherein, the text classification model is a trained alber model.
A computer device, comprising a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, the processor is caused to perform the following steps:

Extract the target text data to be analyzed from the original text;

Preprocessing the target text data to obtain a word segmentation result of the target text data;

The word segmentation result is input into the trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result and based on the target word vector. vector, the target word vector and the target position vector to obtain the target classification label of the target text data; wherein, the text classification model is a trained alber model.
The computer device of claim 9, wherein the computer-readable instructions, when executed by the processor, further cause the processor to perform the following steps:

Extracting keywords in the original text to be described, and forming a keyword set;

Determine the word frequency-inverse document frequency of the keyword set in the corpus of each category based on the TF-IDF model;

Determine the confidence that the original text belongs to each category based on the word frequency-inverse document frequency of the keyword set of the original text in the corpus of each category;

Determine the first-level classification label of the original text according to the confidence that the original text belongs to each category;

Matching the first-level classification label with preset first-level classification label information, and determining whether to use the text classification model to perform text classification on the original text according to the matching result.
The computer device of claim 9, wherein the computer-readable instructions, when executed by the processor, further cause the processor to perform the following steps:

Training the text classification model, the training of the text classification model includes:

obtaining a first training sample set, the first training sample set includes a first training text, and the first training text includes a corresponding first classification label;

Based on the first training sample set, the albert model is pre-trained with the first classification label as a classification target to obtain an initial text classification model;

judging whether the accuracy of the classification result of the initial text classification model is greater than a preset threshold,

If it is greater than the preset threshold, the initial text classification model is used as the final text classification model;

If it is not greater than the preset threshold, perform error correction on the classification label corresponding to the first training text, and iterate the initial text classification model based on the error-corrected first training sample set until the initial The accuracy of the classification result of the text classification model is greater than the preset threshold.
The computer device according to claim 11, wherein, when the processor performs the judging whether the accuracy of the classification result of the initial text classification model is greater than a preset threshold, the process specifically includes:

obtaining a second training sample set, where the second training sample set includes a second training text;

Based on the initial text classification model, obtain the predicted classification label corresponding to the second training text in the second training sample set;

According to the predicted classification label and the second classification label corresponding to the second training text, it is judged whether the accuracy of the classification result of the initial classification model is greater than a preset threshold, wherein the second classification label is manually Annotated second category label.
The computer device according to claim 11, wherein, when the processor performs the pre-training of the albert model based on the first training sample set and using the first classification label as a classification target to obtain an initial text classification model, Specifically include:

dividing the first training sample set into training data and verification data according to a preset ratio;

Inputting the training data into the initial text classification model to be trained for model training;

The trained initial text classification model is verified based on the verification data, and an optimized initial text classification model is obtained according to the verification result.
The computer device according to claim 11, wherein, when the processor performs the error correction for the classification label corresponding to the first training text, it specifically includes:

Reviewing the prediction results, and obtaining the first training text that is correctly predicted and the first training text that is wrongly predicted;

Manually label the first training text with the prediction error, so as to correctly label the label of the first training text with the prediction error.
A storage medium storing computer-readable instructions, wherein the computer-readable instructions, when executed by one or more processors, cause the one or more processors to perform the following steps:

Extract the target text data to be analyzed from the original text;

Preprocessing the target text data to obtain a word segmentation result of the target text data;

The word segmentation result is input into the trained text classification model, and the text classification model obtains the target word vector, target word vector and target position vector corresponding to the target text data based on the word segmentation result and based on the target word vector. vector, the target word vector and the target position vector to obtain the target classification label of the target text data; wherein, the text classification model is a trained alber model.
The storage medium storing computer-readable instructions according to claim 15, wherein, when executed by one or more processors, the computer-readable instructions further cause one or more processors to perform the following steps:

Extracting keywords in the original text to be described, and forming a keyword set;

Determine the word frequency-inverse document frequency of the keyword set in the corpus of each category based on the TF-IDF model;

Determine the confidence that the original text belongs to each category based on the word frequency-inverse document frequency of the keyword set of the original text in the corpus of each category;

Determine the first-level classification label of the original text according to the confidence that the original text belongs to each category;

Matching the first-level classification label with preset first-level classification label information, and determining whether to use the text classification model to perform text classification on the original text according to the matching result.
The storage medium storing computer-readable instructions according to claim 15, wherein, when executed by one or more processors, the computer-readable instructions further cause one or more processors to perform the following steps:

Training the text classification model, the training of the text classification model includes:

obtaining a first training sample set, the first training sample set includes a first training text, and the first training text includes a corresponding first classification label;

Based on the first training sample set, the albert model is pre-trained with the first classification label as a classification target to obtain an initial text classification model;

judging whether the accuracy of the classification result of the initial text classification model is greater than a preset threshold,

If it is greater than the preset threshold, the initial text classification model is used as the final text classification model;

If it is not greater than the preset threshold, perform error correction on the classification label corresponding to the first training text, and iterate the initial text classification model based on the error-corrected first training sample set until the initial The accuracy of the classification result of the text classification model is greater than the preset threshold.
18. The storage medium storing computer-readable instructions of claim 17, wherein the computer-readable instructions are executed by one or more processors to cause the one or more processors to perform the determining the initial text classification When the accuracy of the classification result of the model is greater than the preset threshold, it specifically includes:

obtaining a second training sample set, where the second training sample set includes a second training text;

Based on the initial text classification model, obtain the predicted classification label corresponding to the second training text in the second training sample set;

According to the predicted classification label and the second classification label corresponding to the second training text, it is judged whether the accuracy of the classification result of the initial classification model is greater than a preset threshold, wherein the second classification label is manually Annotated second category label.
18. The storage medium having computer-readable instructions stored thereon of claim 17, wherein the computer-readable instructions are executed by one or more processors to cause the one or more processors to perform the training based on the first training The sample set, when the albert model is pre-trained with the first classification label as the classification target, and the initial text classification model is obtained, it specifically includes:

dividing the first training sample set into training data and verification data according to a preset ratio;

Inputting the training data into the initial text classification model to be trained for model training;

The trained initial text classification model is verified based on the verification data, and an optimized initial text classification model is obtained according to the verification result.
18. The storage medium storing computer-readable instructions of claim 17, wherein the computer-readable instructions are executed by one or more processors to cause the one or more processors to perform the pairing of the first training When correcting the error of the classification label corresponding to the text, it specifically includes:

Reviewing the prediction results, and obtaining the first training text that is correctly predicted and the first training text that is wrongly predicted;

Manually label the first training text with the prediction error, so as to correctly label the label of the first training text with the prediction error.