CN105975497A - Automatic microblog topic recommendation method and device - Google Patents

Abstract

The invention discloses a microblog topic automatic recommendation method and device, wherein the method includes: processing the text content of the microblog based on a neural network model to obtain a feature vector; Classification is performed to obtain topic categories; according to topic categories, topics are automatically recommended for microblogs that do not contain topics. The method can help users and the microblog platform manage massive microblog content. The invention also provides a microblog topic automatic recommendation device.

Description

Microblog topic automatic recommendation method and device

technical field

The invention relates to the fields of computer application technology and social network, in particular to a microblog topic automatic recommendation method and device.

Background technique

Text representation is a crucial step in tasks such as web page retrieval, information filtering, sentiment analysis, etc. In traditional machine learning methods, it usually appears in the form of feature representation. The most commonly used feature representation method in text learning is the bag-of-words model. In the bag of words model, the most commonly used features are words, binary phrases, multivariate phrases (n-grams), and some manually extracted template features. After representing text in the form of features, traditional models often use word frequency, mutual information, PLSA (Probability Latent Semantic Analysis, probabilistic latent semantic analysis), LDA (Latent Dirichlet Allocation, document topic generation model) and other methods to screen out the most effective features. However, traditional methods ignore context information and lose word order information when representing text.

In recent years, pre-trained word vectors and deep neural network models have brought new ideas to natural language processing. With the help of word vectors, some methods of combining semantics have been proposed to represent the semantics of text. Recurrent neural networks can construct the semantics of text in O(n) time. The model processes the entire document word by word and saves all the above semantics into a fixed-size hidden layer. The advantage of RNN is that it can better capture contextual information and model long-distance contextual information. However, the recurrent neural network is a biased model. For example, for the forward recurrent neural network, the later words in the text occupy a more dominant position than the earlier words. Due to the nature of this semantic bias, when the RNN constructs the semantics of the entire text, it will include more information about the latter part of the text. But not all texts are actually focused on the end, which may affect the precision of the semantic representation it generates.

In order to solve the problem of semantic bias, it was proposed to use convolutional neural network to construct text semantics. The convolutional neural network uses the maximum pooling technique to find the most useful text fragments from the text, and its complexity is also O(n). Therefore, convolutional neural networks have greater potential in constructing text semantics. However, existing convolutional neural network models always use relatively simple convolution kernels, such as fixed windows. How to determine the window size is a key issue when using such models. When the window is too small, it may lead to insufficient retention of context information, and it is difficult to accurately characterize words; and when the window is too large, it will lead to too many parameters and increase the difficulty of model optimization. Therefore, it is necessary to consider how to build a model in order to better capture contextual information and reduce the difficulty of choosing the window size.

Research on topics in short texts has been attracting much attention, and how to effectively and preparatively classify text information into topics is a problem that needs to be solved.

Contents of the invention

The object of the present invention is to solve one of the above-mentioned technical problems at least to a certain extent.

For this reason, the first object of the present invention is to propose a method for automatic recommendation of microblog topics. The method can help users and the microblog platform manage massive microblog content.

The second object of the present invention is to propose a microblog topic automatic recommendation device.

In order to achieve the above purpose, the microblog topic automatic recommendation method of the first embodiment of the present invention includes: processing the text content of the microblog based on a neural network model to obtain a feature vector; The text content of microblogs is classified to obtain topic categories; according to the topic categories, topics are automatically recommended for microblogs that do not contain topics.

The microblog topic automatic recommendation method in the embodiment of the present invention first processes the text content of the microblog based on the neural network model to obtain a feature vector, then uses the softmax classifier to classify the text content of the microblog according to the feature vector to obtain the topic category, and finally Automatically recommend topics for microblogs that do not contain topics according to topic categories. The method can help users and the microblog platform manage massive microblog content.

In some examples, the neural network model includes: a convolutional neural network model and a recurrent neural network model.

In some examples, the processing the text content of the microblog based on the neural network model to obtain the feature vector specifically includes: removing messy information in the text content of the microblog, and removing useless stop words according to the stop word list Obtain new text content; obtain the local feature of each basic unit in the sentence by performing a convolution operation on the sentence of the new text content, and perform a maximization operation on the local feature to obtain the feature of the sentence vector; finally, use the cyclic neural network to process the feature vector of the sentence to obtain the feature vector of the text content of the microblog.

In some examples, the messy information includes: @ information, URL information and picture information.

In order to achieve the above-mentioned purpose, the microblog topic automatic recommendation device of the second aspect embodiment of the present invention includes: a processing module, which is used to process the text content of the microblog based on a neural network model to obtain a feature vector; a classification module, through a softmax classifier The text content of the microblog is classified according to the feature vector to obtain a topic category; an automatic recommendation module is configured to automatically recommend topics for microblogs that do not contain topics according to the topic category.

In the microblog topic automatic recommendation device of the embodiment of the present invention, first, the processing module processes the text content of the microblog based on the neural network model to obtain a feature vector, and then the classification module classifies the text content of the microblog according to the feature vector through a softmax classifier to obtain Topic categories, and finally the automatic recommendation module automatically recommends topics for microblogs that do not contain topics according to topic categories. The device can help users and microblog platforms manage massive microblog contents.

In some examples, the neural network model includes: a convolutional neural network model and a recurrent neural network model.

In some examples, the processing module is specifically configured to: remove the messy information in the text content of the microblog, and remove useless stop words according to the stop word list to obtain new text content; The sentence of text content is carried out convolution operation to obtain the local feature of each basic unit in the described sentence, and is carried out maximization operation to described local feature to obtain the feature vector of described sentence; The feature vector is processed to obtain the feature vector of the text content of the microblog.

In some examples, the messy information includes: @ information, URL information and picture information.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and comprehensible from the description of the embodiments in conjunction with the following drawings, wherein:

Fig. 1 is the flow chart of the microblog topic automatic recommendation method according to one embodiment of the present invention;

2 is a schematic diagram of a convolutional layer of a convolutional neural network model according to an embodiment of the present invention;

Fig. 3 is the flow chart that the recurrent neural network model constructs text semantics according to one embodiment of the present invention;

FIG. 4 is a schematic diagram of feature vectors of text content of microblogs according to an embodiment of the present invention;

Fig. 5 is a flowchart of a microblog topic automatic recommendation method according to a specific embodiment of the present invention; and

Fig. 6 is a schematic diagram of an apparatus for automatically recommending microblog topics according to an embodiment of the present invention.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

Fig. 1 is a flow chart of a method for automatically recommending microblog topics according to an embodiment of the present invention.

As shown in Figure 1, the microblog topic automatic recommendation method may include:

S101, process the text content of the microblog based on the neural network model to obtain a feature vector.

It should be noted that, in some examples, the neural network model may include: a convolutional neural network model and a recurrent neural network model.

It is understandable that the high fault tolerance and highly nonlinear description ability of neural network make it widely researched and applied, and the convolutional neural network is the best among all kinds of neural network models. Convolutional neural network is a multi-layer neural network, each layer is composed of multiple two-dimensional planes, and each plane is composed of multiple independent neurons. The network contains a calculation layer and a feature extraction layer. Generally, the input of each neuron is connected to the local domain of the previous layer, and the local features are extracted. Once the local features are extracted, the positional relationship between it and other features is determined; each calculation layer It consists of multiple feature maps, each feature map is a plane, and the weights of neurons on each plane are the same. Since neurons on each mapping surface share weights, the number of free parameters of the network is reduced, thereby reducing the time complexity of network parameter selection. The output connection value of neurons in the network conforms to the "maximum detection hypothesis", that is, in the set of neurons existing in a small area, only the neuron with the largest output will strengthen the output value. According to the hypothesis, only one neuron will be strengthened. The unit of the convolutional neural network is the maximum output unit, and also controls the reinforcement results of the neighboring units. In addition to the input and output layers, the convolutional neural network also has a convolutional layer, a sampling layer, and a fully connected layer. There are several feature maps in the convolutional layer and the sampling layer, and each layer has multiple planes, which are constantly corrected during training. neuron weights. The weights of neurons in the same plane are the same, so they can have the same degree of displacement and rotation invariance. Since the weights are shared, the mapping from one plane to the next can be seen as a convolution operation. The spatial resolution decreases between the hidden layer and the hidden layer, and the number of planes contained in each layer increases, which can be used to detect more feature information. In the convolution layer, the feature map of the previous layer is convolved with a learnable kernel, and the output of the convolution result after passing through the activation function forms the neurons of this layer, thus forming the feature map of this layer. For example, the convolutional layer is shown in Figure 2. Convolutional neural networks can achieve displacement, scaling, and distortion invariance through three methods, namely local receptive fields, weight sharing, and subsampling. The local receptive field means that the neurons of each layer of the network are only connected to the neurons in a small neighborhood of the previous layer. Through the local receptive field, each neuron can extract primary features; weight sharing makes volume Productive neural networks have better parameters and require relatively less training data and time; subsampling reduces the resolution of features and achieves invariance to displacement, scaling, and other forms of distortion.

In addition, recurrent neural networks can construct the semantics of text in O(n) time. The model processes the entire document word by word and saves all the above semantics into a fixed-size hidden layer. The advantage of the cyclic neural network is that it can better capture contextual information and model long-distance contextual information. However, the recurrent neural network is a biased model. For example, for the forward recurrent neural network, the later words in the text occupy a more dominant position than the earlier words. Due to the nature of this semantic bias, when the RNN constructs the semantics of the entire text, it will include more information about the latter part of the text. When obtaining the semantic representation of sentences and documents, it is easy to think of directly following the distribution hypothesis of words to model the documents. However, if the distribution hypothesis is used to directly generate the vector representation of sentences or documents, it will encounter a huge data sparsity problem. If the sentence is regarded as a whole and the word vector model is used to train the representation of the sentence, since most sentences appear only once, the training results will be statistically meaningless. On the other hand, the distribution hypothesis is a hypothesis for word meaning. Whether this method of acquiring semantics through context is effective for sentences and documents remains to be discussed. Therefore, it is necessary to find new ideas to model sentences and documents, and the recurrent neural network is a very good model. Recurrent neural networks were first proposed by Elman et al. in 1990. The core of the model is to input each word in the text one by one in a cyclic manner, and maintain a hidden layer to retain all the above information. Recurrent neural network is a special case of recursive neural network, which can be considered as corresponding to a tree whose right subtree of any non-leaf node is a leaf node. This special structure makes the cyclic neural network have two characteristics: First, due to the fixed network structure, the model can construct the semantics of the text only in O(n) time. This allows recurrent neural networks to more efficiently model the semantics of text. 2. From the perspective of network structure, the number of layers of the recurrent neural network is very deep. There are several words in a sentence, and there are several layers in the network. Therefore, when using traditional methods to train recurrent neural networks, you will encounter the problem of gradient decay or gradient explosion, which requires the model to use more special methods to achieve the optimization process. The construction process of the text semantics by the cyclic neural network is shown in Figure 3: each word is combined with the hidden layer representing all the above to form a new hidden layer, which is calculated cyclically from the first word to the last word of the text. When the model inputs all the words, the hidden layer corresponding to the last word represents the semantics of the entire text. In terms of optimization methods, the cyclic neural network is also slightly different from other network structures. In ordinary neural networks, the backpropagation algorithm can be directly calculated using the chain rule of derivatives. However, in the recurrent neural network, since the weight matrix H from the hidden layer to the next hidden layer is multiplexed, it is very difficult to directly derive the weight matrix. The simplest optimization method of cyclic neural network is backpropagation along time. This method first expands the network, and for each labeled sample, the model updates the hidden layer one by one through the backpropagation technology of the ordinary network, and repeatedly updates the weight matrix H in it. Due to the problem of gradient decay, when using BPTT to optimize the recurrent neural network, only a fixed number of layers is propagated. In order to solve the gradient decay problem, Hochreiter and Schmidhuber proposed the LSTM model in 1997. The model introduces memory units, which can store long-distance information, and is a commonly used optimization scheme for recurrent neural networks.

Specifically, in some examples, processing the text content of the microblog based on the neural network model to obtain the feature vector specifically includes: removing messy information in the text content of the microblog, and removing useless stop words according to the stop word list to obtain new text content. The local feature of each basic unit in the sentence is obtained by performing a convolution operation on the sentence of the new text content, and the feature vector of the sentence is obtained by maximizing the local feature. Finally, the eigenvector of the sentence is processed by the recurrent neural network to obtain the eigenvector of the microblog text content. Among them, the feature vector of the text content of microblog is shown in Fig. 4 .

More specifically, in some examples, the miscellaneous information includes: @ information, URL information and picture information. It is understandable that the cluttered information in the microblog text, such as @ information, URL information, picture information, etc., is removed, and then the Chinese microblog content is word-segmented, and useless stop words are removed according to the Chinese stop word list.

It should be noted that the useful information of microblog text data refers to microblog content.

Among them, the local feature of each basic unit in the sentence is obtained by performing a convolution operation on the sentence of the new text content, and the feature vector of the sentence is obtained by maximizing the local feature. It is understandable that sentence-level vector representation learning is performed on Weibo content. Given a sentence x containing N basic units (r ₁ , r ₂ , ..., r _N ), the basic unit of a word-level sentence is a single character, and the basic unit of a word-level sentence is a word after word segmentation. When computing sentence-level features, two main problems are encountered: different sentences have different lengths, and important information can appear anywhere in the sentence. Using the convolutional layer to model sentences and calculate sentence-level features can solve the two problems mentioned above. Through the convolution operation, the local features of each basic unit id in the sentence can be obtained, and then the obtained local features are maximized to obtain a fixed-length sentence feature vector. In a sentence x containing N basic units (r ₁ , r ₂ , ..., r _N ), the convolutional layer performs matrix-vector operations on each consecutive window of size k. The size k of the convolution window is different, and the local information obtained is also different. Learning by setting an appropriate size of k in the early stage of the experiment. Concatenate the sentence feature vectors generated by all convolutional layers to obtain a new sentence feature vector.

S102. Classify the text content of the microblog according to the feature vector by using a softmax classifier to obtain topic categories.

It should be noted that the classifier may be but not limited to a softmax classifier.

S103. Automatically perform topic recommendation for microblogs that do not contain topics according to topic categories.

The microblog topic automatic recommendation method in the embodiment of the present invention first processes the text content of the microblog based on the neural network model to obtain a feature vector, then uses the softmax classifier to classify the text content of the microblog according to the feature vector to obtain the topic category, and finally Automatically recommend topics for microblogs that do not contain topics according to topic categories. The method can help users and the microblog platform manage massive microblog content.

For example, as shown in Figure 5: based on the automatic recommendation method for microblog topics based on recurrent neural network in the present invention, the present invention has developed a set of automatic recommendation system for Sina microblog topics. The system recommends new microblog content published by users, including two stages: first, the automatic preprocessing stage of the system, which cleans the original microblog content, and then uses convolutional neural network and recurrent neural network to obtain microblog level The second is the topic recommendation stage of the system. The system calls the trained softmax classification model to use the vector representation of Weibo as a feature to classify topics, and recommend topic categories to users. The recommendation results of the system can help users and the microblog platform effectively manage massive microblog data.

In order for people in the field to better understand the automatic recommendation method of microblog topics, the following is a specific description in conjunction with Figure 6: Facing a new microblog, first use the convolutional neural network to obtain the sentence-level vector representation, and then further use the cyclic neural network to learn the microblog Level vector representation, and then use the trained model to classify topics, and automatically recommend topic categories to users.

The microblog topic automatic recommendation method in the embodiment of the present invention first processes the text content of the microblog based on the neural network model to obtain a feature vector, then uses the softmax classifier to classify the text content of the microblog according to the feature vector to obtain the topic category, and finally Automatically recommend topics for microblogs that do not contain topics according to topic categories. The method can help users and the microblog platform manage massive microblog content.

Corresponding to the microblog topic automatic recommendation method provided by the above-mentioned embodiment, an embodiment of the present invention also provides a microblog topic automatic recommendation device, because the microblog topic automatic recommendation device provided by the embodiment of the present invention is the same The microblog topic automatic recommendation method provided has the same or similar technical features, so the implementation of the aforementioned microblog topic automatic recommendation method is also applicable to the microblog topic automatic recommendation device provided in this embodiment, and is no longer used in this embodiment A detailed description. As shown in FIG. 6 , the apparatus for automatically recommending microblog topics may include: a processing module 110 , a classification module 120 , and an automatic recommendation module 130 .

Wherein, the processing module 110 is configured to process the text content of the microblog based on the neural network model to obtain a feature vector.

The classification module 120 uses a softmax classifier to classify the microblog text content according to the feature vector to obtain topic categories.

The automatic recommendation module 130 is used for automatically recommending topics to microblogs that do not contain topics according to topic categories.

In some examples, the neural network models include: convolutional neural network models and recurrent neural network models.

In some examples, the processing module 110 is specifically used to: remove the messy information in the text content of the microblog, and remove useless stop words according to the stop word table to obtain new text content; The convolution operation obtains the local features of each basic unit in the sentence, and maximizes the local features to obtain the feature vector of the sentence; finally, the eigenvector of the sentence is processed by the cyclic neural network to obtain the feature vector of the text content of Weibo .

In some examples, the messy information includes: @ information, URL information and picture information.

In the microblog topic automatic recommendation device of the embodiment of the present invention, first, the processing module processes the text content of the microblog based on the neural network model to obtain a feature vector, and then the classification module classifies the text content of the microblog according to the feature vector through a softmax classifier to obtain Topic categories, and finally the automatic recommendation module automatically recommends topics for microblogs that do not contain topics according to topic categories. The device can help users and microblog platforms manage massive microblog contents.

In the description of the present invention, it should be understood that the terms "first" and "second" are used for description purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.

Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. During execution, one or a combination of the steps of the method embodiments is included.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (8)

1. a microblog topic auto recommending method, it is characterised in that including:

Based on neural network model, the content of text of microblogging is carried out process and obtain characteristic vector；

According to described characteristic vector, the content of text of described microblogging is carried out classification by softmax grader and obtain topic classification；

Automatically the microblogging not containing topic is carried out topic recommendation according to described topic classification.

2. microblog topic auto recommending method as claimed in claim 1, it is characterised in that described neural network model includes: Convolutional neural networks model and Recognition with Recurrent Neural Network model.

3. microblog topic auto recommending method as claimed in claim 1, it is characterised in that described based on neural network model The content of text of microblogging carries out process obtain characteristic vector and specifically include:

Remove the gibberish in the content of text of described microblogging, and obtain new according to disabling the vocabulary useless stop words of removal Content of text；

The local of each elementary cell in described sentence is obtained by the sentence of described new content of text is carried out convolution operation Feature, and described local feature is carried out maximum operation obtain the characteristic vector of described sentence；

Finally utilize Recognition with Recurrent Neural Network that the characteristic vector of described sentence is processed the spy of the content of text obtaining described microblogging Levy vector.

4. microblog topic auto recommending method as claimed in claim 3, it is characterised in that described gibberish includes :@ Information, URL information and pictorial information.

5. the automatic recommendation apparatus of microblog topic, it is characterised in that including:

Processing module, obtains characteristic vector for the content of text of microblogging being carried out process based on neural network model；

Sort module, carries out classification according to described characteristic vector to the content of text of described microblogging by softmax grader and obtains Topic classification；

Automatically recommending module, for automatically carrying out topic recommendation to the microblogging not containing topic according to described topic classification.

6. the automatic recommendation apparatus of microblog topic as claimed in claim 5, it is characterised in that described neural network model includes: Convolutional neural networks model and Recognition with Recurrent Neural Network model.

7. the automatic recommendation apparatus of microblog topic as claimed in claim 5, it is characterised in that described processing module specifically for:

Remove the gibberish in the content of text of described microblogging, and obtain new according to disabling the vocabulary useless stop words of removal Content of text；

The local of each elementary cell in described sentence is obtained by the sentence of described new content of text is carried out convolution operation Feature, and described local feature is carried out maximum operation obtain the characteristic vector of described sentence；

Finally utilize Recognition with Recurrent Neural Network that the characteristic vector of described sentence is processed the spy of the content of text obtaining described microblogging Levy vector.

8. the automatic recommendation apparatus of microblog topic as claimed in claim 7, it is characterised in that described gibberish includes :@ Information, URL information and pictorial information.