Integrating learned and explicit document features for reputation monitoring in social media

Currently, monitoring reputation in social media is probably one of the most lucrative applications of information retrieval methods. However, this task poses new challenges due to the dynamicity of contents and the need for early detection of topics that affect the reputations of companies. Addressing this problem with learning mechanisms that are based on training data sets is challenging, given that unseen features play a crucial role. However, learning processes are necessary to capture domain features and dependency phenomena. In this work, based on observational information theory, we define a document representation framework that enables the combination of explicit text features and supervised and unsupervised signals into a single representation model. Our theoretical analysis demonstrates that the observation information quantity (OIQ) generalizes the most popular representation methods, in addition to capturing quantitative values, which is required for integrating signals from learning processes. In other words, the OIQ allows us to give the same treatment to features that are currently managed separately. Empirically, our experiments on the reputation-monitoring scenario demonstrated that adding features progressively from supervised (in particular, Bayesian inference over annotated data) and unsupervised learning methods (in particular, proximity to clusters) increases the similarity estimation performance. This result is verified under various similarity criteria (pointwise mutual information, Jaccard and Lin’s distances and the information contrast model). According to our formal analysis, the OIQ is the first representation model that captures the informativeness (specificity) of quantitative features in the document representation.

1. For instance, just considering the occurrence of a few words as text features is enough to obtain an empty result in a standard web search engine.

2. Note that considering words as information pieces is not equivalent than considering words as features, due to the effect of repeated words:

   $$\begin{aligned} {\mathcal {I}}\big (\{w_1,w_2,w_2\}\big )= & {} -\hbox {log}\big (P_{d\in \mathcal {D}}(tf(d,w_1)\ge 1)\cdot P_{d\in \mathcal {D}}(tf(d,w_2)\ge 2)\big )\ne \\\ne & {} -\hbox {log}\big (P(w_1)P(w_2)P(w_2)\big ). \end{aligned}$$

3. That is, pairs with equal similarity in the measure count a half in the probability estimation.

We thank the anonymous reviewers for their very useful comments, which have added value to the manuscript. The work was supported by the Ministerio de Economía y Competitividad, TIN Program (Vemodalen), under Grant Number: TIN2015-71785-R.

Authors and Affiliations

1. National Distance University ETSI - UNED, C/ Juan del Rosal, 16, 28040, Madrid, Spain

   Fernando Giner, Enrique Amigó & Felisa Verdejo

Corresponding author

Correspondence to Enrique Amigó.

Conflict of interest

The authors declare that they have no conflict of interest.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: formal proofs

Proposition 3.1

The proof is straightforward. According to the fuzzy set operators:

where

\(\square \)

Property 3.1

From \({\gamma }(d_1) \ge {\gamma }(d_2), \ \ \forall \gamma \in \Gamma \), it follows:

Then,

This implies that

And therefore, according to Definition 3.3:

\(\square \)

Property 3.2

Notice that if we add a feature, the new observation is more restrictive than the initial observation, and thus, the set of messages which verify the new observation is contained in the set of messages which verify the initial observation, \({\mathcal {O}}_{\Gamma \cup \{\gamma '\}}(d) \subseteq {\mathcal {O}}_{\Gamma }(d)\). Then,

And therefore, according to Definition 3.3:

\(\square \)

Property 3.3

By Proposition 3.1:

Given that

we finally get, \( {\mathcal {I}}\big ({\mathcal {O}}_{\Gamma }(d_1)\cup {\mathcal {O}}_{\Gamma }(d_2)\big )\ge {\mathcal {I}}\big ({\mathcal {O}}_{\Gamma }(d_1)\big )\). Similarly, we can get the same result for \(d_2\). \(\square \)

Property 3.4

By hypothesis,

which is equivalent to

\(\square \)

Property 3.5

Consider two features, \(\gamma _1\), \(\gamma _2 \in \Gamma \), given a message, \(d \in \mathcal {D}\), it produces an observation under \(\gamma _1\), \({\mathcal {O}}_{\gamma _1}(d)\), whose Observation Information Quantity is:

Given that g is a strict monotonic function,

\(\square \)

Property 3.6

Assume that we have a finite set of messages, the proof of this proposition is a direct consequence of the representativity of the messages by the features. If we have an infinite set of features, then they will describe every message, and messages will be unequivocally determined by the values of a set of features. \(\square \)

Property 3.7

Given a fixed message, \(d \in \mathcal {D}\), consider all the messages, \(d' \in \mathcal {D}\), which verify the inequalities:

These inequalities are equivalent to (by definition of \(\gamma ^{-1}\)):

Notice that \({\gamma }(d)\) and \({\gamma }(d')\) are non-negative numbers; therefore, these inequalities imply that: \({\gamma }(d) = {\gamma }(d')\). Then, the Observation Information Quantity is:

which is equivalent to:

\(\square \)

Proposition 5.1

Given the vocabulary, \(\mathcal {V} = \{w_1, \ldots , w_n\}\), consider the set of features as \(\Gamma = \{occ_{w_1},\ldots ,occ_{w_n}\}\), and given a message from the collection, \(d \in \mathcal {D}\), we are interested in computing the described OIQ, \({\mathcal {I}}_{occ_{w_i}}\big (d\big )\).

Assuming information additivity and considering text words as basic linguistic units, we have

Notice that, if \(w_j \ne w_i\), then \({occ_{w_i}}(w_{j}) = 0\). Thus, \(P\big ({occ_{w_i}}(w') \ge 0 \big ) = 1\), since by definition \({occ_{w_i}}(d) \ge 0\), \(\forall d \in \mathcal {D}\). Therefore, in the last summation all the terms are null, except for \(w_{j} = w_{i}\). In this case, we have that \({occ_{w_i}}(w_i) = 1\), and given that by definition of the function \({occ_{w_i}}(.)\), its maximum value is 1, we can say that \({occ_{w_i}}(w') \ge 1\) is equivalent to \({occ_{w_i}}(w') = 1\). Therefore, the probability \(P\big ({occ_{w_i}}(w') = 1 \big )\) is exactly \(P(w' = w_i) = P(w_i)\). And, \({\mathcal {I}}_{occ_{w_i}}\big (d\big ) \propto - \log \big ( P(w_i) \big )\).

One of the assumptions is that every word is equiprobable, i.e. \(P(w_i) = k\), \(1 \le i \le n\), for an arbitrary k. In order to achieve the result, we can choose k in such a way that \(- \log (k) = 1\). And finally, the summation gives us the \(tf(w_i, d)\). \(\square \)

Proposition 5.2

Given the vocabulary, \(\mathcal {V} = \{w_1, \ldots , w_n\}\), considering the set of features as, \(\Gamma = \{occ_{w_1},\ldots ,occ_{w_n}\}\), and given a message from the collection, \(d \in \mathcal {D}\), we are interested in computing the described OIQ, \({\mathcal {I}}_{occ_{w_i}}\big (d\big )\).

Assuming information additivity and considering messages as basic linguistic units, we have

Notice that, if \(w_{j} \ne w_{i}\), then \({occ_{w_i}}(w_j) = 0\). Thus, \(P_{d' \in \mathcal {D}}\big ({occ_{w_i}}(d') \ge 0 \big ) = 1\), since by definition \({occ_{w_i}}(d') \ge 0\), \(\forall d' \in \mathcal {D}\). Therefore, in the last summation all the terms are null, except for \(w_{j} = w_{i}\). In this case, we have as many terms as the number of times that the word \(w_i\) appears in the message d, i.e. \(tf(w_i, d)\). Moreover, we have that \({occ_{w_i}}(w_j) = 1\), and given that by definition of \({occ_{w_i}}(.)\), its maximum value is 1, we can say that \({occ_{w_i}}(d') \ge 1\) is equivalent to \({occ_{w_i}}(d') = 1\). Therefore, the expression \(-\log \Big (P_{d' \in \mathcal {D}}\big ({occ_{w_i}}(d') = 1 \big ) \Big )\) is exactly \(-\log \Big ( P_{d' \in \mathcal {D}}(w_i \in d')\Big ) = idf(w_i)\). And thus, \({\mathcal {I}}_{occ_{w_i}}\big (d\big ) = tf(w_i, d) \cdot idf(w_i)\). \(\square \)

Proposition 5.3

Given the vocabulary, \(\mathcal {V} = \{w_1, \ldots , w_n\}\), and considering as features:

Assuming feature independence, we have

Being \(\mathcal {S}\) all the possible sequences of words which form a message, in the previous formula we have that:

Notice that \({\gamma _{i,j}}(w_1, \ldots , w_m)\) is equal to zero for all the sequences of the form \((w_1, \ldots , w_m)\) except for the sequences which verify that \(w_j = w_i\). Since by definition, \({\gamma _{i,j}}(.) \ge 0\), in the summation all the terms are null, except for the sequences which verify \(w_j = w_i\). In these cases, we have that \({\gamma _{i,j}}(w_1, \ldots , w_m) = 1\), and given that by definition of \({\gamma _{i,j}}(.)\), its maximum value is 1, we can say that \({\gamma _{i,j}}(w_1', \ldots , w_k') \ge 1\) is equivalent to \({\gamma _{i,j}}(w_1', \ldots , w_k') = 1\). Therefore, we have the next equality on probabilities:

And finally, with trivial algebraic operations, we have:

\(\square \)

Proposition 5.4

Considering the definition of Lin’s distance and assuming information additivity,

Assuming feature independence, it is equivalent to:

\(\square \)

Proposition 5.5

We will start from the \(IDF_{N\hbox {-}gram}\) term weighting of an n-gram, g, using the same notation of \(|\phi (g)|\) for the message frequency of g, and \(|\mu (g)|\) for the amount of messages containing at least one subsequence of the n-gram g.

Considering both sets of features, \(\Gamma \) and \(\Gamma '\), the OIQ of an n-gram in each set of features is computed by:

Replacing these expressions in the definition of the \(IDF_{N\hbox {-}gram}\):

\(\square \)

Proposition 5.6

We will start from the \(\mu _{d}(\gamma _i)\) term weighting of a message.

Considering two different scenarios relating to the relevancy of messages, \(\mathcal {D}_1\) and, \(\mathcal {D}_2\), the OIQ of a message is computed by:

Replacing these expressions, we get:

\(\square \)

Reprints and permissions