Comparing intermittency and network measurements of words and their dependence on authorship

Our database contains five novels from each of eight authors who lived between 1809 and 1975, which are available in an online repository (http://www.gutenberg.org/). The list of books is given in section 1 of the supplementary information (SI) available from . To avoid the effects from the length of the texts, each book was limited to their first 18 200 tokens, which correspond to the length of the shortest book. In the remainder of this section, we illustrate our results using the book The Adventures of Sally by P G Wodehouse. The results for all books appear in SI (section 3) and are discussed in section 3 below.

2.1.1. Pre-processing of the text

Complex networks have been used to characterize different properties of languages [5–10, 14, 33]. Here we are interested in author-specific characteristics and therefore we adopt a network description based on word co-occurrence [5, 7–9, 14, 33], where nodes are words and links are established between subsequent words. This procedure is illustrated in figure 1. The network is defined by a set V = {v₁, v₂, ..., v_n} of vertices, and a set E of edges and is represented as a nonsymmetric weighted matrix W. By construction, W is a square matrix of size n, where n is the number of distinct words after the pre-processing step. The elements w_ij of W indicate the strength that v_i is connected to v_j (v_i → v_j), i.e. the number of times word v_j appears immediately after word v_i. Additionally, we used the nonweighted and undirected network corresponding to W, denoted by the matrix A whose elements a_ij = 1 if the words represented by the vertices v_i and v_j appeared as neighbors at least once in the text. Otherwise a_ij = 0.

Zoom In Zoom Out Reset image size

We shall use the statistical properties of complex network measurements in the networks W and A. In this section, we discuss the word-specific local measurements, and in section 3.1, we show how to connect them to obtain a global characterization of the network. The number of occurrences N_i of each word i represented by node v_i is

where sⁱⁿ and s^out are, respectively, the weight (or strength [34, 35]) of the ingoing and outgoing edges of node v_i.⁴ Therefore, the degree of each node v_i is the frequency of appearance of word v_i and the degree distribution of W is proportional to the normalized frequency () distribution of words, given by Zipf's law [36, 37]. Below we discuss three typical measurements: the clustering coefficient, average shortest path length and betweenness centrality.

2.2.1. Clustering coefficient C

The clustering coefficient (C) measures the probability that the neighbors of a given vertex v_i are connected. This measurement has been widely employed in complex networks, e.g. to verify the presence of communities [38–40] and to distinguish random networks from other small world networks [5, 41]. Traditionally, the clustering coefficient is defined without considering weights or directions as

which is equivalent to the fraction of the number of triangles among all possible triads of connected nodes and therefore ranges from 0 to 1. With regard to the interpretation of this measure, Ferrer i Cancho and Solé [5] found that the clustering coefficient of networks representing text was much larger than that expected just by chance (i.e. the value expected for the corresponding random networks).

We singled out the words (and their neighbors) with the highest and lowest values of the clustering coefficient in the book The Adventures of Sally by P G Wodehouse, which are shown in table 2 for the frequency N = 5.⁵ From the definition, one expects words with the highest C to have neighbors also connected to each other. This is the case for the words 'sand' and 'excitement'. On the other hand, words whose neighbors are not related to each other at all display low values of C (e.g. there is no link between the neighbors of 'full' or between the neighbors of 'high'). Qualitatively, the clustering coefficient quantifies how words are connected to specific contexts. Indeed, the words 'sand' and 'excitement' tend to be more restricted to a specific context, while 'full' tends to appear in a myriad of contexts. Therefore, it seems that the clustering coefficient can be useful in detecting authorship by quantifying the tendency for using semantic-specific or generic words.

Table 2. Words of the book The Adventures of Sally with the highest and lowest clustering coefficients (the average clustering 〈C〉 = 0.085), for words with N_i = 5. The five words with C = 0 were randomly selected among the 18 words with N = 5 and C = 0.

Word	Neighbors	N	C
Shortly	twelve, see, say, Sally, news,	5	0.27
	never, heaven, find, enter and Carmylle
Excitement	thing, suppressed, Sally, mince, last,	5	0.25
	can, come, bristle, brief and apart
Sand	watch, want, sit, shuffling, seat,	5	0.18
	here, golden, first, dark and Roville
Nose	voice, tip, sort, smut, smooth,	5	0.18
	Sally, oh, glance, tell and come
Country	time, still, somewhere, say, place	5	0.18
	may, happen, great and glorious
Startle	shy, seem, mill, little, gratify,	5	0.00
	gather, first, everyday, displeased and considerably
High	recess, mouth, motive, lapse, figure,	5	0.00
	even, disposal, critical, collar and check
Gold	voice, spin, pencil, loan, knob,	5	0.00
	information, high, heavy, frame and buy
Gift	tongue, take, sort, potential, mean,	5	0.00
	few, easily, compensating, blessing and acquire
Full	Tuesday, peal, later, home, happy,	5	0.00
	gratitude, gleaming, glance, color and battle

2.2.2. Average shortest path length L

A shortest path (or geodesic path) between two nodes is defined as the path whose sum of edge weights is minimum. We start by defining d_ij as the length of the shortest path between v_i and v_j (in this case A is employed). Then the average shortest (or geodesic) path length for v_i (L_i) is the average shortest path to all other (n − 1) nodes of the network:

which takes low values if v_i is close to the other nodes.

The words with the lowest L include the characters 'Sally' (L = 2.35,N = 347) and 'Fillmore' (L = 2.51,N = 138), in addition to high-frequency words, such as 'say' (L = 2.45, N = 349), 'good' (L = 2.46,N = 107) and 'man' (L = 2.50,N = 193). As for the words with the highest L, we found: 'white-clad' (L = 6.33, N = 1), 'affability' (L = 6.31, N = 1), 'whirl' (L = 5.89,N = 1), 'jazz' (L = 5.87,N = 1) and 'war-aims' (L = 5.84, N = 1). Interestingly, all these five words appeared only once in the text, indicating that one of the reasons for a high L could be the low frequency N. However, L is not only a consequence of the frequency N of the words, as low-frequency words can also take low values of L. This is illustrated in table 3, which compares words with the same N but different L. The frequency has a limited influence on L, with a Pearson correlation Corr(L, N) = − 0.36 calculated over all words. Actually, the determining factor is the neighborhood of the word. To understand why this happens, consider the words 'affability' and 'repose'. While the former has as neighbors the words 'jaunty' (N = 1) and 'white-clad' (N = 1), the latter has as neighbors the words 'Sally' (N = 347) and night (N = 20). Therefore, one may infer that L actually quantifies the importance of a word according to its distance to the most frequent words. Since we removed stopwords, the shortest path may be thought of as quantifying the distance from a word to the core-content words of the book.

Table 3. Comparing the average shortest path length L for words with the same frequency N of the book The Adventures of Sally. For a given N, L may vary widely, which shows the dependence of L on the neighborhood connectivity.

Word	Ni	Li	Word	Ni	Li
Red	5	3.71	Earth	5	2.99
Shudder	4	3.97	Lucky	4	3.00
Maxwell	3	5.55	Funny	3	3.10
Dark	2	5.15	Kiss	2	3.08
Affability	1	6.34	Repose	1	3.11

2.2.3. Betweenness

Betweenness B is a measurement of centrality, with higher values being assigned to the nodes considered as the most relevant in terms of linking different words. In other words, with B one attempts to quantify the frequency of access of each node, assuming that a given target node in the network is reached from a specific source node via shortest paths. Betweenness is defined as follows. Let ηⁱ_st be the number of distinct shortest paths between the source node v_s and the target node v_t that pass through the node v_i. If g_st is the total number of shortest paths between v_s and v_t, then B_i is given by

In the context of text analysis, high-frequency words tend to have high B. However, some words may play the role of articulation points by linking concepts related to distinct communities. To illustrate this, we show in table 4 that words with similar N may take very different B. A comparison between the left and right columns suggests that words with high B connect concepts because of their probable appearance in various contexts. Therefore, similarly to the clustering coefficient C, the betweenness centrality B seems to quantify the variety of contexts in which a word can appear. Note, however, that B is based on a global connectivity pattern, in contrast to C.

Table 4. Comparing the betweenness B for words of the book The Adventures of Sally with the same frequency N. For a given N, the betweenness may vary widely, since low-frequency words may have high betweenness as they may appear in different contexts.

Word	Ni	Bi	Word	Ni	Bi
Say	349	745 634	Sally	347	1 192 881
Know	143	243 357	Fillmore	138	393 955
Tell	65	53 904	Gerald	62	108 528
Allow	20	15 816	Roville	21	32 449
Heaven	10	1 147	Second	10	22 004
Rugger	5	855	Worthy	5	10 503
Fish	4	174	Spectator	4	14 746
Paper-Knife	3	233	Group	3	8 320
Worship	2	44	Sell	2	8 346
Thaw	1	11	Price	1	8 295

The uneven distribution of words across different documents is an essential feature exploited in statistical natural language processing. For instance, by investigating words appearing overconcentrated in specific documents (when compared with their overall frequency) one can detect keywords, topics and authorship [27, 42]. This is the basic idea of tf–idf (term frequency–inverse document frequency) and related measures that are also at the core of search engines [42]. However, there are numerous situations where the comparison with a general database is not available or is not interesting: for instance, when authorship has to be attributed without previous knowledge of texts written by the potential authors. Here, we approach these problems by taking advantage of the finding that words are unevenly distributed not only across documents but also within them [16–18, 21–24].

The quantification of the uneven distribution of words has been proposed based on measures commonly used by physicists [16, 24]. Following [21, 22, 24], we use the statistics of recurrence times, a standard quantification of intermittency or burstiness in time series [18, 19]. In texts, time is counted by the number of words and for each word i the recurrence time T_j is defined as the number of words between two successive occurrences of i (the j and j + 1 occurrence) plus one. For instance, the recurrence times for the word 'the' in the previous sentence are T₁ = 9 and T₂ = 7. A word that appears N_i times in a text of size N_T leads to a sequence of N_T − 1 inter-occurrence times {T₁,T₂,...,T_NT−1}. In order to incorporate also the time until the first T_f and after the last T_l occurrence of the word, we consider T_N = T_f + T_l. In this case , where the overline denotes average over the different T_j's. Note that the mean recurrence time gives no additional information other than the frequency N_i. The intermittency of the word appears in the variance of T_j's around and can be quantified by , where . Randomly distributed words in the text have I = 1 (in the limit of large N and small N_i/N_T), intermittent words have I > 1 and words appearing in regular intervals have I < 1. We calculate the intermittency measure for all words with N_i ⩾ 5 in each of the books (filtered texts) described in section 2.1. The words with N_i < 5 were considered to lack statistics and were disregarded.

In table 5, we compare words with highest and words with similar frequency. It is clear that the most intermittent words (largest ) are topical words (e.g. names of characters and locations), regardless of their frequency. Indeed, 15 out of the 16 most intermittent words are directly connected to specific characters. Similar behavior is observed in all books of our database. Intermittency is therefore a good characterization of topical words that in turn plays an important role in the author-specific characteristic of the texts. The relationship between and the function of the words has been investigated in detail in [16–18, 21, 22, 24]. In the next section, we explore the fact that these properties are also author specific [17].

Table 5. In the book The Adventures of Sally by P G Wodehouse, there are N_T = 15 173 words (tokens), 3657 different word types and 716 words with N_i ⩾ 5. The five words with highest are shown in the left part of the table. For comparison, in the right we show for each of these words another word with the closest frequency.

Word	Ni		Word	Ni
Jules	26	4.31	Turn	26	1.55
Hobson	31	4.09	Here	31	1.35
Ginger	115	3.86	Get	117	1.24
Carmyle	54	3.60	Feel	53	0.87
Bunbury	20	3.59	People	20	1.39

In the previous section, we introduced five quantities characterizing properties of words in the text: frequency (N), average shortest path length (L), betweenness (B), clustering coefficient (C) and intermittency (). The values of these quantities for all words in the books in our database can be found in SI (section 3), available from . We now analyze the global distribution of these measurements for all the words in a given book by plotting the empirical probability density function ρ(X) for the measurements X = {N,L,B,C,I}. Figure 2 shows the results for one book, and similar distributions were obtained for the other books. The shortest path L, clustering C and intermittency I have a well-defined peak and width (akin to a Gaussian distribution), but the frequency N and betweenness B have broad tail distributions (as in power-law distributions ρ(X) ∼ X^−α). The tail in N corresponds to the well-known Zipf's law, which also appears in B as expected from the large correlation between B and N (Corr(B,N) = 0.95 in the book of figure 2). With the two different types of behavior, we propose two sets of measurements, one for X = {L,C,I} and another for X = {N,B}.

Zoom In Zoom Out Reset image size

Our goal is to obtain quantities characterizing important features of these distributions to be used as global measurements of the books. The most natural choice is the average value 〈X〉, where corresponds to an average over the M different words. For the network measures L,C and I this corresponds to the average values over nodes, a quantity considered as characteristic of the network [7, 12, 14, 43]. For X = {N,B}, the high frequency words contribute strongly to 〈X〉 due to the long tails. To compensate for this effect, we consider also a modified average defined as 〈X〉₂ ≡ 〈log X〉 for X = {N,B}. For X = {L,C,I} the opposite is true, i.e. 〈X〉 is dominated by the large number of low-frequency words. Accordingly, we introduce a modified average as , i.e. a weighted average with weights proportional to the logarithm of the frequency. The quantities 〈X〉 and 〈X〉₂ are expected to give a good account of 'typical' values of X. However, in section 2 we mentioned that important information is conveyed by words with large X, i.e. in the tails of the distributions shown in figure 2. In order to characterize the fat-tail distributions of X = {N,B}, we used the coefficient α_X of a power-law fit to the tails of ρ(X).⁶ An additional motivation for using α_N comes from the suggestion in [45] that it serves as a quantification of the style of texts. The large values of X = {L,C,I} were characterized by calculating the skewness of ρ(X), a measure of the asymmetry of the distribution. In summary, the three features we use for each of the five quantities X = {N,B,L,C,I} are

These features are given in figure 2 for one book (see SI (section 3), available from , for all 40 books). Obviously, the choice of the quantities above is inevitably arbitrary. Our choice was intended to capture features of the distribution, rather than giving a parametric description of the full distribution. In particular, the power-law fit in equation (7) does not intend to fully describe the distributions, as is apparent in figures 2(d) and (e).

In order to quantify the ability of the features described above to distinguish between authors, we employ machine learning algorithms which induce classifiers from a training database. The robustness of our results is tested with three widely used algorithms based on different principles. The first is known as C4.5 [46], and generates decision trees based on the information gained by each feature; the second algorithm is the Naive Bayes [47], which is based on the Bayes theorem; and the third and simplest algorithm is the nearest neighbor [48], which classifies an unknown instance according to the nearest neighbor of that instance in a normalized space involving all features. For more details, see SI (section 4), available from .

We consider the problem of distinguishing between eight authors, using five books to represent each author's style. More specifically, each book described in section 2.1 was characterized by the set of 15 features discussed in section 3.1 (〈X〉, 〈X〉₂ and γ(X) for X = {N,B,L,C,I}). The authorship assignment was performed using the algorithms in section 3.2 applied to a training dataset independent of the test book using the cross validation methodology (see SI (section 4)). This technique ensures that the training and evaluation sets are different and it is equivalent to assigning the authorship of one book in an experiment where four books of eight authors were used as a training dataset. The final output of the algorithms is the assignment of a specific author to each book tested, and the efficiency is quantified simply as the fraction of successful assignments.

The results are summarized in table 6 and indicate accuracy rates between 42.5 and 50.0% when all 15 features were used. These results were statistically significant by a large amount, confirming that these features successfully capture author-specific characteristics. To further explore the accuracy of different methods, we considered the cases in which only some of the features were included in the algorithms. We tested all 2¹⁵ = 32 768 combinations of the 15 features and obtained a best result of 65.0% of correct assignments.

In evaluating the importance of the different features for the final results, it is essential to identify their mutual dependence. We start from the list of all 2¹⁵ = 32 768 combinations of features ordered by decreasing accuracy (as shown in SI (tables S1–S4)). We wish to quantify when feature y appears in the top of this list. To this end, we count the fraction of the 2¹⁴ feature combinations that include y with accuracy higher than or equal to a threshold. The final estimate is then given by the area-under-the-curve of the ROC plot obtained by varying the threshold [49]. This procedure is equivalent to the Mann–Whitney U test [50]. The motivation for using this method is that it evaluates the importance of a specific feature by taking into account how it combines with the other features to improve the accuracy of the prediction. The method depends both on the prediction algorithm and on the other features.

The features were ranked based on the method described above. The results for the three prediction algorithms are given in the first three columns of table 7. The three features appearing as the most prominent are 〈N〉 (average frequency), γ(I) (skewness of intermittency) and 〈N〉₂ (average logarithmic frequency). In order to state the importance of features beyond a specific algorithm, it is important to quantify to what extent the results obtained for the three algorithms (the first three columns) are consistent with each other. To this end, we compute the Spearman's rank correlation and obtain the values 0.29 (p-value =0.145), 0.49 (p-value =0.032) and 0.67 (p-value =0.003) for the pairs C4.5/kNN, C4.5/Bayes and kNN/Bayes, respectively. The p-values are computed under the null hypothesis that the rankings are independent. Altogether, the three p-values indicate that the three rankings are consistent with each other. This is a strong indication that our analysis goes beyond algorithm-specific results and indeed captures the influence of the features.

It is interesting to compare the results with evaluations taking into account each feature separately. This can be done either by considering the accuracy of the prediction using only the specific feature or by comparing the information gained by including the feature [51]. This last method has the advantage of being independent of the prediction algorithm. These results are shown in the last four columns of table 7. Note that some features appearing as very important in multiple features analysis are not informative when taken alone (e.g. the skewness of the intermittency γ(I)). On the other hand, features that are well ranked in the single feature analysis do not always appear among the most important features when multiple features are considered (e.g. the weighted average of the clustering 〈C〉₂). These observations show the nontrivial mutual dependence of the features. To further explore this, we performed a factorial analysis (see SI (section 5), available from ) using the 12 most important features in table 7, with the most important combinations being summarized in table 8. As expected from table 7, in fact γ(I) appears among the best two combinations of features in all three algorithms, which confirms that its effectiveness is correlated with its interdependence on other features.

We are now in a position to use the word-specific analysis (section 2) and the distribution (section 3.1) of the quantities X = {N,B,L,C,I} to assess the importance of the different features 〈X〉,〈X〉₂ and γ(X) in table 7:

• Frequency. This was the most efficient quantity for recognizing authorship with 〈N〉 and 〈N〉₂ among the three most important features. Noting that 〈N〉 is proportional to the inverse number of distinct words M:
  
  one infers that the distinguishing feature between the authors captured by 〈N〉 is the different vocabulary sizes. The modified average 〈N〉₂ = 〈log N〉 also captures this aspect, including the proportion of frequent and infrequent words. On the other hand, the poor performance of γ(N) (=α in ρ(N) ∼ N^−α) is a clear signature of the universal, author-independent character of Zipf's law (at a fixed book size [37]).
• Betweenness. The average betweenness 〈B〉 was useful because of its strong correlation with the vocabulary size of the book M (the last column in table 7). In network terms, this corresponds to a linear relationship between 〈B〉 and network size (M is the number of nodes) and can be understood by noting that the number of terms in the sum of definition of B_i in equation (4) is proportional to M², so that 〈B〉 is expected to scale linearly with M for a fixed book size. The fact that 〈B〉₂ and γ(B) had a poor performance indicates that the number of words with high betweenness is not a relevant feature to distinguish between authors.
• Shortest path. This was the network quantity with best performance. 〈L〉 quantifies the typical distance of words to the central hubs of the network (frequent words). The good performance points to a dependence on the style of the authors. The poorer performance of 〈L〉₂ and γ(L) indicates that the style dependence in L is more prominent in the typical values than, respectively, in the frequent and large L words.
• Clustering. The poor performance of all values related to this quantity suggests that authors have very little freedom in choosing the clustering of words co-occurrence networks. The last position in the ranking of γ(C) in table 7 suggests that the fraction of words used in specific contexts (high C) is author independent. The two averages 〈C〉₂ and 〈C〉 take similar values (as seen in figure 2 and in SI (section 3), available from ; recall the restriction N_i ⩾ 5 used in section 2.2). They perform well only when used alone, possibly because of their correlation to vocabulary size M.
• Intermittency. Apart from the frequency, intermittency was the most important quantity in table 7 with the skewness of the distribution γ(I) playing a prominent role. In view of the results in section 2.3, γ(I) may be interpreted as the fraction of all words that are topical or 'keyword like'. The poor performance of 〈I〉 is not surprising since I is normalized by frequency () and therefore 〈I〉 ≈ 1 is expected. Indeed, from all five quantities the I features have shown altogether the smallest absolute value of correlation with vocabulary size (table 7), explaining why even γ(I) has a poor relative performance when used alone. Finally, 〈I〉₂ performs better than 〈I〉, suggesting that frequent words are more relevant.

Even if the main goal of this paper is to evaluate the importance of different factors, it is also useful to compare the accuracy of our results with other methods of authorship attribution. Uzuner and Katz  [52] used a database of books similar to ours, produced by eight authors. They used five sets of features, including simple statistics and more sophisticated syntactic analysis (table 3 of [52]). Our best results (accuracy of 65%) is comparable to their second best case obtained using 'syntactic elements of expression' (62%), being significantly worse only than their best result, achieved using function words (87%). In an extensive review, Grieve  [31] reported accuracies obtained with a set of 34 features varying between 33% and 87% for the case of five authors, and between 18 and 80% for ten authors (table 9 of [31]). Our best results are above the median of their results achieved by using different features. Their best results again are based on the relative frequency of function words. These results are in accordance with the long tradition started by Mosteller and Wallace [27] to use the frequency of function words to distinguish between authors.

In order to confirm this in our database, we implemented a series of prediction schemes using the frequency N_i of frequent (mostly function) words. In contrast with the approach described in this paper that used average and scaling properties as features, now the frequencies of specific words are used directly as input features of the prediction algorithms. We used only the kNN algorithm because the other algorithms did not provide good results when too many features were included. When the list of 70 stopwords from table 2.5 of [27] was used, we obtained an accuracy of 62.5%, i.e. comparable to our best results. Following the work [31], we considered two other lists of words: all 1978 words that appear in at least one book of each of the authors, leading to an accuracy of 90%; and all 209 words that appear at least once in every book in our database, leading to an accuracy of 82.5%. We recall that in order to concentrate on analysis that focuses on words with pronounced semantic content instead of function words, we have deliberately excluded a list of stopwords that contributed 80% of the cases listed in [27]. Therefore, our best combination of features compares well to other methods which demand more sophisticated syntactic analysis of the text. Measurements of complex network and intermittency are indeed able to capture many of the author-dependent characteristics.

In order to illustrate how measures analyzed in this paper can be complementary to traditional methods, we have performed a very simple experiment using as features the frequency and intermittency of the set of words composed of the five most frequent words in each book. The accuracy in classifying the books only by frequency was 72.5% and only by intermittency was 37.5%. Although this last accuracy rate is not impressive, it is statistically significant (p = 2.2 × 10⁻⁴) and shows that the intermittency values of specific words across distinct authors are different. The accuracy is increased to 80% when both features are included. It remains to be shown in future works how our results can improve state-of-the-art methods of authorship attribution.

We have shown that the styles of different authors leave fingerprints in very general statistical measures of texts based on the network of co-occurrence of words and on intermittency or burstiness of words. The statistically significant scores obtained in authorship attribution unequivocally show that the style dependence of these features can be used in practice. Regarding the prominence of the different features, we note that both the results and ranking of features may depend on the database, selected features and attribution algorithms. Accordingly, as emphasized in [31], different algorithms and features have to be tested in a given corpus before any real application of authorship attribution. However, the robustness of our results using three radically different attribution algorithms strongly suggests that the different features have importance that go beyond specific algorithms. Two features should be highlighted: (i) the skewness of the distribution of intermittent words γ(I), which is based on the long-scale distribution of words and detects the extent to which topical words (keywords, large ) were used in the book; and (ii) the mean shortest path of the word co-occurrence network 〈L〉, which is based on the short-range connectivity of words and detects the typical distance of words to all other words. The different nature of these two quantities suggests a complementary role in capturing both short- and long-scale properties of the text, as well as typical and exceptional words.

Our focus in this paper was on the evaluation of the different features, rather than on maximizing the efficiency of the authorship attribution algorithms. This is apparent when comparing the best accuracy rates we achieved using our approach (62.5%) and using previous proposals (90.0%), as discussed in section 4.2. A further limitation of approaches based on intermittency and networks is that they require large pieces of text. While the root of the success of previous methods is the observation that function words are a powerful tool for detecting the style of authors [27, 31, 52], in the complex network and intermittency approaches used in this paper the focus is on content words. In this sense the results we achieve can be thought of as being complementary to the analysis using function words. More specifically, our results suggest that using γ(I) and 〈L〉 can improve authorship recognition techniques when used in combination with the many different features currently employed [31]. Finally, the successful application of these measurements to characterize the style of authors suggests that the quantities discussed here can be further explored in other linguistic tasks, an approach that has been limited to a few works (see, e.g., [14, 53]).