How can we measure the similarity between texts? One approach would be to define distances between them. A text can be represented by a vector, after which a "similarity function", or a distance, can be defined to compare the vectors.
For example, we can record whether a word type (a vocable) is present or absent. A text is then represented by a vector whose components have the value 0 or 1. The number n of components in the vector (the dimension of the vector space) is the number of word types considered—that is, the size of the vocabulary. Numbering the word types from 1 to n, a text X is represented by a vector (*xi)i*=1…*n where xi = 1 if word type i appears in the text, and xi = 0 otherwise. This vector representation is generally used to calculate the proportion of vocabulary shared by the texts. This is the Jaccard index*, defined by:
Sim(X, Y)=∑i=1nxiyin\text{Sim(X, Y)} = \sum_{i=1}^{n} \dfrac{{x_i}{y_i}}{n}
where X = (*xi)i*=1…*n and Y = (yi)i*=1…*n are two texts of n* words.
Let's take two examples. Text S reads: "The Sun says: "I am the Sun."" Text L reads: ""I am the Sun", says the Moon." Then n = 7 and Sim(S, L) = (11 + 11 + 01 + 11 + 01 + 11 + 11) / 7 = 5/7.

Count of the word types found across both texts, listed in alphabetical order.

Size isn't everything… ----------------------------------
The similarity index is therefore 1 when both texts use exactly the same vocabulary, and 0 when they have no words in common. The latter can occur when comparing two texts written in different languages.
We can refine this index by replacing the presence-or-absence information for a word type M with the number of times M occurs in the text. Each component of the vector then gives the number of occurrences of M. With this kind of vector representation, we could use the standard Euclidean distance:
DistE(X, Y)=∑i=1n(xi−yi)2.\text{DistE(X, Y)} = \sqrt{\sum_{i=1}^{n} (x_i - y_i)^2}.

Taking the number of occurrences of each word type into account.

In our example,
DistE(S, L)=0+0+1+1+1+1+0=2.\text{DistE(S, L)} = \sqrt{0+0+1+1+1+1+0}=2.
In practice, however, absolute occurrence counts depend heavily on text length. The longer the text, the larger the component values. We therefore use the cosine of the angle θ between the two vectors as a similarity measure:
\text{SimCos(X, Y)} = \text{\cos} \; \theta = \dfrac { \sum_{i=1}^{n} {x_i}{y_i}} { \sqrt{\sum_{i=1}^{n} x_i ^2} \sqrt{\sum_{i=1}^{n} y_i ^2}}
Since the vector components are always positive, the similarity measure ranges from 0 to 1. It equals 1 when the angle between the vectors is 0—that is, when the two vectors are identical (the same occurrence count for every word type) or collinear (their occurrence counts are multiples of one another). The similarity is 0 when the two vectors are orthogonal—that is, when they share no word types.
In our example,
SimCos(S, L)=7117,\text{SimCos(S, L)} = \dfrac{7}{ \sqrt{11} \sqrt{7}},
which is approximately 0.8.
To account for the length of the texts, measured in words, we can also replace occurrence counts with the relative frequency of each word type. A vector component can then be interpreted as the probability of encountering a given word type in the text.
A text is now represented by a vector (*xi)i*=1…N where *xi = ni / T, with ni denoting the number of times word type i appears in the text and T the length of the text, measured in words. With this type of representation, we can use an intertextual L1* distance:
δ(X, Y)=12∑i=1nxi−yi\delta \text{(X, Y)} = \dfrac{1}{2} {\sum_{i=1}^{n} x_i - y_i }
δ(X, Y)=12∑i=1n∣xi−yi∣\delta \text{(X, Y)} = \dfrac{1}{2} {\sum_{i=1}^{n} |x_i - y_i |}

Texts S and L represented by frequency vectors.

The intertextual distance has a fairly natural interpretation as an average number of differing words per hundred words, or as the proportion of word types that must be "changed" to turn one text into the other. Thus, δ(S, L) = (1/2)(4/7) = 2/7, and changing two words out of seven allows us to "turn" one text into the other (disregarding word order).
-
A little order… ---------------
The preceding approaches do not take word order into account. Moreover, in S and L, if the Sun is the sun, it is reasonable to suppose that the moon follows it (from the verb "to follow"). In some applications, it is essential not to confuse "suis" (from être, "to be") with "suis" (from suivre, "to follow"), "été" (from être, "to be") with the season été ("summer"), or "est" (from être, "to be") with the direction est ("east"). It is therefore often necessary to work with texts in which each word is annotated with its lemma (dictionary form) and part of speech.
In practice, this type of vector representation is widely used in information retrieval—for example, to retrieve texts similar to an initial text used as a query. A basic query text may simply be a string of a few keywords. Using the vector representation and a similarity calculation, a document search engine can rank the documents returned from most relevant (most similar to the string of keywords) to least relevant. Search engines such as Google, Yahoo and Bing can also use this type of representation to compare web pages matching a query and remove duplicates from the list of results supplied to the user.
In literary studies, intertextual distance is also used to investigate what brings different genres closer together or sets them apart (novels, drama, spoken vs written French, poetry, the epistolary genre…) and to debate the authorship of a work (Romain Gary vs Émile Ajar, Molière vs Corneille). And such research can provoke plenty of controversy…