Lexical density
Lexical density is a concept in computational linguistics that measures the structure and complexity of human communication in a language.<ref name="Halliday1985p61">Michael Halliday (1985). Spoken and Written Language. Deakin University. pp. 61–64. ISBN 978-0-7300-0309-0.</ref> Lexical density estimates the linguistic complexity in a written or spoken composition from the functional words (grammatical units) and content words (lexical units, lexemes). One method to calculate the lexical density is to compute the ratio of lexical items to the total number of words. Another method is to compute the ratio of lexical items to the number of higher structural items in a composition, such as the total number of clauses in the sentences.<ref name="Castello2008p48"/><ref name="Camiciottoli2007p73"/>
The lexical density for an individual evolves with age, education, communication style, circumstances, unusual injuries or medical condition,<ref>Paul Yoder (2006). "Predicting Lexical Density Growth Rate in Young Children With Autism Spectrum Disorders". American Journal of Speech-Language Pathology. 15 (4): 362–373.</ref> and his or her creativity. The inherent structure of a human language and one's first language may impact the lexical density of the individual's writing and speaking style. Further, human communication in the written form is generally more lexically dense than in the spoken form after the early childhood stage.<ref name="Halliday1985c5c6">Michael Halliday (1985). Spoken and Written Language. Deakin University. pp. 61–75 (Chapter 5), 76–91 (Chapter 6). ISBN 978-0-7300-0309-0.</ref><ref name="Johansson2009p1">Victoria Johansson (2009). Developmental aspects of text production in writing and speech. Department of Linguistics and Phonetics, Centre for Languages and Literature, Lund University. pp. 1–16. ISBN 978-91-974116-7-7.</ref> The lexical density impacts the readability of a composition and the ease with which the listener or reader can comprehend a communication.<ref>V To; S Fan; DP Thomas (2013). "Lexical density and Readability: A case study of English Textbooks". The International Journal of Language, Society and Culture. 37 (7): 61–71.</ref><ref>O'Loughlin, Kieran (1995). "Lexical density in candidate output on direct and semi-direct versions of an oral proficiency test". Language Testing. SAGE Publications. 12 (2): 217–237. doi:10.1177/026553229501200205. S2CID 145638000.</ref> The lexical density may also impact the memorability and retention of a sentence and the message.<ref>Perfetti, Charles A. (1969). "Lexical density and phrase structure depth as variables in sentence retention". Journal of Verbal Learning and Verbal Behavior. Elsevier BV. 8 (6): 719–724. doi:10.1016/s0022-5371(69)80035-6. ISSN 0022-5371.</ref>
Discussion
The lexical density is the proportion of content words (lexical items) in a given discourse. It can be measured either as the ratio of lexical items to total number of words, or as the ratio of lexical items to the number of higher structural items in the sentences (for example, clauses).<ref name="Castello2008p48">Erik Castello (2008). Text Complexity and Reading Comprehension Tests. Peter Lang. pp. 49–51. ISBN 978-3-03911-717-8.</ref><ref name="Camiciottoli2007p73">Belinda Crawford Camiciottoli (2007). The Language of Business Studies Lectures: A Corpus-assisted Analysis. John Benjamins Publishing. p. 73. ISBN 978-90-272-5400-9.</ref> A lexical item is typically the real content and it includes nouns, verbs, adjectives and adverbs. A grammatical item typically is the functional glue and thread that weaves the content and includes pronouns, conjunctions, prepositions, determiners, and certain classes of finite verbs and adverbs.<ref name="Halliday1985c5c6"/>
Lexical density is one of the methods used in discourse analysis as a descriptive parameter which varies with register and genre. There are many proposed methods for computing the lexical density of any composition or corpus. Lexical density may be determined as:
- <math> L_d = (N_{\mathrm{lex}}/N) \times 100 </math>
- Where:
- <math>L_d</math> = the analysed text's lexical density
- <math>N_{\mathrm{lex}}</math> = the number of lexical or grammatical tokens (nouns, adjectives, verbs, adverbs) in the analysed text
- <math>N</math> = the number of all tokens (total number of words) in the analysed text
Ure lexical density
Ure proposed the following formula in 1971 to compute the lexical density of a sentence:
- Ld = The number of lexical items/The total number of words * 100
Biber terms this ratio as "type-token ratio".<ref name="Biber2007p97">Douglas Biber (2007). Discourse on the Move: Using Corpus Analysis to Describe Discourse Structure. John Benjamins Publishing. pp. 97–98 with footnote 7. ISBN 978-90-272-2302-9.</ref>
Halliday lexical density
In 1985, Halliday revised the denominator of the Ure formula and proposed the following to compute the lexical density of a sentence:<ref name="Halliday1985p61"/>
- Ld = The number of lexical items/The total number of clauses * 100
In some formulations, the Halliday proposed lexical density is computed as a simple ratio, without the "100" multiplier.<ref name="Castello2008p48"/><ref name="Halliday1985p61"/>
Characteristics
Lexical density measurements may vary for the same composition depending on how a "lexical item" is defined and which items are classified as lexical or as a grammatical item. Any adopted methodology when consistently applied across various compositions provides the lexical density of those compositions. Typically, the lexical density of a written composition is higher than a spoken composition.<ref name="Castello2008p48"/><ref name="Camiciottoli2007p73"/> According to Ure, written forms of human communication in the English language typically have lexical densities above 40%, while spoken forms tend to have lexical densities below 40%.<ref name="Castello2008p48"/> In a survey of historical texts by Michael Stubbs, the typical lexical density of fictional literature ranged between 40% and 54%, while non-fiction ranged between 40% and 65%.<ref name="Camiciottoli2007p73"/><ref>Mark Warschauer; Richard Kern (2000). Network-Based Language Teaching: Concepts and Practice. Cambridge University Press. pp. 107–108. ISBN 978-0-521-66742-5.</ref><ref>Michael Stubbs (1996). Text and Corpus Analysis: Computer Assisted Studies of Language and Culture. Wiley. pp. 71–73. ISBN 978-0-631-19512-2.</ref>
The relation and intimacy between the participants of a particular communication impact the lexical density, states Ure, as do the circumstances prior to the start of communication for the same speaker or writer. The higher lexical density of written forms of communication, she proposed, is primarily because written forms of human communication involve greater preparation, reflection and revisions.<ref name="Castello2008p48"/> Human discussions and conversations involving or anticipating feedback tend to be sparser and have lower lexical density. In contrast, state Stubbs and Biber, instructions, law enforcement orders, news read from screen prompts within the allotted time, and literature that authors expect will be available to the reader for re-reading tend to maximize lexical density.<ref name="Castello2008p48"/><ref>Michael Stubbs (1986). "Lexical density: A technique and some findings". In Malcolm Coulthard (ed.). Talking about Text. University of Birmingham: English Language Research. pp. 27–42.</ref> In surveys of lexical density of spoken and written materials across different European countries and age groups, Johansson and Strömqvist report that the lexical density of population groups were similar and depended on the morphological structure of the native language and within a country, the age groups sampled. The lexical density was highest for adults, while the variations estimated as lexical diversity, states Johansson, were higher for teenagers for the same age group (13-year-olds, 17-year-olds).<ref>Victoria Johansson (2008). "Lexical diversity and lexical density in speech and writing: a developmental perspective". Linguistics and Phonetics Working Papers. Lund University. 53: 61–79.</ref><ref>Sven Strömqvist; Victoria Johansson; Sarah Kriz; H Ragnarsdottir; Ravid Aisenmann; Dorit Ravid (2002). "Toward a crosslinguistic comparison of lexical quanta in speech and writing". Written Language and Literacy. 5: 45–67. doi:10.1075/wll.5.1.03str.</ref>
See also
References
Further reading
- Ure, J (1971). Lexical density and register differentiation. In G. Perren and J.L.M. Trim (eds), Applications of Linguistics, London: Cambridge University Press. 443–452.
