Understanding the mechanisms of language comprehension

  • Dr Julie A Van Dyke, a senior research scientist in the field of psycholinguistics at Haskins Laboratories, Connecticut, USA, is helping to unravel the mechanisms underlying language comprehension, including the processes that lead to poor understanding when reading or listening.

    Language comprehension is one of the most automatic tasks that humans perform. Yet it is also one of the most complex, requiring the simultaneous integration of many different types of information, such as knowledge about letters and their sounds, spelling, grammar, word meanings, and general world knowledge. In addition, general cognitive abilities such as attention monitoring, inferencing, and memory retrieval are used in order to organise this information into a single meaningful representation.

    For the most part, we take the ability to accomplish this task for granted. However, for those with language-based disabilities – including developmental disabilities (such as dyslexia, specific comprehension impairment, or speech deficits) and acquired disabilities (such as language impairment after brain injury) – assembling all this information accurately is a major challenge. While clinicians and educators are on the frontlines in helping individuals to overcome these challenges, scientists in the field of psycholinguistics are conducting the basic research that investigates: how the brain processes spoken and written language; what brain functions go awry in the case of language disability; and how to most effectively remediate deficits when they occur.

    Dr Julie A Van Dyke and her team at Haskins Laboratories are working to develop a computationally precise model, referred to as a cognitive architecture, of the mechanisms that support language comprehension. This work requires methods that reveal information both about the real-time processes associated with comprehension, and about how the products of comprehension are later accessed and retrieved for subsequent use. This is necessary to address questions such as: Are all words, or word meanings, equally likely to be retrieved? Which type of information (phonological, grammatical, semantic) is used first, or has more influence over how a person understands a text? What processes do people use to correct themselves if they realise that they’ve misunderstood a text? What if they don’t realise when this happens? For the most part, readers and listeners are completely unaware of how their brain is managing these tasks, yet the answers to questions like these provide crucial clues about the nature of the cognitive architecture for language processing.

    Dr Van Dyke and colleagues have shown that both poor and skilled readers alike employ efficient, direct-access retrieval during listening comprehension Quote_brain

    A major part of language comprehension is integrating new information with what is already known. Thus, Dr Van Dyke’s work focuses especially on memory processes. Her research is driven by two hypotheses: the first concerns the potential contribution of poor memory retrieval to reading disability, through failure to integrate the right information at the right time during comprehension; the second concerns the role that interference from similar representations of words (or phrases or clauses) plays in inefficient memory retrieval. For example, in collaboration with Dr Brian McElree (New York University, USA), she showed that when a reader’s attention is directed towards a group of ‘fixable things’ (e.g. table, sink, truck), he or she has a more difficult time processing the verb ‘fixed’ in a sentence like, ‘It was the boat that the guy who lived by the sea fixed in two days’. This is because readers have a harder time focusing on fixing ‘the boat’ (the correct interpretation) because they become distracted by the other ‘fixable’ things that are also prominent in their memory. This type of interference occurs when retrieval cues become associated with other similar items in memory. In this example, association was along semantic lines (‘tables’, ‘sinks’, ‘trucks’, and ‘boats’ are all ‘fixable’). However, interference can also arise along phonological lines (for example, words that rhyme: ‘hat’ > ‘mat’, ‘sat’) or lexical lines (words that have an associated meaning: ‘hat’ > ‘hair’, ‘wig’). Dr Van Dyke has suggested that retrieval interference such as this is pervasive in language processing, and is one of the main factors affecting comprehension ability.

    Gathering the Data
    Dr Van Dyke and her team employ a range of experimental methods in their research. The real-time behavioural measurement techniques they use include: eye-tracking technology to study people’s eye movements while they are reading (see Figure 1); speed–accuracy tradeoff (SAT), which tracks changes in response accuracy with the accumulation of information over time; and electroencephalography (EEG), which records the electrical activity of the brain as it processes information (see Figure 2). These techniques allow the language processing architecture of the brain to be characterised with precision, and are complemented by magnetic resonance imaging (MRI), which allows the team to characterise the neurobiological foundations of skilled and disabled reading.

    Figure 1 Poor readers (panels C, D) display different eye-movement patterns to good readers (Panels A, B), particularly in sentences with increased syntactic processing demands (panels B, D).

    Figure 1
    Poor readers (panels C, D) display different eye-movement patterns to good readers (Panels A, B), particularly in sentences with increased syntactic processing demands (panels B, D).

    Dr Van Dyke’s research is also rare in its effort to create an individualised skill profile for all study participants. To accomplish this, her team administers a broad range of cognitive assessments which provides information on language-related abilities such as phonological processing, word decoding, knowledge of syntax, vocabulary, and reading experience, as well as on more general cognitive capacities such as IQ, processing speed, and visual memory. A particular emphasis in her project is to include data from individuals who are not high-school graduates and who do not (and may not plan to) pursue higher education. This group has been largely overlooked in previous language research, but is essential for providing a representative sample of reading ability in the general population. Because the scale of such datasets is large – they are more comprehensive than is typical in psycholinguistic research – Dr Van Dyke and her colleagues have pioneered the use of innovative analysis techniques. With Dr Kazunaga Matsuki and Dr Victor Kuperman (McMaster University, Canada), Dr Van Dyke has demonstrated that a machine-learning technique known as Random Forests can determine which assessments are most important for explaining variation in her experimental measures. The outcome of these studies can provide critical guidance about which instructional techniques are best matched to particular subgroups of readers.

    Managing Information
    A functional cognitive architecture supporting language comprehension must accommodate the fact that linguistic input – words, phrases, sentences, texts – contains many different kinds of information, but can only be processed serially (e.g., in a word-by-word fashion). This is the essential challenge for the language comprehension system: to reconcile the hard constraint of the serial experience of linguistic information, with the simultaneous processing and comprehension of its multi-dimensional features. In some cases, this is not so challenging. For example, understanding a phrase like, ‘the dog waited by the door’ is straightforward, because each word can be easily integrated with what came before it. However, in many cases, words that must be understood together are separated from each other, as here: ‘the dog, which had barked at the cat in the neighbour’s yard all morning, waited by the door’. Understanding this sentence puts more emphasis on a person’s ability to organise information in memory and to make appropriate connections: in this case, to realise that it is the dog that waited, and not someone or something else that can ‘wait’, such as the cat. To do this, a comprehender must accurately encode the grammatical relations in the sentence, and use these grammatical cues to select the appropriate animate noun for the verb ‘waited’.

    Figuring out how these processes occur is important because the human ability to manage and process several separate pieces of information concurrently is limited: some studies suggest we can only hold between one and four items of information in memory at a time. Therefore, a complex sentence cannot be understood in its entirety as one block piece of information. Rather, there is a memory mechanism that encodes and stores partially processed information, which is later retrieved and patched together into a (hopefully) coherent understanding.

    Figure 2 Event-related brain activity in good and poor readers while reading short passages. Good readers are better at realising when additional information must be retrieved (increased P300), and at integrating that information into a coherent meaning representation (increased P600).

    Figure 2
    Event-related brain activity in good and poor readers while reading short passages. Good readers are better at realising when additional information must be retrieved (increased P300), and at integrating that information into a coherent meaning representation (increased P600).

    With her colleagues, Dr Van Dyke has demonstrated that a direct-access information retrieval mechanism that operates based on retrieval cues is the key to skilled adult reading. Such a mechanism can compensate for limited memory capacity, explain individual variation in language comprehension, and account for specific difficulty encountered in understanding certain complex grammatical constructions. In principle, retrieval cues can be virtually anything, such as a smell, a sound, a colour or a place, that acts as a guide to what a person is supposed to remember. In the case of written or spoken language comprehension, it is a person’s encoding of the linguistic dimensions (e.g., phonology, grammar, meaning) that matters most. People who have comprehension difficulties tend to have low-quality word representations stored in memory. These poor representations can lead to confusions among similarly encoded items, and poor comprehenders can have difficulty retrieving the correct representations when they need them.

    Finding Effective Remediations

    Figure 3 Both encoding and retrieval interference invoked increased activation in dorsal-lateral pre-frontal cortex, which has been associated with conflict control. However, retrieval interference was uniquely associated with posterior activations, which have been related to the integration of syntactic structures.

    Figure 3
    Both encoding and retrieval interference invoked increased activation in dorsal-lateral pre-frontal cortex, which has been associated with conflict control. However, retrieval interference was uniquely associated with posterior activations, which have been related to the integration of syntactic structures.

    It may seem logical to assume that poor readers do not possess the cue-based retrieval system used by skilled readers, perhaps instead relying on inefficient, exhaustive searches of the contents of memory. However, with Dr Clinton Johns (Haskins Laboratories), Dr Van Dyke has shown that both poor and skilled readers alike employ efficient, direct-access retrieval during listening comprehension. This means that poor readers possess the same basic language-processing architecture as skilled readers. Potential remediations should therefore focus on reducing poor readers’ sensitivity to interference from distracting information. Dr Van Dyke’s work points to the importance of using instructional techniques that encourage learners to draw distinctions between word meanings and grammatical functions, as well as direct their attention to appropriate linguistic cues when processing a sentence. Beyond helping children and adults learn to read, this research may also provide a new perspective for understanding acquired language deficits (e.g., due to stroke or brain injury), including potentially identifying networks of brain regions associated with managing different types of interference (see Figure 3).

  • haskins_logo_978Haskins Laboratories is a private, not-for-profit research institute founded in 1935, with a scientific mission to investigate the biological basis of speech, language and reading, and their related disabilities. Together with long-standing collaborators from University of Connecticut, Yale University, and over 40 international partners, Haskins has pioneered the scientific theories that guide current clinical and educational remediations for speech and reading disabilities. The over-arching mission of the Laboratories is to leverage cutting-edge science to enable those with language impairments to participate more fully in society.
  • How close is your research to facilitating the development of remediation strategies that can help poor readers improve their comprehension skills?
    While the scientific basis in support of word reading remediations is extensive, research on effective methods for teaching comprehension is weak and inconsistent. This is partly due to the complexity of the comprehension task. A much greater hindrance, however, is the lack of detailed process models of how comprehension takes place. For example, one of the most influential models of reading comprehension – known as ‘The Simple View of Reading’ (Gough & Tunmer, 1986) – suggests that once a student learns to decode words, then reading comprehension follows automatically from their oral language comprehension ability. Yet it is exceedingly common for a child with perfectly normal oral language abilities to exhibit difficulty comprehending written material. In order to provide effective remediations for such children, it is crucial to have a clear understanding of how the processes of comprehending written texts differ from comprehending oral language. This is one of the primary goals of my research. Methods that track comprehension processes as they unfold in real time, together with targeted experimental manipulations enable us to identify which types of linguistic knowledge and/or general cognitive abilities (such as retrieving information from memory) matter most when reading specific types of texts (i.e., narratives, expositions, discourses, etc.). Data of this sort can be directly translated into classroom practices that can improve how children are taught to read.

    How similar are the reading disabilities that you focus on to the more well-known reading disabilities such as dyslexia?
    Dyslexia is a disability at the word reading level, and is present in roughly 10% of the general population. Obviously, dyslexia can produce poor comprehension; when individual words cannot be efficiently recognised, comprehension will be poor. The remediation in this case should be at the word level, focused on improving decoding ability. In contrast, since our focus is specifically on comprehension processes, we are more interested in those who have normal word decoding ability but still struggle to comprehend. This is an extremely common occurrence, and for this reason much of our research does not target individuals with diagnosed disabilities. However, at the extreme end of this spectrum are those who have perfectly normal decoding abilities but who have reading comprehension abilities that are at least two standard deviations below the level expected for their age. These individuals have been referred to as ‘Specifically Poor Comprehenders’ (SPC) and they have a prevalence similar to that of dyslexia (roughly 10% of the general population). When our goal is to focus on a specific clinical population, this is our group of interest. Recent work with Dr Nicole Landi (University of Connecticut) suggests that SPC do not notice when they retrieve incorrect information, and hence cannot correct themselves. In contrast, skilled comprehenders realise when they’ve experienced retrieval interference and do correct themselves.

    What are the main implications of adopting a retrieval–interference approach in the study of comprehension difficulties?
    Our research represents a major paradigm shift in the field of reading and language comprehension. Traditionally, the primary explanation for poor comprehension was low working memory capacity, with an associated assumption that a person’s working memory capacity was an innate and immutable trait. From this perspective, remediations are only weakly motivated, since the core-deficit is viewed as part of an individual’s genetic makeup. The breakthrough associated with the cue-based retrieval approach is the understanding that the mechanisms of language processing (including reading comprehension) require only the most minimal memory capacity. This means that a person’s innate memory ability need not constrain their language (or reading) comprehension. According to our theory, comprehension ability is determined by whether information can be retrieved efficiently when it is needed. To achieve this, two major components must be in place: individuals must have high quality linguistic representations and they must have the ability to resist interference from distracting information. These two components lead directly to specific interventions that can more efficiently target the causes of comprehension difficulty.

    Is there conclusive evidence for interference in encoding and, if so, how does it affect language comprehension?
    Language is apprehended phonologically – even written text is transformed into ‘silent speech’ as we read. This means that phonological features play a fundamental role during encoding: if an individual cannot perceive important phonological distinctions, those distinctions will not be part of the individual’s memory representation. For comprehension, however, a word’s phonological form serves mainly as a ‘handle’ that allows access to its meaning. This meaning, together with the word’s grammatical context, is what matters most in comprehension. We examined this idea in a study led by Dave Kush (Norwegian University of Science and Technology), in which we asked whether interference from phonologically similar items held in memory (i.e., coat, wrote, note) would affect how participants read ‘It was the boat that the guy who drank coffee sailed in two days’. We found definite effects of interference from the distracting words at the point of reading ‘boat’. However, these differences had no bearing on how well participants comprehended the sentence. What seems to matter more is whether readers can perceive distinctions along semantic or syntactic dimensions, and this is entirely dependent on the quality of their semantic and syntactic encodings. Interestingly, data from a recent fMRI study show a qualitatively different pattern of brain activation for encoding interference (Panel A in Figure 3) versus retrieval interference (Panel B in Figure 3).

    In your research you refer to the idea of a ‘cognitive architecture’. Could you explain this term?
    The use of the term ‘architecture’ is a very deliberate choice. Physical architectures create human thoroughfares by specifying how structures are supported and defining how spaces are separated. Similarly, a cognitive architecture specifies a thoroughfare for information. Just as a wall will separate people, with doorways to control how they interact, so too will a cognitive architecture separate types of information, and include constraints on how and when information interacts. Thus, when we speak of a cognitive architecture we are referring to a particular processing model that incorporates hypotheses about the mechanisms that support the structure (e.g., memory storage and retrieval, attentional control), what types of information enters into the structure (e.g., lexical, grammatical, semantic, pragmatic) and when, and how these components work together, both in normal and impaired processing. For example, in our model a crucial architectural question is about how different types of retrieval cues are weighted during retrieval. If a distractor matches the retrieval cues on their semantic dimension, but not on their syntactic dimension, will it be retrieved? Or more narrowly, which of several different syntactic cues matters more in a given context? These questions are akin to asking whether the door between active memory and passive memory is like a sliding door, through which anything could pass, or whether it is more like a revolving door, which imposes a particular order on the entities that pass through it. Highly articulated models of this sort play a crucial role in language research, and especially in disability research, because they capture the underlying regularities in how the brain manages information, and reveal ‘pressure points’ in the system where breakdowns may occur.

  • Research Objectives
    Dr Van Dyke explores the cognitive architecture that supports online reading and language comprehension processes, with a special focus on memory retrieval as a key component of skilled and disabled language use.

    National Institute of Child Health and Human Development


    • Morgan L Bontrager (Haskins)
    • Andrew A Jahn (Haskins)
    • Clinton L Johns (Haskins)
    • Victor Kuperman (McMaster)
    • Dave Kush (Norwegian University of Science and Technology)
    • Kazunaga Matsuki (McMaster)
    • Ashley G Lewis (Haskins)
    • Nicole Landi (University of Connecticut and Haskins)


    img_0020_edittedDr Julie A Van Dyke is a Senior Research Scientist at Haskins Laboratories, and holds adjunct Professorships at McMaster University and the City University of New York Graduate Center. She earned her PhD in Cognitive Psychology from the University of Pittsburgh, a MSc in Computational Linguistics from Carnegie Mellon University, and BA in Computer Science from the University of Delaware. When not in the lab, Julie enjoys spending time with her husband and four daughters. Her research is inspired by mothering a child with spoken language impairment and dyslexia.

    Dr Julie A Van Dyke
    Haskins Laboratories
    300 George Street, Suite 900
    New Haven, CT 06511

    E: HaskinsCompLab@gmail.com
    T: +1 203-865-6163 x 214
    W: hwww.haskinslabs.org/research/research-areas/haskins-complab

    LinkedIn: drjvandyke
    Facebbook: HaskinsCompLab
    Twitter: @HaskinsCompLab

  • Understanding the mechanisms of language comprehension

About Author

Leave A Reply