Sept. 3rd ~ A Meta-Analytic Review of Comprehension Deficits in Students with Dyslexia; Improving English reading fluency and comprehension for children with reading fluency disabilities + More!
The Weekly Email That Keeps You Informed of the Latest Reading Research!
Welcome! This is Volume 2, Issue No. 15
Welcome to the Reading Research Recap, a weekly newsletter featuring the latest reading research published in peer-reviewed scientific journals. The goal of the Recap is to share recent scientific findings and foster an appreciation of science as a way to navigate the world. I try to make this one of the most informative emails you get each week.
Updates
Welcome to all the new subscribers!!
It is back-to-school season for so many, and I know a lot of you have told me you are overwhelmed and too busy to keep up with reading about all the new reading studies coming out. Therefore, instead of presenting a new study in the “Deep Dive” section this week, I am re-posting a few old write-ups of studies that came out in the past few months.
A Meta-Analytic Review of Comprehension Deficits in Students with Dyslexia (open access - in press @ Annals of Dyslexia)
“Beyond the established difficulties of individuals with dyslexia in word recognition and spelling, it remains unclear how severe their difficulties in comprehension are. To examine this, we performed a meta-analytic review. A random-effects model analysis of data from 76 studies revealed a large deficit in reading comprehension in individuals with dyslexia compared to their chronological-age (CA) controls (g = 1.43) and a smaller one compared to their reading-level (RL) matched controls (g = 0.64). Individuals with dyslexia also differed significantly from their CA controls in listening comprehension (g = 0.43). Results further showed significant heterogeneity in the effect sizes that was partly explained by orthographic consistency (the deficits were larger in languages with low orthographic consistency) and vocabulary matching (the deficits were larger in studies in which the groups were not matched on vocabulary). These findings suggest, first, that individuals with dyslexia experience significant difficulties in both reading and listening comprehension, but the effect sizes are smaller than those reported in the literature for word reading and spelling. Second, our findings suggest that the deficits in reading comprehension are likely a combination of deficits in both decoding and oral language skills.”
Improving English reading fluency and comprehension for children with reading fluency disabilities (open access via shareable link)
“In the English language, students who read words accurately but have impairments in reading fluency are under-studied. The associated difficulties they have with comprehending text make it particularly important to delineate effective interventions for these students. Counter to suggestions that these readers need interventions focused on text reading, we examined the effects of a decoding-focused intervention. The intervention targeted decoding-related skills, including speeded training on sublexical spelling patterns. We examined the efficacy of this program for students with fluency-defined disabilities, and compared gains to those for students with accuracy-defined disabilities. In the initial phase of the program, readers with fluency-defined disabilities made greater gains in fluency, while readers with accuracy-defined disabilities made larger gains in word reading accuracy. The mean fluency score for readers with fluency-defined disabilities came within the average range across the intervention, as did reading comprehension for both groups. Readers' mastery on speeded learning of sublexical spelling patterns predicted unique variance in fluency outcomes, beyond variance accounted for by pre-test fluency and word reading accuracy. The results support intervention approaches focused on decoding-related skills for students who have fluency-defined disabilities and are consistent with theories of reading fluency that identify a role for automaticity with sublexical spelling patterns.”
Effects of Integrated Literacy and Content-area Instruction on Vocabulary and Comprehension in the Elementary Years: A Meta-analysis
“This study synthesized impacts of integrated literacy and content-area instruction (i.e., science, social studies) on vocabulary and comprehension outcomes in the elementary years (i.e., kindergarten through fifth grade). A systematic search of the extant literature identified 35 (quasi)experimental studies. Random-effects models were used to combine effect sizes across studies. Results of meta-analysis revealed that the overall effects were positive and significant for vocabulary (effect size [ES] = 0.91) and comprehension (ES = 0.40). Moreover, a significant positive effect was observed for standardized comprehension outcomes (ES = 0.25), but not for standardized vocabulary outcomes. Supplementary analysis including studies with content knowledge outcomes demonstrated the positive and significant overall effect for content knowledge (ES = 0.89). In addition, no significant moderators of the effect sizes were found among features of research design and characteristics of interventions, perhaps partly due to the small number of studies. The results of our meta-analysis indicate that integrated literacy and content-area instruction has potential to enhance vocabulary words taught to students and comprehension in the elementary years, with the additional benefit of simultaneously cultivating science and social studies knowledge.”
The assignment and distribution of the dyslexia label: Using the UK Millennium Cohort Study to investigate the socio-demographic predictors of the dyslexia label in England and Wales (open access)
“…Using secondary data from the UK’s Millennium Cohort Study, this paper examines the socio-demographic factors that predict whether a child’s teacher identifies them as dyslexic at age 11. Gender, season of birth, socio-economic class and parental income are found to be significant predictors of the dyslexia label. Therefore, factors seemingly unrelated to the clinical aspects of dyslexia influence whether a child is identified as dyslexic in England and Wales. This suggests that label may not be evenly distributed across a population; furthermore, it may also indicate that resources for support may not be fairly allocated. The findings further support the argument that a ‘dyslexic sub-group’ within poor readers is created due to the impact of environmental factors.”
Dual-stage and dual-deficit? Word recognition processes during text reading across the reading fluency continuum (open access)
“Central questions in the study of visual word recognition and developmental dyslexia are whether early lexical activation precedes and supports decoding (a dual-stage view) or not (dual-route view), and the locus of deficits in dysfluent reading. The dual-route view predicts early word frequency and length interaction, whereas the dual-stage view predicts word frequency effect to precede the interaction effect. These predictions were tested on eye movements data collected from (n = 152) children aged 9–10 among whom reading dysfluency was overrepresented. In line with the dual-stage view, the results revealed an early word frequency effect in first fixation duration followed by robust word length effect in refixation probability and an interaction of word frequency and word length in summed refixation duration. This progression was advanced in fluent reading to be observable already in first fixation duration. Poor reading fluency was mostly explained by inflated first fixation durations, and to stronger word frequency and length effects in summed refixation duration. This pattern of results suggests deficits in early letter encoding and slowness in serial grapheme-phoneme conversion. In contrast to the widely held belief, the holistic orthographic processing of words seemed to be intact.”
A Comparison of Priors When Using Bayesian Regression to Estimate Oral Reading Fluency Slopes
“…Findings suggest that the prior specification for the residual distribution had, on average, a trivial effect on the accuracy of the slope. However, specifications for growth rate and precision of slope were influential, and virtually all variants of Bayesian regression evaluated were superior to OLS. Converging evidence from both simulated and observed data now suggests Bayesian methods outperform OLS for estimating CBM growth slopes and should be strongly considered in research and practice.”
Do you know what you are reading for? Exploring the effects of a task model enhancement on fifth graders' purposeful reading
“Drawing on theories of purposeful reading, this study explored task conditions that support fifth graders' selective reading of documents in order to answer questions. More specifically, we investigated whether promoting students' elaboration of their task model increases task performance…Participants provided more correct answers in the task model condition than in the control condition, but there was no difference in response time. Unexpectedly, eye movement analysis did not reveal any differences between conditions concerning the number of visits and visit time to relevant and irrelevant areas of the document…A better representation of the information to be searched facilitated students' identification of relevant information. Unexpectedly, however, the task conditions did not result in any significant difference in students' patterns of document search.”
Does Co-Teaching Improve Academic Achievement for Students with Disabilities? A Meta-Analysis
“Co-taught students benefitted more than their special education classroom peers…Co-teaching was more beneficial for secondary rather than for elementary students…Several studies omitted important descriptors thus more rigorous future studies are needed…”
Re-posted from June 25th:
Comparing Technology-Based Reading Intervention Programs in Rural Settings
Paper here.
Note for newcomers. In the “deep dive” section, I usually summarize a recent article that has more direct implications for practice, yet was published behind a paywall. making it hard to access by practitioners- the people who, arguably, need it most!
I wanted to cover this particular paper because it presents research on two reading programs that are frequently asked about in the popular reading science Facebook groups: Lexia and iStation.
Background
Technology (reading software programs) has been put forth as a solution for improving reading outcomes when schools lack resources and have poor teacher training. Specifically, technology can increase the teacher to student ratio when providing intervention and can have the expertise baked in to the product, so that teacher training/professional development is not as urgent. However, there is very little research on these reading software programs (referred to as Integrated Learning Systems (ILSs) in the paper).
“However, the evidence base for ILSs is concerningly thin, despite their near unre- strained use within schools. To address this gap, this study investigated the effectiveness of two ILSs to resolve read- ing deficits among students at-risk.”
Rationale
ILSs are being rapidly adopted in Tier 2 RTI frameworks to provide intervention to struggling readers. Tier 2 interventions must have evidence demonstrating effectiveness, but this research is lacking. To be clear, the authors distinguish this from use of ILSs as a supplement to teacher-driven, in-person instruction.
There are a few studies that show positive effects, but the effects are generally small and the overall research base is limited. That is why the authors decided to conduct this study.
“Limited research exists on the effectiveness of ILSs for students at-risk and students in rural areas, com- pared with traditional in-person Tier 2 intervention, and with the programs’ blended model implemented.”
Research Questions
Research Question 1 (RQ1): Does the implementation of Lexia and/or iStation—including the blended model supplements—for students identified as at- risk of reading deficits result in gains in literacy beyond that observed in a BAU condition?
Research Question 2 (RQ2): Which of the two pro- grams result in the greatest gains in literacy when IT is considered (i.e., efficiency)?
Methods
Experimental Design
The authors used two separate 2x2 experimental factorial designs. The first variable was condition (BAU-business as usual or an ILS) and the second variable was time of test (pretest or posttest).
One school used iStation and the other used Lexia
Participants were randomly assigned to BAU (business as usual) or the ILS for that school (iStation or Lexia)
Participants
Students came from 2 rural, mid-western elementary schools in neighboring districts
In the first school, 17, 13, 12, & 5 students were found eligible in grades 1, 2, 3, & 4 (respectively) and in the other school it was 20, 11, 10, & 7
Within one school, 24 students were assigned to iStation, and 24 to BAU. The same was true for the second school, except 24 were assigned to Lexia and 24 to BAU.
The two school used different measures to screen for children. One school used DIBELS while the other used STAR by Renaissance Learning.
Dependent Variables
The Woodcock-Johnson IV: broad reading, basic reading, and reading fluency clusters
Fastbridge CBMs: readingCBM and COMPefficiency
An adapted version of the Behavioral Observation of Students in Schools instrument
Intervention
Lexia
Lexia is an adaptive reading program that teaches phonological awareness, phonics, structural analysis, fluency, vocabulary, and comprehension. More on Lexia here.
Each student worked 1 on 1 with the Lexia program with their own computer
A reading specialist was trained in the Lexia program and oversaw the Lexia intervention group
All supplementary instructional recommendations made by Lexia were followed by the reading specialist
iStation
iStation is adaptive and works through the 5 components listed in the National Reading Panel (phonemic awareness, alphabetic knowledge, fluency, vocabulary and reading comprehension). More about iStation here.
If a child does not master a skill, iStation provides instructional recommendations for 1 on 1 work for the teacher and student. It is important to note that for this study, all of these recommendations were followed.
Each student worked 1 on 1 with iStation and the teacher redirected/ helped as needed
One interventionist (who was trained on the iStation program) oversaw the students who received the iStation intervention
Results
“The two ILSs performed similarly to traditional pull-out intervention across both studies. One main effect was found favoring iStation, and no interactions were observed in either case. In a sense, this is reassuring. Technology is expensive (i.e., purchasing software and tablets), but likely less so than hiring, training, and maintaining additional faculty and staff. Underserved schools also have an elevated risk of teacher turnover (Redding & Henry, 2018).”
They also reported on instructional time (IT)
“Regarding the two computer-based intervention, Lexia required less than half the amount of time to implement with fidelity relative to iStation, 155.01 average minutes of IT versus 414.30 average minutes of IT, and also required less time to implement than the BAU, when considering allocated time divided by the number of participating students.”
Limitations
The small sample size limited analyses within the grade levels.
One intervention was administered by a school support staff while the other was administered by a reading specialist. While both were trained on the ILS being used, it could have influenced results.
They measured instructional time based on designated time not actual time spent in intervention.
The did not measure long-term gains.
Practical implications/ Take-home Message
“For schools facing financial constraints, rural schools particularly so, ILSs hold promise for serving as a supplement or replacement to traditional Tier 2 intervention.”
Another “Deep Dive” re-posted from May 7th:
A multi-sensory tutoring program for students at-risk of reading difficulties
Evidence from a randomized field experiment
By Anna Aldenius Isaksson, Martin Bøg and Jens Dietrichson
From the corresponding author: “Unfortunately, I do not have an open access copy of the article version but there is a working paper that is open access. It contains all the main results, as well as some sensitivity tests that did not make into the published version due to space constraints (the working paper version is actually also peer-reviewed, our funders provided that). The working paper can be downloaded here.”
Background on why I chose to cover the paper
I chose to cover this research paper because I think small, randomized, controlled experiments have incredible power to inform educational practice and because of the recent Orton-Gillingham meta-analysis. I think there was a slight uproar among the reading community when OG (which is known for multi-sensory phonics instructions) was not found to be as effective as people assumed. I think this paper, however, does find strong support for multi-sensory tutoring of Swedish children with reading difficulties.
*Note. this study used Swedish children as participants, “Swedish has for example a more transparent orthography than English (but more opaque than e.g., Finnish and German), which could make phonological instruction less important.”
Background
We still don’t know what makes certain interventions yield better outcomes, and there is still a lot of variation in effect sizes even among similar interventions.
Rationale
“We use a randomized field experiment with 161 students in 12 Swedish schools to examine the short-term impact of a literacy program—Läsklar (“ready-to-read”)—developed by the third author.”
The Intervention
“The program, which targets the students most at-risk of reading difficulties in kindergarten and first grade, combines three main components: (1) it uses multi-sensory methods aimed at activating many senses and thereby enhancing memory formation and retrieval; (2) it focuses instruction on phonological awareness and phonics; and (3) students are tutored one-to-one or one-to-two by teachers.”
Frequency & duration
3-4 times a week for about 8-10 weeks, over 30-35 total session and each session lasts about 10-15 minutes
Teacher/Tutor Training
was accomplished vis short films and on-site visits to model the intervention and answer any questions
Specific details of the sessions are in the open access working paper linked above.
Participants
Students were randomized within schools (161 students total in 12 schools)
130 in K; 31 in 1st
45% of the children were female
24% of the entire sample had some sort of specific risk (language disabilities or other)
Randomization & Assignment
They chose to randomize students either to the intervention (“Lasklar”) or a waiting list (who eventually got the intervention). They did this for two reasons. First, every student will get the intervention which makes schools more willing to participate. Second, it decreases the likelihood that those on the waiting list will seek outside tutoring in the meantime (which could confound the results).
There is a detailed explanation of the randomization procedures used in the working paper linked above.
After randomization, but before the start of the experiment, the intervention and the wait-list control group were checked for differences on key variables that could influence the outcomes and no differences were found between groups.
Waitlist Control Group
The waitlist control group received “business as usual” instruction in their classrooms while they waited to receive the intervention, “The regular kindergarten instruction in most schools was based on some version of the Bornholm model (e.g., Lundberg et al., 1988; Häggström, 2007). The Bornholm model uses structured metalinguistic games and exercises that aim to help students discover the phono- logical structure of language. The games include listening to verbal and nonverbal sounds, rhyming games and rhymed stories, segmentation of sentences into words and investigation of word length, clapping hands to mark syllables, recognizing phonemes in the initial position of and within words, and prosodic games (see, e.g., Lundberg et al., 1988). The Bornholm model was always implemented in whole class.”
Outcome Measures & Testers
Children were tested pre and post intervention on Decoding, Letter Knowledge, Phonological Awareness, and Self-efficacy, enjoyment, and motivation. Only Decoding and Letter Knowledge were standardized tests, the others were researcher-developed
Testers were blind to student condition (intervention or control)
Implementation Fidelity
The authors state that they did not measure implementation fidelity so as to make the trial “…as naturalistic as possible.” I found this idea interesting, because there is so much emphasis on implementation fidelity, but really, I guess, you would want an intervention to “work” even if there was not high implementation fidelity. This study was very pragmatic in the sense that they wanted to keep routines as naturalistic as possible (what teachers/schools would be doing even if the researchers never approached them). They designed their study around such known teacher and school behaviors, rather than designing a study “in the lab” then providing a heavy arsenal of support for a short time period, then leaving and hoping teachers would maintain it. That said, they did follow up with teachers to ask how many sessions were actually implemented. Only a few children received drastically below the target level of 30-35 sessions.
Analysis
They pre-registered the analysis plan- which is something I do not often come across! That simply means that, before they even started the study, they specified which statistical analyses they would run. This means that the likelihood of p-hacking is decreased. In the interest of space, I am not going to detail all the analyses here. You can check out the full description in the working paper linked above.
Results
“We find large positive effects on our two primary outcome measures, a standardized test of decoding (g = 1.07) and a standardized test of letter knowledge (g = 1.03), and positive effects on measures of phonological awareness (g = 0.56) and self-efficacy (g = 0.57). The effects are insignificant on measures of enjoyment and motivation, which may be explained by ceiling effects.”
The effect sizes they found were quite large in relation to separate benchmarks for measuring effect sizes such as the What Works Clearinghouse.
Furthermore, they go on to do a cost effectiveness analysis on page 36, comparing their results to other similar programs, “The program costs about 6,400 SEK/700 USD/630 Euro/17 teacher hours per treated student the first time a school implements it (and gets cheaper in further iterations), which together with the large effect sizes implies that the program compares favorably to most other similar programs in terms of cost-effectiveness.”
While the recent OG meta-analysis may not have found strong effects for OG style interventions, there is not enough evidence to say that multi-sensory interventions are generally not effective in improving early reading outcomes. I think the conclusions that the authors of the OG meta make regarding legislation are important: OG should not be a mandated intervention - like it is in some states.
That said, the authors of the OG meta still state OG style interventions are promising, and so do other researchers in the field of reading research: “We agree that the scientific evidence presented in this meta-analysis supports this cautious conclusion. Research does not suggest that OG interventions do not work. Instead, research findings do not provide firm evidence of effectiveness for OG interventions, although the mean effect size of 0.32 in favor of OG interventions constitutes evidence of promise and suggests the need for future research.” That quote taken from this open access commentary on the OG meta.
Overall, I think this study shows us it is too soon to dismiss the effectiveness of multi-sensory early reading interventions. I also think the study did a nice job taking naturalistic processes into consideration and looking at variables like cost, lack of implementation fidelity, etc.