June 11th~ A model-based meta-analytic examination of specific reading comprehension deficit; Teaching Reading to African-American Children; Improving Academic Vocabulary for Adolescent Students
The Weekly Newsletter That Keeps You Informed of the Latest Reading Research!
Updates
No updates
A model-based meta-analytic examination of specific reading comprehension deficit: how prevalent is it and does the simple view of reading account for it?
“Many individuals with poor reading comprehension have levels of reading comprehension that are consistent with deficits in their ability to decode the words on the page. However, there are individuals who are poor at reading comprehension despite being adequate at decoding. This phenomenon is referred to as specific reading comprehension deficit (SRCD)…The prevalence of SRCD is best represented not as a single number but as a continuous distribution in which prevalence varies as a function of the magnitude of the severity of the deficit in reading comprehension relative to the level of decoding. Examining the joint distribution of the residuals with reading comprehension makes clear that the phenomenon of reading comprehension that is poor relative to decoding occurs throughout the distribution of reading comprehension skill. Although the simple view of reading predictors of listening comprehension and decoding makes significant contributions to predicting reading comprehension, nearly half of the variance is unaccounted for.”
Improving Academic Vocabulary for Adolescent Students With Disabilities: A Replication Study
“Creating Habits That Accelerate the Academic Language of Students (CHAAOS) is a vocabulary intervention developed by O’Connor et al. to improve the academic vocabulary of middle school students with disabilities. This study was designed as a replication of O’Connor et al.’s study; CHAAOS lessons were taught to 33 sixth graders who received special education services for disabilities in English/Language Arts classes. Researcher-designed vocabulary and comprehension assessments for the taught words were administered pre- and post-instruction. The performance of students in this study was compared with the performance of students in O’Connor et al.’s study. Results demonstrated that the present students made similar gains in vocabulary and comprehension compared with the previous CHAAOS students and scored significantly higher than the previous comparison students who did not receive any instruction on the words. These findings support the use of CHAAOS for improving the academic vocabulary of students with disabilities.”
Evaluating a Screening Battery for Identifying Risk for Reading Failure in First Grade (open access conference paper)
I am not sure if this paper is peer-reviewed. Some conferences have peer-review, but not all, and I am not sure what AERA’s policy is.
“Of the many RTI implementation problems, the focus of our paper is on the need for better screening for early identification of elementary school students at risk for reading failure. In our reading intervention research, we use a battery of brief, objective, norm-referenced, reliable and valid assessments….The combination of predictors correctly classified 97.4% of the cases, with Grade 1 TOSREC scores and kindergarten CELF-5 Sentence Comprehension scores as significant predictors of Grade 1 RTI status. Acknowledging the limitation of a small convenience sample, our finding that a small set of screening tests can accurately identify students needing RTI by the beginning of first grade is promising and warrants further investigation.”
Teaching Reading to African-American Children: When Home and School Language Differ
open access review/commentary- not a research paper/scientific study
Fostering word fluency of struggling third graders from Germany through motivational peer-tutorial reading racetracks (open access)
“Results demonstrated a significant performance increase in the treatment group, relative to the control group. The effect size can be considered very high (partial η2 = .76), indicating that this brief training has the potential to enhance the word recognition of struggling elementary students.”
White matter in infancy is prospectively associated with language outcomes in kindergarten
“Findings (open access) link white matter in infancy with school-age language abilities, suggesting that structural organization in infancy sets an important foundation for long-term language development.”
When two vowels go walking: An ERP study of the vowel team rule
“In an event-related potential (ERP) study of the vowel team rule in American English (“when two vowels go walking, the first does the talking”), we used a visual lexical decision task to determine whether words that do (e.g., braid) and do not (e.g., cloud) follow the rule elicit different processing, and to determine if this extends to nonwords (e.g., braip, cloup). In 32 young adults, N1 amplitude distinguished between rule-following and rule-breaking items: N1 amplitude was more negative to rule-breaking words and nonwords. In contrast, there were no significant effects of vowel team rule adherence on N400 amplitude. Behaviorally, participants responded more quickly and accurately to rule-following words, a pattern not observed for nonwords. These findings demonstrate that adherence to the vowel team rule can be indexed by both neural and behavioral measures in fluently reading young adults.”
Anxiety, motivation, and competence in mathematics and reading for children with and without learning difficulties (open access preprint)
“Knowledge of the relations among learners’ socio-emotional characteristics and competencies as they engage in mathematics and reading is limited, especially for children with academic difficulties…participants with higher competence had lower anxiety and higher motivation. Higher anxiety was also associated with lower motivation.”
Statistical and explicit learning of graphotactic patterns with no phonological counterpart: Evidence from an artificial lexicon study with 6–7-year-olds and adults (open access preprint)
“Children are powerful statistical spellers, showing sensitivity to untaught orthographic patterns. They can also learn novel written patterns with phonological counterparts via statistical learning processes, akin to those established for spoken language acquisition. It is unclear whether children can learn written (graphotactic) patterns which are unconfounded from correlated phonotactics…In post-tests, children and adults incidentally generalized over such context-based constraints that varied in complexity. Explicit instruction further benefitted pattern generalization, supporting the practice of teaching spelling patterns, and there was a relationship between explicit learning and literacy scores. We are first to demonstrate that statistical learning processes underlie graphotactic generalizations among developing spellers.”
Implementing Vocabulary Strategies to Improve Reading Comprehension for English Learners (dissertation- not yet peer-reviewed)
Commercially Developed Tests of Reading Comprehension: Gold Standard or Fool’s Gold?
A Commentary authored by Nathan Clemens and Doug Fuchs- located here.
I usually don’t cover commentaries in the “in-depth” section, but this one was really interesting, and it wasn’t open access, so I thought it would be beneficial to share some of their main points more broadly.
Background
Reading comprehension, as a psychological construct, is a difficult skill to assess, given that it is comprised of several interrelated sub-skills that are often hard to tease apart. In order to better understand the effects of an intervention, many reading comprehension intervention studies use several outcomes measures. For example, they will use both 1) researcher-created reading comprehension tests (which often have similar or identical passages as those used in the intervention), and 2) commercially-developed tests of reading comprehension like the Weschler Individual Achievement Test or the Gates–MacGinitie Reading Test.
Generally speaking, larger effect sizes are seen on researcher-created tests of reading comprehension compared to the commercially-developed tests. This has led people to believe that maybe the skills that were taught did not transfer or generalize to passages/text that were not as similar to the passages/texts that the students were trained on in the intervention:
“The stronger performance on researcher-made tests has tended to be discounted because of the alignment between the researchers’ tests and interventions. Weaker performance on the commercially developed tests has been interpreted as program participants’ failure to generalize from what was learned during intervention to measures of reading comprehension unaligned with it (Slavin, 2019).”
A Differing Viewpoint
The authors, however, argue that the above belief might not be valid.
But first, Why Did Commercial Reading Tests Become the Gold Standard?
The authors point to statements made by Slavin (2019) on Best Evidence Syntheses and the What Works Clearinghouse that led people to believe that researcher-created tests were “over-aligned” with the intervention and should
“Reports of effect sizes from researcher/developer measures should be treated as implementation measures, not outcomes. The outcomes emphasized should only be those from independent measures” (para. 9).”
The What Works Clearinghouse (2020):
“A third requirement of outcome measures [in efficacy studies under review by the WWC] is that they not be overaligned with the intervention....When outcome measures are closely aligned with or tailored to the intervention, the study findings may not be an accurate indication of the effect of the intervention. (p. 84).”
Gersten et al. (2005)(as well as statements from IES and NSF) take a more “middle-of-the-ground approach”, suggesting that a range of outcome measures be used in studies.
That said, the authors feel as if this more balanced approach has not been widely recognized, as meta-analyses often exclude studies that do not include measures of commercially-developed test of reading comprehension.
While the authors acknowledge that some researcher-developed comprehension tests suffer from problems such as poor reliability, they also state that commercially-developed tests deserve the “…same degree of inspection (and skepticism) typically shown researcher-made measures.”
Issues with Commercially-Developed Tests of Reading Comprehension
Reading Comprehension is complex
Their main argument against commercially-developed reading comprehension tests has to do with the complexity of reading comprehension itself. Since there is no universally accepted definition of the construct of reading comprehension, test developers choose to emphasize certain aspects/components of the construct over others, leading to an “underrepresentation of the construct.”
Is a Single Test Really Measuring All of Reading Comprehension?
Put simply: commercially-developed tests focus on speed (needs to be administered quickly) and accurate scoring (multiple choice vs. more open-ended formats). But, by focusing on these two priorities, they might not actually be measuring reading comprehension fully, and that is where researcher-developed measures can help out. Furthermore, because there is no accepted definition of the construct of reading comprehension, existing tests have been found to have relatively low correlations with each other.
Bringing it All Home with An Example
There was one section of the commentary that really brought it all home for me. On page 5 the authors really hone in on whether it makes sense to use commercially-developed tests of reading comprehension. They mention that one of the most commonly used commercial reading comprehension tests is the Passage Comprehension subtest from the Woodcock-Johnson Tests of Achievement. But, this subtest is comprised of short passages in a cloze format (fill-in-the-blank/multiple choice) and performance on it is controlled, in a large part, by a student’s decoding ability. So;
“If the intent of a developer of a reading comprehension program is to strengthen inference making, say, in science and social studies texts, does it make sense for the developer to measure program efficacy by using the Passage Comprehension subtest? We suspect that many would say no. Yet, more than a few program developers do precisely this (or its equivalent).”
So, What Now?
Researchers And Test Developers Should be Aware of A New Design Framework…
Each of the following components can be conceptualized along an alignment continuum (and alignment refers to the overlap with the specific instructional program being used in the intervention).
Strategies/Skills (narrow vs broad focus)
Passage Content (do the passages being used correspond “tightly, loosely, or not at all”?)
Layout & Format (similar to the intervention or different?)
Question Types (e.g., cloze, multiple-choice, retell, open-ended for question types)
Response Modes (written, verbal, both?)
If you want more detail on the above dimensions I suggest reaching out to the authors to get a copy of the paper which has a nice table/matrix with more detail.
How to Use the Framework?
“Frameworks such as this may also help researchers describe their tests, and the connections between their tests and hypothesized mechanisms of change, more clearly and persuasively. Such a framework may help researchers operationalize, clarify, and justify the use of self-made and commercial tests, thereby providing a basis for clearer and more productive explanations of a program’s effects.”
Take-Home Point
Commercially-developed tests of reading comprehension should not blindly be regarded as the “gold standard” outcome measure. Researcher-developed tests can provide a more nuanced picture of what was actually learned. For practitioners, I think the key take-home point is that there is not a universally accepted definition of reading comprehension and different tests might be measuring different things. This latter point is covered more in this paper by Dr. Clemens and and others.