Oct. 30th ~ 🎃 A Spooky Edition of the Recap: Screening Screeners; Monster P.I.; The Data Mountain Program for Increasing Reading Fluency; Robots & reading plus Survey Results
The Email Newsletter That Keeps You Informed of The Latest Reading Research!
Welcome to the Reading Research Recap, a weekly newsletter featuring the latest reading research published in peer-reviewed scientific journals. The goal of the Recap is to share recent scientific findings and foster an appreciation of science as a way to navigate the world. I try to make this one of the most informative emails you get each week. If you enjoy this issue, please share it. I am always interested in improving the newsletter and welcome feedback.
Welcome! This is Issue No. 25
“Scientists who simplify to get heard will never succeed in pleasing everyone, especially not those colleagues who think scientist should stay out of the public arena whenever there is a call for the simplification of the science. If we do avoid commenting entirely, then we abdicate the popularization of scientific issues to someone who is probably less knowledgeable or responsible. The bottom line is simply staying out of the fray is not taking the ‘high ground’ – it is just passing the buck.”
-Steve Schneider Stanford biologist
👆🏽 Changed the quote. This quote really spoke to me when I was deciding whether or not to even start the Recap back in May. As a former researcher/PhD student, I was worried about simplifying the science because science (as a process and as results/data) is really nuanced and complex. I read several science communication articles and books and eventually did decide to go forward with the Recap (obvi!) and this was one of the quotes that helped sway me…
Survey responses from last week’s survey are at the bottom of the newsletter! Thanks for the feedback!
Sometimes I make two newsletters available in one month for my readers who signed up for the free version (versus weekly for my paid subscribers). This is one of those times, because: 1) I wanted everyone to see the results of the survey if they were interested, and; 2) because I have a favor to ask of you all.
I have a favor to ask…Feedback on A Mobile App Wanted!
I have a favor to ask of my readers who are parents or teachers of children learning to read or anyone (tutors/specialists that analyze reading errors): can you help me with some product research by filling out this survey? I know people hate surveys, but it will help give me valuable feedback toward building a useful product. There is also the opportunity to leave your email if you want to be an early beta tester of my ideas/ and prototype! Thank you so much (in advance) for doing this!
Some More Background on the survey (if you care…)
Some of you know that the Recap is just a side project for me. My “real” job is building products as part of Elemeno. As such, I am interested in building products to help all readers crack the alphabetic code. This involves taking two ideas I helped develop as part of my PhD research out of the lab and creating useful products for teachers and parents.
The project pitch (grant idea) I submitted to the National Science Foundation’s SBIR/STTR program was accepted (yay!), so now I can submit a full grant proposal. This is exciting, however, I really want to make sure I build a tool that stakeholders such as teachers, parents of children learning to read, tutors, specialist, etc. find useful!! my big fear is that I get the grant, make a product, and nobody finds it useful. So, I want to do some customer discovery on the front end!
Okay, sorry for the long intro this week…onto the Research!!
📊📉Research📚
Highlights (research presented in brief)
Increasing reading fluency using the Data Mountain program (A single-case research design): “In the pilot study, students’ fluency increased to an average of 22 wpm in the self‐monitoring and goal‐setting phase, and to a further 9 wpm in the motivation training phase. In the replication study, students increased an average of 18 wpm with the Data Mountain program.” More on the Data Mountain program: https://incontext.education.utexas.edu/data-mountain-journey-educational-behavioral-success/
“This study investigates three subtle ways that struggling readers may perform differently than their typically developing peers in this experimental context: sensitivity to the frequency distribution from which speech tokens are drawn, bias induced by previous stimulus presentations, and fatigue during the course of the task.” They did not find difference on any of the three between children with dyslexia and typically-developing peers. (Open Access)
“The Reciprocal Reading programme was delivered successfully over approximately 6 months in a sample of 315 students (aged 11-13) from 14 schools in five English districts with high socio-economic disadvantage.” They found positive effects and it was easy to implement.
Pre-registered Systematic Review & Meta-Analysis preprint from a Swedish group of researchers- though there were only 4 studies that were included (Open Access, but not clear if it has been peer-reviewed yet) “Students at-risk for reading difficulties benefit from the tier 2 interventions provided in the included studies, although the CI [confidence interval] indicates the effect can be anything from trivial to just below moderate. Gersten et al.’s (2020) high quality meta-analysis found effects of Hedges’ g=0.41 in the area of word or pseudoword reading. Our present study, with stricter criteria, suggests that the findings from Gersten et al. (2020) are robust even when subjected to tougher examination. As the US national evaluation of RtI (Balu et al., 2015) found negative effects on reading outcomes for tier 2 reading interventions our result must be regarded as important to the field.”
“As such, the current study provides novel evidence for the existence of a pre-reading relation between functional measures of syllable-rate processing, structural organization of the arcuate fasciculus and cognitive precursors of reading development.” (Open Access)
The robot will now read to your kid: “Conversational agents can simulate an effective reading partner by asking children questions and providing feedback. Having dialogue with a conversational agent or with an adult enhances children’s story comprehension. Adults elicit longer, more lexically diverse, and more relevant responses while conversational agents promote intelligibility. Disembodied conversational agents have great potential in enriching young children’s language experiences.
https://journals.sagepub.com/doi/abs/10.1177/1534508420966383
Deeper Dive
How cool is it that a study on “Monster, P.I.” came out right before Halloween!
Background: “Currently, language beyond vocabulary knowledge is rarely assessed in the standardized tests that schools and teachers use to guide their instruction (see Adlof & Hogan, 2019, for a discussion). That is largely due to the complicated nature of the language problem space and the challenges in assessing language as current assessments often take a long time, are individually administered, or do not assess the multidimensional nature of the areas of language that support reading comprehension. As such, teachers do not have language data to guide their instruction, and hence, fine-tuned language instruction in schools is rare.”
Rationale for Study: “Our study explores the reliability and validity of Monster, P.I.”
Sample: “Across the 3 years of the study, we worked in an urban district in the Southeastern United States. For our first research question, we used our full sample of 3,214 fifth through eighth graders (n = 1,026 fifth graders, 742 sixth graders, 715 seventh graders, and 731 eighth graders)…” “53.4% female and 33.3% Black, 40.4% White, 21.9% Latino, 4.3% Asian, and <1% American Indian. Fifty percent of students were classified as economically disadvantaged, and 8% were identified as English language learners (ELLs).”
Methods/Analysis: Multiple-group item response modeling , examining correlations and “…hierarchical multiple regression modeling to explore links between performance on the Monster, P.I. assessment and standardized measures of reading achievement.”
Take-Home Message: “This study provides reliability and validity evidence for a CAT [computerized adaptive testing], gamified, standardized assessment of language for students from Grades 5 to 8.
Limitations: “…others have argued that considering just morphology, vocabulary, and syntax is too simplistic.” Also, they did not examine individual differences, and only had one syntax task
Practical Implications: “What is important about this assessment is that it provides scores at smaller levels (i.e., language areas and/or skills within language areas), which can inform instruction in ways that general language scores do not.”
In-Depth
Screening Screeners Note: I cannot tell if this has been peer-reviewed yet. That said, I am still covering it because it is not an intervention so much as information on screeners.
My Two Cents on The Above Study
It would be neat if someone took the extra step and made a table or list of all the popular screeners out there with information on the population it was validated in, grouped by outcome measure, and sortable by strength of correlation coefficients. That way, we would know the best screeners out there for what we are trying to predict…I think their app is really neat but (to me) it seems like the process of choosing a screener could still be simplified more. I just don’t see administrators sitting around playing with their tool, but in fairness, they did say it was for researchers, too.
Also here is a good website that goes into a little more detail (but is still accessible) about screeners using COVID as an example.
Survey responses! Thanks for the feedback!
Great, I will keep it at this length!
Great, I will keep a mix of both styles.
Hmm, I need to think about this one more. Seems like this one was more divided. Most people wanted comments from teachers/practitioners, with more detail on studies coming in second. I guess for the people who wanted more information on studies, could you email me (if you don’t mind not being anonymous) and let me know what type of detail you want? If you want me to be able to do all the studies like I treat the one “in-depth” one, that just won’t be feasible right now. But if there is something else I can address, feel free to reach out!
Additional Comments/Feedback (and my responses)
I really appreciate your work. As a teacher, its important that I am well-informed and up-to-date on the latest, relevant information. My concern is always making sure that the sources can be trusted and that it is not contradictory to what we see in the classroom. Thank you for all your work.
Thank you!!! Thank you!!!
I appreciate your efforts!
Thank you so much for this! I direct a program for dyslexics and love to stay current!
Thank you so much for providing this resource!
Keep up the great work
I'm studying your newsletter in order to put 'a recap of the recap', translated in Italian, into my newsletter.
This is an amazing service you provide. I am an academic coach dealing with blended learning. I am normally more on top of research than I am this year. Your newsletter is REALLY helping with that! THANK YOU!!!
Thank you. This is a TON of work and much appreciated. I will subscribe in a couple of months when I have more time.
I find the Recap really useful as an academic - who struggles to juggle teaching, marking etc. Thank you for doing this.
Thank you. This is a TON of work and much appreciated. I will subscribe in a couple of months when I have more time.
Thank you for your time in putting this together! I often bring the research to my team to discuss and without the Recap I wouldn't have time to find the studies myself. Huge thanks!
It is a fantastic resource and one that I have great joy in reading each week. My only feedback is that I hope you can continue to grow the Recap so that you can enlarge it and go into greater depth/provide more studies, but I think it's amazing and an immense amount of work for one person. Perhaps there may be some teachers/students out there who would like to collaborate with you and contribute to the Recap as something to put on their resumes? That might help you to grow.
Yes, I would love that! If there are teachers or students willing to help give feedback/thoughts/commentary or contribute to the summarization process in exchange for putting it on their resume please reach out to me! (You can simply respond to this email)
I can follow research better than your average teacher, but my suggestion is to leave a blurb simplifying the outcomes in everyday vernacular. We can share this with classroom teachers then who really struggle sometimes to understand research and therefore avoid reading it. BTW, thanks so much for doing this for all of us! You are so right when you explain that we are blocked by paywalls. Thank you!
I try and do this with the “implications” or “take-home” message at the end of the research articles I cover in-depth. But, I could be better about some of the acronyms, so I will be mindful. Thanks for the feedback!
Hi! I may be alone here, but I’m a mom who is interested in reading research insomuch that it helps me teach my kids. I would appreciate a more lay-person explanation of the research. For example, I don’t automatically understand what “inhibition” in kindergarten readers is; I don’t know what a CBM is, etc. Have you heard of The Skimm? Or Emily Oster? I feel like both do a great job of making things very accessible to a lay audience. Your newsletter may be targeted toward fellow PhDs / educators and I’m an outlier... but I just wanted to share my input in case you’re also hoping to reach parents! Thanks for all you do!
Thanks for this feedback! Yeah, it is hard to decide on the level of detail (enough to satisfy PhDs/academics yet accessible enough for lay people). I will try and be better about doing this. I have heard of the skimm and I will revisit it to see what they are doing and get inspiration. I do really struggle with simplifying the science too much because I do not want to misinterpret findings/lead people down the wrong path. I think I can have a “Neena’s two cents” section which is my attempt at a short, accessible opinion of what the research means for practice, with the caveat that it could be wrong, so be sure to get second opinions as well…
In your SubStack interview, you mentioned narrowing down the number of studies you analyze. If there are any that you decide not to provide details on, but would still be useful for practitioners, would you be able to include links or citations for those at the bottom of your email?
I may have misstated that in the interview: I do include all the links to any research that has direct relevance for practitioners (and even some- like neuroimaging studies- that do not). If the paper is purely stats or methods-related, then I leave it out. But, I do not fully “analyze” all the research- maybe that is confusing? I just cover 1 or 2 studies in-depth each week and then put the links to the rest that I do not cover in the “highlights” section. So, yes, all the research that is useful for practitioners that gets caught in my “net” is in the newsletter!