January 29th ~ Educator Perspectives on Data-Based Decision Making: Syllable-level Hyphenation; Scope & Sequence Project
The Weekly Email That Keeps You Informed of the Latest Reading Research!
Welcome to the Reading Research Recap, a weekly newsletter featuring the latest reading research published in peer-reviewed scientific journals. The goal of the Recap is to share recent scientific findings and foster an appreciation of science as a way to navigate the world. I try to make this one of the most informative emails you get each week. If you enjoy this issue, please share it. I am always interested in improving the newsletter and welcome feedback.
Welcome! This is Issue No. 36
“Science means constantly walking a tightrope between blind faith and curiosity; between expertise and creativity; between bias and openness; between experience and epiphany; between ambition and passion; and between arrogance and conviction – in short, between and old today and a new tomorrow.” — Henrich Rohrer
News/Notes
I’m working on a project that compares the scopes and sequences of different reading programs and I would love feedback! Scroll to the bottom of the newsletter to read more about it. (Just FYI you can always send me feedback by replying to this email directly. It goes right into my gmail inbox.)
The images in the scope and sequence project may cause your email carrier/server to clip this message because it is over most email limits…so, you might have to click a link that says '“click to view entire message”
Research Highlights 📚
Treatment Fidelity: What It Is and Why It Matters
“Our take‐home message is that when one is interpreting intervention studies, it is important to consider whether the interventions were applied as planned, or with fidelity.”
Impaired Perception and Neural Processing of Rules in Developmental Dyslexia
“Based on these results, the speech processing difficulties in dyslexia extend beyond phoneme discrimination and basic auditory feature extraction. Challenges in implicit extraction and effortless adoption of complex auditory rules may be central for language learning difficulties in dyslexia.”
A Critical Evaluation of Dyslexia Information on the Internet
“We found the majority of webpages do not meet basic source credibility criteria, much of the content contradicts or is unsupported by research, and most pages convey information in an authoritative discourse, making it seem irreproachable. Building on the findings, we offer criteria for evaluating dyslexia information and suggestions for research and practice. We focus on the need for less divisive, more collaborative dialogue, along with research among stakeholders with multiple perspectives.”
The Effect of Syllable-level Hyphenation on Novel Word Reading in Early Finnish Readers: Evidence from Eye Movements (Open Access!)
“Target words were read faster with repeated exposure but there were no effects regarding grade or hyphenation. The use of hyphenation does not give rise to enhanced processing of phonology in novel words and is likely to hinder the processes connected to the use of orthography.”
Executive functions and English reading comprehension growth in Spanish-English bilingual adolescents
“Results provide cautious support for the importance of executive functions to reading comprehension development for bilingual adolescents and suggest multiple pathways by which they contribute via established componential skills.”
Deeper Dive
Educator Perspectives on Data-Based Decision Making in Germany and the United States
Background
Using data to inform instruction leads to better student outcomes. Therefore, there is a growing interest in how data-based decision making (DBDM), also referred to as ‘data-based individualization’ (DBI) is being implemented.
Data can be formal (systematically collected qualitative or quantitative information), or informal (perhaps extracted from a conversation). It can come from different levels: school system, school, classroom, or individual students.
School administrators and school culture play a large part in whether and how data is used to inform and individualize instruction
Germany, unlike the US, does not have established frameworks for implementing DBDM, which is why the authors wanted to compare the two countries.
Rationale
“The purpose of this study was to examine how educators in Germany and the United States understand and approach data and DBDM.”
Methods & Sample
They gave a survey to 116 general and special education teachers in the US and 87in Germany from a variety of states (in both countries).
They analyzed the answers to the open-ended questions using qualitative text analysis to identify themes using predetermined criteria.
Results
Data Use
“Interestingly, respondents in Germany indicated less usage of student progress data than respondents in the United States. Perhaps this is because of the focus on Multitiered Systems of Support (MTSS), RTI, and DBI in the United States at this time in which progress monitoring is an essential component.”
“Another unique finding was the focus on lesson observations/observation data and on student behavior data by participants from the United States. In the United States, there are many academic frameworks or models that schools adopt that rely heavily on observational data, for instance guided reading (Fountas & Pinnell, 1996), which might point to the focus on data that aligns with these types of more holistic, observational frameworks.”
Data Characteristics
German teachers reported using data more on individual students versus classrooms, for which they provide the following reasoning: “One explanation for this could be that in the United States there is a higher level of comparison between classes within a school or district given the emphasis on high-stakes tests in which classroom-level data is reflective of teacher performance…”
DBDM Characteristics
“Importantly, participants in both countries attached great importance to the subcategory of instructional goals and the decisions that educators make in their everyday lives with foci on instructional changes, monitoring progress, and refer self-evaluation results to inspection.”
Implications
They authors state that there needs to be more professional development and coaching around DBDM and that there needs to be more of an emphasis on DBDM best practices in pre-service coursework for educators.
Thanks for reading!
Scope & Sequence Project
General Notes
What is this?
The short version:
The purpose was to design a simple, graphical way for teachers (parents, or administrators) to compare scope and sequences (the introduction of letter-sounds) across different reading programs. This is not the finished product, but a draft. I know that it is hard to make comparisons in this format and was thinking of developing a percent match or “most alike” algorithm. Let me know what would be useful!
The (much) longer version:
This project grew out of a chapter of my dissertation in which I was comparing the scope and sequences of explicit phonics instruction (grapheme-phoneme correspondences) across the major popular reading programs for first grade.
Only the programs that were in my dissertation were double-coded and interrater reliability for those was high.
It was cost-prohibitive to access the Teachers Editions for all of these, so I used the online scope and sequence (made available by the publishers) instead. One would think the online scope and sequence would be accurate and detailed, but I found almost all of them to be extremely confusing (see my notes below on the individual programs). I reached out to several companies and some were able to clarify sources of confusion, some were as confused as I was, and other companies did not understand the importance of listing the sound and the spellings (graphemes) in a scope and sequence.
If you find an error, let me know!
This is not…
This is not a list of the best phonics programs and I think it would be dangerous to make inferences like that based on sequence or coverage. For example, just because Levelled Literacy Intervention covers many GPCs, it does not necessarily mean that it is a better program than one with less GPCs taught. Explicit, systematic teaching of phonics is really important, and this list has nothing to say about the quality of phonics instruction. Another important point is that the scope and sequences listed here simply denote the first time a GPC was introduced, and it does not display the time spent teaching or reviewing that particular GPC.
How to read the chart
Reading programs and book sets are listed alphabetically down the first column
GPCs (letter-sounds) are listed alphabetically across the top column
I used a master chart of about 369 GPCs, but started with the GPCs listed in the Common Core State Standards
Pink indicates the GPC was presented early on in the sequence, purple = somewhere in the middle, and blue= toward the end
Full charts
A - EU
EW-N
O-SH
SH-Z
Some Takeaways from the chart…
Aside from a few columns of bright pink (generally the common consonants), I would say there is (moderate?) variation in the order that reading programs (and book series) introduce letter-sounds. I would be curious what others think.
Variation in GPC scope and sequence has implications for alignment of core and supplemental programs, choosing additional reader/book sets, and even curricula used by outside service providers/tutors.
There is some evidence that students who receive supplemental instruction that reinforces phonic concepts being learned in their core instruction made greater gains than students who received supplemental instruction in new, but related, material (Wonder-McDowell, Reutzel, & Smith, 2011).
Other Notes
Possibly comparing apples to oranges?
While the chapter in my dissertation focused only on the major reading programs used in first grade, I’ve since expanded the scope to include book series such as decodable readers, and lesser- known reading instructional programs.
One might make the argument that this is like comparing apples to oranges. I would agree, however, from a practical standpoint, I think it is useful since educators are often looking to supplement their existing program with supplemental programs, or additional decodable books, in which case it would be important to make sure the scope and sequences align (Wonder-McDowell, Reutzel, & Smith, 2011).
Another important point is that these programs (especially decodable books) do not always align to a grade level, whereas some of the curricula are listed explicitly for certain grade levels. So, keep that in mind when looking at the differences in scopes and sequences.
Method Notes
Some introduced letter-sounds in sets/groups. I did not give them all the same number. Rather, each was assigned a number according to the order it was listed in the scope and sequence or teacher’s manual.
Individual Notes on the Programs/Books Series that I did recently (i.e., not as part of my dissertation) are listed below. So, the programs listed below were not double-coded. For methods notes on the ones that were part of my dissertation, keep scrolling.)
Power Readers: consonants are missing from the scope and sequence (?); letter-sounds (GPCs) in the “Heart Words” they listed were entered in order on my scope and sequence
Junior Learning Readers: Phase 1-6: unclear if they are listing sounds or letters (graphemes), or switching between the two; it seems like they are most likely listing letters (graphemes), but they do not state which sounds are they referring to. For example, for ‘oo’ are they referring to the sound in ‘book’, ‘moo’ or ‘flood’?
Dog On A Log Books: It was unclear if digraphs are just for the main (most common) sound or for all of them. So, for example, when it says “ch” does that mean as in “chirp”, “chorus”, or “charade”? I assumed it was just for the main/most common sound. For “silent e, including -ke” I was not sure if this included u-e as in cute and rude.
High Noon Dandelion Launchers Set 1 (books 1-7B): They used letters to represent sounds, but they did not specifically and explicitly list the sounds being represented. So, when they listed ‘o’, I assumed it was the short o sound based on inferences made from the surrounding ‘sounds’ and the scope and sequence as a whole.
Pip & Tim Decodable Books, Online Scope and Sequence: (Scroll down to see the scope and sequence). This series had similar issues as the others: I could not tell what sounds they taught based on the letters they listed. For example, for ‘ea’ did they just introduce “eat” or did they also introduce “great”? Without purchasing the books, it is hard to be certain.
Primary Phonics Levels K-6 (Books & Program), Online Scope and Sequence: They state “17 consonant sounds” for Level K, but they do not state which 17 consonant sounds they cover (I made some inferences). They also had the same issue as other programs: when they listed digraphs which could have multiple sounds, they did not state which sounds they introduced.
All About Reading Levels 1-4, Online Scope and Sequence: (Note that you have to click on one of the Levels’ Teacher Manuals and scroll to Appendix B in order to see the full scope and sequence of phonograms taught in levels 1-4). This was one of the more clear/easy-to-follow scope and sequences I saw. They showed the sounds as well as example words (graphemes), not just one or the other, like other programs.
Geodes, Online Scope and Sequence (Note: since Geodes says that it follows Wilson Fundations, I used the Wilson Fundations Levels K-3 scope and Sequence: However, I could not find it on their website, so I used their “LiveChat” feature and one of their representatives sent over a pdf. So, I recommend doing that if you want to follow my steps.) Their scope and sequence was quite confusing when it came to parsing the specific letter-sounds being taught and when. Like other programs, they did not list the full letters and sounds being taught.
Flyleaf Parts 1-14, Online Scope and Sequence: Their scope and sequence was generally clearer/more interpretable than others because the gave the sound and the spelling being introduced. I did encounter minor areas of confusion. For example, what sound is /hw/ spelled wh? Is it like whole or or like where? I had to make a few inferences.
Hill Readers Levels 1-2 (Books 1-13), Online Scope and Sequence: This scope and sequence was a bit unclear. They state that all Hill Readers contain sight words but did not list them. They also did not mention if/when consonant sounds are introduced. I made the assumption that they make the assumption that children already know the major consonant sounds.
Half Pint Kids Decodable Books Levels A-C (Sets 1-18): This scope and sequence had the same issues as others in that they did not list sounds and spellings. Also, I found it interesting that short e was introduced relatively late. They listed /sh/ as a blend, next to fr and sn which is a little confusing. They did this as well for /ch/ and others.
SPELD SA Sets 1-15, Online Scope and Sequence: Their scope and sequence was generally clearer/more interpretable than others because the gave the sound and the spelling being introduced. I did find it odd that their example word for long e was “female” (since it had two e’s with different sound representations). But, maybe that is the word used in their books. If not, they should change it to something like ‘me.’
Reading for All Learners (only Set 1!) Online Scope and Sequence: So this one was really confusing. It said 27 new sounds under the first red section, but lists upper and lower case letters. I e-mailed them to see if the upper and lower case letters represent different sounds. They were equally confused and sent me a new draft (that they said they would put up on their website soon). However, the new draft had some very confusing parts, so I stopped at set one and will continue once they publish their final draft.
Recipe for Reading Workbooks 1-8 Alphabet Series, Online Scope and Sequence: This scope and sequence had the same issues as others in that they did not list sounds and spellings.
Methods for the Programs Covered in My Dissertation
(these were mostly the larger basal reading programs put out by the big ed publishers)
Sample.
Since the goal of this chapter was to investigate differences in the scope and sequence across a wide variety of reading curricula, the author chose to examine 12 different programs representing different types of instruction: core and supplemental. Furthermore, the sample was designed to include reading curricula that varied in terms of their evidence of effectiveness for beginning readers as listed by the What Works Clearinghouse (see Table 9 for information on the included reading curricula). Several of the major educational publishers were included (Pearson, McGraw-Hill, Fountas & Pinnell, & Scholastic, etc.). Teacher’s editions for first grade core and supplemental reading curricula were located through the campus curriculum library, via interlibrary loan, or purchased. In some cases, purchasing a new set of teacher’s editions was cost prohibitive so the phonics scope and sequence was located online or by e-mailing the publisher if it could not be secured via interlibrary loan (see Table 9 for the source of information). When the scope and sequence was ambiguous (and many were), the first author reached out for clarification via e-mail to the publisher. These e-mail exchanges did not always result in clarification of which specific GPC was being taught. In this case, the benefit of the doubt was given to the publisher. Due to the somewhat ambiguous nature of these decisions, a second coder was used.
Recording of GPCs.
A comprehensive list of GPCs in the English language was modeled after Berndt, Reggia, and Mitchum (1987, Appendix B). The Berndt (1987) list of GPCs was used as a reference because it was the most comprehensive list of GPCs the author was able to locate, comprised of 369 GPCs in the English language. GPCs that were explicitly taught were given an ordered number (based on when it was taught in the reading curricula) next to the appropriate GPC on the Berndt list. Only GPCs that were explicitly taught were assigned a number. For example, if a word was listed in the lesson plan that contained a certain GPC, but the actual lesson plan did not indicate that the teacher state the sound while indicating the letter(s) (i.e. pointing, underlining, etc.) then this particular GPC was not counted as being explicitly taught. An example of this can be found on pg. 162 of the Green System Lesson Guide for lessons 1-60. The word “our” is listed as a necessary word card for the lesson, and students practice building it, but nowhere in the lesson does it instruct teachers to indicate that the letters “ou” combine to make the sound /OW/.
High frequency words or sight words that the curricula instructed children to memorize were not counted as being explicitly taught since the graphemes were not pointed out nor were the individual phonemes modeled.
Several programs taught sub-lexical units of words that were larger than graphemes. For example, S.P.I.R.E. teaches “-old” and “-ost”, but these are redundant with the GPC for o - /long o/. Therefore, if the curricula previously covered those, then they were not counted again. This also brings up an important point about coverage. This investigation recorded the first use, not the total coverage (time spent across the curricula) teaching a particular GPC.
Inter-rater reliability coding.
A doctoral student in special education served as a second coder and inter-rater reliability was calculated. While the authors initially felt that the need for an additional coder should be obsolete because the GPCs should be explicitly listed in the teacher edition or scope and sequence and, therefore; easy to interpret, we quickly found out this was not the case. For example, the teacher’s editions often explicitly taught sub-lexical “chunks” of words that were larger than GPCs. Extra care was warranted in these situations so that the appropriate GPCs could be parsed and made sure they were not redundant with earlier taught GPCS.
Another issue that frequently arose was the fact that the online scope and sequence that was used in lieu of purchasing a new set of teacher’s editions did not explicitly state which GPCs were being covered. For example, in Reading Street’s (Pearson, 2013) online scope and sequence (page 3, for Grade 1, unit 2, week 3) it states “consonant digraphs wh, ch, tch” however it does not state which sounds are covered. For example, the grapheme ‘wh’ can make the sound /H/ as in ‘whole’ or /W/ as in ‘what’. However, the scope and sequence does not state which of the sounds they are teaching. There are other instances of this in the scope and sequence as well (Unit 2, week 1, ’th’ is taught but it is unclear if they are referring to the voiced or unvoiced /TH/. For the aforementioned reasons, the author decided a second coder was necessary. Furthermore, all the data is available upon request from the author.
Analysis
Calculating inter-rater reliability.
Inter-rater reliability for scope (i.e. which GPCs were covered) was assessed by calculating both percent agreement and an unweighted Cohen’s kappa, the latter accounts for chance agreement. Percent agreement was calculated by dividing total agreements by total cases. Kappa was calculated using the ‘kappa2’ function from the package ‘irr’ (Gamer, 2019) in R version 3.5.3 (R Core Team, 2019).
What Next?
I’m not sure where I will take this side project next…depends on the feedback I get.
I’m not sure if this chart is helpful or if people would prefer a percentage alignment or an algorithm to calculate alignment…I would love any feedback you have- you can reply to this email and it goes right into my inbox!
Thanks!
Neena