Archive

Posts Tagged ‘diagnostic assessment’

Response to Intervention: What Just Won’t Work

Having served as a reading specialist at elementary, middle, and high school levels (and even part-time at the community college level), I have taught numerous reading and writing intervention courses and trained teachers to doing so. With the new emphasis on Response to Intervention (RtI) voices of real-world teaching experience need to begin shouting quickly and boldly to be heard. Although I commend the International Reading Association (IRA) for assigning reading assessment a prominent role in their Response to Intervention (RtI) document; however, the language of the document betrays certain pedagogical presuppositions and is, at points, flat unrealistic. For reference, the document is found at  Let’s take a look at one section of this document to see if my analyses ring true.

On page two, the IRA Commission lists these guiding principles under the subheading of “Assessment”:

“Assessments, tools, and techniques should provide useful and timely information about desired language and literacy goals. They should reflect authentic language and literacy activities as opposed to contrived texts or tasks generated specifically for assessment purposes. The quality of assessment information should not be sacrificed for the efficiency of an assessment procedure.”

Clearly, the commission has in mind the content, form, and delivery of diagnostic, formative, and summative assessments, particularly reading assessments.

Presupposition #1 Authentic Text is Better than Contrived Text for Assessment Purposes

Since when did reading assessments have to use authentic language? As a writer of numerous reading and writing assessments, contrived text is often essential to produce an effective assessment. In fact, it is nigh on to impossible to create assessments with internal validity that don’t use contrived text. Good assessments isolate variables to ensure that we really do test what we are supposed to be testing.

One example should suffice to demonstrate how unworkable and unreliable authentic language can be when used for reading assessments. At random, I opened up to the middle (pp. 679-680) of one of my favorite novels: Fyodor Dostoyevsky’s The Brothers Karamazov. I skimmed to find the beginning of a start-to-finish passage of typical length for a one-minute fluency assessment and copied such below.  Feel free to time your reading out loud, keeping track of word attack accuracy, unknown vocabulary, and comprehension as you read.

‘Glory be to God in Heaven,

(6) Glory be to God in me…

(12) ‘That verse came from my heart once, it’s not a verse, but

(24) a tear…. I made it myself… not while I was pulling the captain’s

(37) beard, though..’

(39) ‘Why do you bring him in all of a sudden?’

(49) ‘Why do I bring him in? Foolery! All things come to an

(61) end; all things are made equal. That’s the long and short of

(73) it.’

(74) ‘You know, I keep thinking of your pistols.’

(82) ‘That’s all foolery, too! Drink, and don’t be fanciful. I love

(93) life. I’ve loved life too much, shamefully much. Enough!

(102) Let’s drink to life, dear boy, I propose the toast. Why am

(114) I pleased with myself? I’m a scoundrel, but I’m satisfied

(124) with myself. And yet I’m tortured by the thought that I’m a

(136) scoundrel, but satisfied with myself. I bless the creation. I’m

(146) ready to bless God and His creation directly, but… I must

(157) kill one noxious insect for fear it should crawl and spoil life

(169) for others…. Let us drink to life, dear brother. What can be

(181) more precious than life? Nothing! To life, and to one queen

(192) of queens!’

(194) ‘Let’s drink to life and to your queen, too, if you like.’

(206) They drank a glass each. Although Mitya was excited

(215) and expansive, yet he was melancholy, too. It was as though

(226) some heavy, overwhelming anxiety were weighing upon

(233) him.

(234) ‘Misha… here’s your Misha come! Misha, come here, my

(243) boy, drink this glass to Phoebus the golden-haired, of tomorrow

(254) morn..’

(255) ‘What are you giving it him for?’ cried Pyotr Ilyitch, irritably.

(266) ‘Yes, yes, yes, let me! I want to!’

(274) ‘E — ech!’

(275) Misha emptied the glass, bowed, and ran out.

(283)

Words Read in One Minute ____ – Miscues = ____ Net Fluency Score

How did you do? Difficult passage? Not so, according to the Flesch-Kincaid readability scores: Reading Level 1.1  Reading Ease 94.6. Average Word Length 4.0.

As illustrated above, using authentic language is far from an accurate means of assessing one’s fluency. Would you use this 1.1 grade level passage as a diagnostic assessment and follow with a Dr. Seuss 1.1 grade level passage to formatively assess progress two months later? Of course not. Most real-text reading passages of a length suitable for fluency assessments have similar variables as in the Dostoyevsky passage above: They are necessarily out of context and they include unfamiliar language, including names, idiomatic expressions, vocabulary, and culturally-based word choice.

Authentic text does not meet the standards of reliability we need to measure baseline ability or growth. The results cannot be generalized in any meaningful way. Even using the same source for subsequent fluency assessments provides no guaranteed compatibility. Most importantly, authentic language does not give the reading diagnostician the information needed to differentiate instruction. We need to isolate variables with contrived text to insure that we are using accurate reading assessments to inform our instruction. And this is true with all forms of reading assessments, including reading comprehension and phonics (mysteriously not even mentioned in the RtI document) diagnostic instruments. How could a comprehension test effectively measure how much a third-grader understands without using a controlled vocabulary? How could a phonics test measure a sixth-grader’s ability to decode without using nonsense words to isolate the variable of sight word knowledge?

Presupposition #2 Quality Assessments Must be Inefficient

On page two, the IRA Commission lists these guiding principles under the subheading of “Assessment”:

“Assessments, tools, and techniques should provide useful and timely information about desired language and literacy goals. They should reflect authentic language and literacy activities as opposed to contrived texts or tasks generated specifically for assessment purposes. The quality of assessment information should not be sacrificed for the efficiency of an assessment procedure.”

Now, the commission does not say that quality assessments must be inefficient, but by their own criteria they effectively preclude efficient assessment design, form, and delivery. See their referenced document: Standards for the Assessment of Reading and Writing developed jointly by the International Reading Association and the National Council of Teachers of English (2010) a a case in point.

It would seem that the IRA Commission wants to have its cake and eat it, too. The commission equates assessment quality with authentic language testing. Authentic language testing involves long in-context reading passages, whole-to-part (e.g. miscue analyses), no nonsense words, multiple measures, etc. and necessitates individual administration. Ever done a complete Individual Reading Inventory?  Pretty time-consuming—hours for an individual student. Individualized assessments require significant training to both correctly administer and accurately interpret results. Inefficient and flat unrealistic. Job protection for reading specialists, special education teachers, and reading coaches?

Although using inclusive language to encourage teachers to be responsible for diagnostic assessments and progress monitoring, the real-world application of the above RtI principles would be to maintain the status quo:

1. Reading specialists, special education teachers, and reading coaches as the “keeper of the keys” and 2. Intervention instruction based upon canned-all-students-start-on-page-one programs, rather than upon diagnostic assessments that will enable teachers to differentiate instruction.

In the real world, there is not enough time to assess students, according to the IRA principles. Teachers do not have the requisite training to assess, interpret data, and accurately inform their instructional decision-making, using the inefficient authentic language assessments. In fact, many of the teachers assigned to reading intervention classes are not the most experienced teachers.

My suggestions? Let’s leave our presuppositions behind and live in the real world. Let’s get off our high horses and train teachers to use simple whole-class, multiple-choice diagnostic reading assessments, so that they can effectively differentiate reading instruction for their intervention students. Sacrifice authentic language? Have a negligible impact on accuracy (debatable) by assessing whole-class? Oh, well… well worth the sacrifices, if teachers will be able to use assessments to inform and differentiate instruction for their intervention students.

Here are some free diagnostic assessments, created by a team of reading specialists, that are user-friendly, simple to score and analyze, and designed to enable teachers of all levels of expertise to differentiate reading instruction: assessments Now, that’s RtI that does work.

The Teaching Reading Strategies (Reading Intervention Program) is designed for non-readers or below grade level readers ages eight-adult. Ideal as both Tier II or III pull-out or push-in reading intervention for older struggling readers, special education students with auditory processing disorders, and ESL, ESOL, or ELL students. This full-year (or half-year intensive) program provides explicit and systematic whole-class instruction and assessment-based small group workshops to differentiate instruction. Both new and veteran reading teachers will appreciate the four training videos, minimal prep and correction, and user-friendly resources in this program, written by a teacher for teachers and their students.

The program provides 13 diagnostic reading and spelling assessments (many with audio files). Teachers use assessment-based instruction to target the discrete concepts and skills each student needs to master according to the assessment data. Whole class and small group instruction includes the following: phonemic awareness activities, synthetic phonics blending and syllabication practice, phonics workshops with formative assessments, expository comprehension worksheets, 102 spelling pattern assessments, reading strategies worksheets, 123 multi-level fluency passage videos recorded at three different reading speeds, writing skills worksheets, 644 reading, spelling, and vocabulary game cards (includes print-ready and digital display versions) to play entertaining learning games.

In addition to these resources, the program features the popular Sam and Friends Guided Reading Phonics Books. These 54 decodable books (includes print-ready and digital display versions) have been designed for older readers with teenage cartoon characters and plots. Each 8-page book introduces two sight words and reinforces the sound-spellings practiced in that day’s sound-by-sound spelling blending. Plus, each book has two great guided reading activities: a 30-second word fluency to review previously learned sight words and sound-spelling patterns and 5 higher-level comprehension questions. Additionally, each book includes an easy-to-use running record if you choose to assess. Your students will love these fun, heart-warming, and comical stories about the adventures of Sam and his friends: Tom, Kit, and Deb. Oh, and also that crazy dog, Pug. These take-home books are great for independent homework practice.

Teaching Reading Strategies and Sam and Friends Guided Reading Phonics Books BUNDLE

Teaching Reading Strategies and Sam and Friends Guided Reading Phonics Books

FREE DOWNLOADS TO ASSESS THE QUALITY OF PENNINGTON PUBLISHING RESOURCES: The SCRIP (Summarize, Connect, Re-think, Interpret, and Predict) Comprehension Strategies includes class posters, five lessons to introduce the strategies, and the SCRIP Comprehension Bookmarks.

 

 

 

Get the SCRIP Comprehension Strategies FREE Resource:

Get the Diagnostic ELA and Reading Assessments FREE Resource:

Reading, Spelling/Vocabulary, Study Skills , , , , , , , , , , ,