274 CounCil for ExCEptional ChildrEn
Mr. Long is a special education teacher in an urban school district. Three times per year, he uses Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Next to assess his students’ oral reading fluency (ORF) skills at their chronological grade level. Mr. Long conducts weekly progress monitoring of all students who score below the expected benchmark score for words read correctly per minute (WCPM). Students are assessed at either their grade level, if they are reading at or above 50 WCPM according to the DIBELS Next progress- monitoring guidelines (Dynamic Measurement Group, 2012), or at their instructional level based on results from a survey level assessment. To conduct the assessments, Mr. Long takes students out of the classroom during various times of the day. Depending on the time of day, Mr. Long uses different setting locations, including the hallway, a conference room, and an unused classroom. Students are taken individually or in small groups, depending on how far away he must take them for the assessment.
After a few weeks, Mr. Long notices one of the students, Laine, has inconsistent scores in her data set (see Figure 1). Laine, a third-grade student with a specific learning disability, had scores of 61, 43, 75, and 57 WCPM over 4 weeks. Mr. Long compares Laine’s scores with those of other students in the group and notices the other students’ scores are more consistent. For example, Mason’s scores are 66, 71, 71, and 72 WCPM during the same time period (see Figure 2). Mr. Long consults with the school’s reading specialist and finds out that “high variability” includes a range of 10 or more words read correctly above or below the trend line. Because Mr. Long has graphed Laine’s data with a trend line characterizing the data, he can quickly determine that her data are highly variable. Mr. Long realizes that highly variable data can obscure what Laine’s true progress might be. He sees the need to collect more data to determine if the variability can be reduced before a good decision about changing her intervention can be made.
CBM is useful and effective for monitoring student progress in important skills, such as reading, mathematics, and writing. Research has shown that (a) CBM can be easily implemented and interpreted by teachers (e.g., Fuchs, Deno, & Mirkin, 1984), (b) student outcomes have improved when teachers use CBM to inform instructional decision making (e.g., Fuchs, Fuchs, Hamlett, & Stecker, 1991), (c) reliable and valid measures have been developed that predict important student outcomes (e.g., Fuchs, Fuchs, & Maxwell, 1988; Kim, Petscher, Schatschneider, & Foorman, 2010; Wayman, Wallace, Wiley, Tichá, & Espin, 2007), and (d) CBM can be an integral component of multi-tiered systems for identifying and monitoring students’ academic needs (e.g., Kovaleski, VanDerHeyden, & Shapiro, 2013; M. R. Shinn, 2007). CBM for reading (CBM-R) is an efficient and effective research-based progress- monitoring tool to monitor student growth in reading and to evaluate the effectiveness of targeted instruction (Good et al., 2011; Hosp, Hosp, & Howell, 2016). CBM-R is easy to administer and requires minimal resources, such as time and materials. Furthermore, the feedback teachers receive from administering CBM-R can inform instructional decision making and provide critical data about
individual student progress toward reading goals. Given the utility of CBM-R, it is widely used as a key data source for instructional and eligibility decision making (Ardoin, Christ, Morena, Cormier, & Klingbeil, 2013).
The most commonly used CBM-R is ORF (CBM ORF). CBM ORF is a research-based, standardized assessment of connected text that is administered to individual students. CBM ORF is a good indicator of a student’s current skill level and predictor of future reading performance (Deno, Fuchs, & Marston, 2001; Fuchs, Fuchs, Hosp, & Jenkins, 2001; Kim et al., 2010). CBM ORF requires the student to use a variety of different literacy skills, such as decoding, vocabulary, and comprehension (Hosp et al., 2016). CBM ORF originated in the 1970s, when practitioners randomly selected passages from the curriculum materials used in the classroom (e.g., Deno, 1985; Deno, Marston, Shinn, & Tindal, 1983). This practice increased the utility and validity of the measure for making instructional decisions; however, researchers found that student performance on passages within a grade level varied substantially, decreasing the reliability of these measures (see Hintze & Christ, 2004). Later iterations of CBM ORF included development of passages equated based on readability formulae (e.g., Aimsweb;
Figure 1. Laine, third-grade student curriculum-based measurement oral reading fluency, high variability to moderate variability
TEACHING ExCEptional ChildrEn | May/JunE 2018 275
M. M. Shinn & Shinn, 2002; DIBELS, 6th ed.; Good & Kaminski, 2002). Unfortunately, student performance on these passages continued to be excessively variable within grade levels (e.g., Poncy, Skinner, & Axtell, 2005). Excessive variability makes the data difficult to interpret, and therefore, recommendations for instructional modifications become unclear.
Currently, CBM ORF passages have been written using readability formulae for initial equating, then field-tested with students to choose the most equivalent passages to include in published sets (e.g., DIBELS Next; Good et al., 2011; easyCBM; Alonzo, Tindal, Ulmer, & Glasgow, 2006; FastBridge; Christ & Colleagues, 2015). Although some researchers have found persistent variability among these more modern passages (Cummings, Park, & Schaper, 2013), those studies were conducted with higher-performing students than those who are typically included in progress monitoring (e.g., students scoring at or above benchmark at screening). Other researchers found that when passages are implemented as intended, such as to progress monitor students below or well below benchmark, acceptably low levels of variability are seen (O’Keeffe, Bundock, Kladis, Yan, & Nelson, 2017;
Tindal, Nese, Stevens, & Alonso, 2016). Given the challenges presented by excessive variability, educators should be aware of possible sources of
variability and have strategies to prevent and address variability in CBM ORF progress monitoring. These strategies should be followed in addition to the recommendations from the specific publisher of the CBM ORF in use and from general recommendations for implementing and interpreting CBM (e.g., Hosp et al., 2016).
Indicators of Excessive Variability in CBM ORF Progress Monitoring
Educators need to determine how much variability is too much when evaluating student progress-monitoring data. Typically, educators evaluate progress-monitoring data using time
series graphs, with words read correctly on each measurement occasion graphed over time. When educators use visual analysis to determine if a student is making adequate progress or not, multiple graphical components can affect this decision. For example, the amount of variability and the degree of slope in the data can make evaluation decisions more or less accurate, with higher variability and lower slope making decisions substantially less accurate (Nelson, Van Norman, & Christ, 2017; Ottenbacher, 1990; Van Norman & Christ, 2016). If inaccurate decisions are made based on variable data, students who need a change in intervention may not receive it, whereas students who do not need a change may experience an unneeded change in intervention. For CBM ORF, researchers have suggested that very low variability exists when most (i.e., 2/3) of the data points fall within five correctly read words per minute (five above and five below) of a trend line,
and acceptable variability exists when most of the data points fall within 10 correctly read words per minute (10 above and 10 below) of a trend line (Christ, Zopluoglu, Monaghen, & Van Norman, 2013). These values are based on ranges across grade levels (e.g., Christ & Silberglitt, 2007); therefore, students who read more slowly would have lower limits of variability that are acceptable. If available through an electronic database (e.g., AimswebPlus; Pearson, 2017), researchers recommend making these determinations based on confidence intervals, which are generated statistically with the student data (Christ & Silberglitt, 2007). Values that fall outside these ranges can be considered extreme values, which can
Figure 2. Mason, third-grade student curriculum-based measurement oral reading fluency, very low variability
![C:\Users\mbrown3\AppData\Local\Microsoft\Windows\Temporary Internet Files\Content.IE5\3QIURI7L\MC900441498[1].png](https://www.sweetstudy.com/files/edd-capstone-rubrics-and-checklists-by-stage-docx)
EdD Capstone Rubrics and Checklists by Stage
