Appendix A – Annotated Outline

Assisting Students Struggling with Reading:

Response to Intervention (RtI) and Multi-Tier

Intervention in the Primary Grades

Assisting Students Struggling with Reading:

Response to Intervention (RtI) and Multi-Tier

Intervention in the Primary Grades

IES PRACTICE GUIDE

NCEE 2009-4045 U.S. DEPARTMENT OF EDUCATION

WHAT WORKS CLEARINGHOUSE

The Institute of Education Sciences (IES) publishes practice guides in education to bring the best available evidence and expertise to bear on the types of systemic challenges that cannot currently be addressed by single interventions or programs. Authors of practice guides seldom conduct the types of systematic literature searches that are the backbone of a meta-analysis, although they take advantage of such work when it is already published. Instead, authors use their expertise to identify the most important research with respect to their recommendations, augmented by a search of recent publications to ensure that research citations are up-to-date.

Unique to IES-sponsored practice guides is that they are subjected to rigorous exter- nal peer review through the same office that is responsible for independent review of other IES publications. A critical task for peer reviewers of a practice guide is to determine whether the evidence cited in support of particular recommendations is up-to-date and that studies of similar or better quality that point in a different direction have not been ignored. Because practice guides depend on the expertise of their authors and their group decision-making, the content of a practice guide is not and should not be viewed as a set of recommendations that in every case de- pends on and flows inevitably from scientific research.

The goal of this practice guide is to formulate specific and coherent evidence-based recommendations for use by educators addressing the challenge of reducing the number of children who fail to learn how to read proficiently by using “response to intervention” as a means of both preventing reading difficulty and identifying stu- dents who need more help. This is called Response to Intervention (RtI). The guide provides practical, clear information on critical RtI topics and is based on the best available evidence as judged by the panel. Recommendations in this guide should not be construed to imply that no further research is warranted on the effective- ness of particular RtI strategies.

IES PRACTICE GUIDE

Assisting Students Struggling with Reading: Response to Intervention and Multi-Tier Intervention in the Primary Grades

February 2009

Panel Russell Gersten (Chair) InstructIonal research Group

Donald Compton VanderbIlt unIVersIty

Carol M. Connor FlorIda state unIVersIty

Joseph Dimino InstructIonal research Group

Lana Santoro InstructIonal research Group

Sylvia Linan-Thompson unIVersIty oF texas—austIn

W. David Tilly heartland area educatIon aGency

Staff Rebecca Newman-Gonchar InstructIonal research Group

Kristin Hallgren MatheMatIca polIcy research

NCEE 2009-4045 U.S. DEPARTMENT OF EDUCATION

This report was prepared for the National Center for Education Evaluation and Re gional Assistance, Institute of Education Sciences under Contract ED-07-CO-0062 by the What Works Clearinghouse, which is operated by Mathematica Policy Research, Inc.

Disclaimer

The opinions and positions expressed in this practice guide are the authors’ and do not necessarily represent the opinions and positions of the Institute of Education Sci- ences or the U.S. Department of Education. This practice guide should be reviewed and applied according to the specific needs of the educators and education agency using it, and with full realization that it represents the judgments of the review panel regarding what constitutes sensible practice, based on the research that was available at the time of publication. This practice guide should be used as a tool to assist in decision-making rather than as a “cookbook.” Any references within the document to specific educa tion products are illustrative and do not imply endorse- ment of these products to the exclusion of other products that are not referenced.

U.S. Department of Education Arne Duncan Secretary

Institute of Education Sciences Sue Betka Acting Director

National Center for Education Evaluation and Regional Assistance Phoebe Cottingham Commissioner

February 2009

This report is in the public domain. While permission to reprint this publication is not necessary, the citation should be:

Gersten, R., Compton, D., Connor, C.M., Dimino, J., Santoro, L., Linan-Thompson, S., and Tilly, W.D. (2008). Assisting students struggling with reading: Response to Intervention and multi-tier intervention for reading in the primary grades. A practice guide. (NCEE 2009-4045). Washington, DC: National Center for Edu- cation Evaluation and Regional Assistance, Institute of Education Sci ences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/ publications/practiceguides/.

This report is available on the IES website at http://ies.ed.gov/ncee and http://ies. ed.gov/ncee/wwc/publications/practiceguides/.

Alternative formats On request, this publication can be made available in alternative formats, such as Braille, large print, audiotape, or computer diskette. For more information, call the alternative format center at (202) 205-8113.

( iii )

Assisting Students Struggling with Reading: Response to Intervention and Multi-Tier Intervention in the Primary Grades

Contents

Introduction 1

The What Works Clearinghouse standards and their relevance to this guide 2

Overview 4

Scope of the guide 8

Checklist for carrying out the recommendations 9

Recommendation 1. Screen all students for potential reading problems at the beginning of the year and again in the middle of the year. Regularly monitor the progress of students who are at elevated risk for developing reading disabilities. 11

Recommendation 2. Provide differentiated reading instruction for all students based on assessments of students’ current reading levels (tier 1). 17

Recommendation 3. Provide intensive, systematic instruction on up to three foundational reading skills in small groups to students who score below the benchmark on universal screening. Typically these groups meet between three and five times a week for 20–40 minutes (tier 2). 19

Recommendation 4. Monitor the progress of tier 2 students at least once a month. Use these data to determine whether students still require intervention. For those still making insufficient progress, school-wide teams should design a tier 3 intervention plan. 24

Recommendation 5. Provide intensive instruction daily that promotes the development of various components of reading proficiency to students who show minimal progress after reasonable time in tier 2 small group instruction (tier 3). 26

Appendix A. Postscript from the Institute of Education Sciences 32

Appendix B. About the authors 35

Appendix C. Disclosure of potential conflicts of interest 38

Appendix D. Technical information on the studies 39

References 50

( iv )

ASSISTING STUDENTS STRUGGLING WITH READING: RESPONSE TO INTERVENTION AND MULTI-TIER INTERVENTION IN THE PRIMARY GRADES

List of tables

Table 1. Institute of Education Sciences levels of evidence for practice guides 3

Table 2. Recommendations and corresponding levels of evidence 6

Table 3. Recommended target areas for early screening and progress monitoring 13

Table 4. Foundational reading skills in grades K–2 21

Table 5. Progress monitoring measures in grades K–2 25

Table D1. Studies of tier 2 interventions in grades K–2 reading that met What Works Clearinghouse standards 41

( 1 )

Introduction

In the primary grades students with read- ing difficulties may need intervention to prevent future reading failure. This guide offers specific recommendations to help educators identify students in need of in- tervention and implement evidence-based interventions to promote their reading achievement. It also describes how to carry out each recommendation, including how to address potential roadblocks in imple- menting them.

We, the authors, are a small group with ex- pertise in various dimensions of this topic. Several of us are also experts in research methodology. The recommendations in this guide reflect not only our expertise and experience but the findings of rigor- ous studies of interventions to promote reading achievement.

Each recommendation received a rating that describes the strength of the research evidence that has shown its effectiveness. These ratings—“strong,” “moderate,” or “low”—are defined as:

Strong refers to consistent and generaliz- able evidence that a program causes bet- ter outcomes.1

1. Following WWC guidelines, we consider a posi- tive, statistically significant effect, or an effect size greater than 0.25, as an indicator of posi- tive effects.

Moderate refers to evidence from studies that allow strong causal conclusions but cannot be generalized with assurance to the population on which a recommenda- tion is focused (perhaps because the find- ings have not been widely replicated) or to evidence from studies that are generaliz- able but have more causal ambiguity than offered by experimental designs (such as statistical models of correlational data or group comparison designs for which equivalence of the groups at pretest is uncertain).

Low refers to expert opinion based on rea- sonable extrapolations from research and theory on other topics and evidence from studies that do not meet the standards for moderate or strong evidence.

Table 1 details the criteria used to deter- mine the level of evidence for each rec- ommendation. For questions about what works best, high-quality experimental and quasi-experimental studies, such as those meeting the criteria of the What Works Clearinghouse (www.whatworks.ed.gov), have a privileged position. The evidence considered in developing and rating these recommendations included experimental research on providing differentiated in- struction in a general education classroom and rigorous evaluations of intensive read- ing interventions. We also examined stud- ies on the technical adequacy of batteries of screening measures.

INTRODUCTION

( 2 )

The What Works Clearinghouse standards and their relevance to this guide

The panel relied on WWC Evidence Stan- dards to assess the quality of evidence supporting educational programs and practices and apply a level of evidence rating to each recommendation. The WWC addresses evidence for the causal validity of instructional programs and practices using WWC Standards. Information about these standards is available at http://ies. ed.gov/ncee/wwc/references/standards/. The technical quality of each study is rated and placed into one of three categories:

• Meets Evidence Standards for random- ized controlled trials and regression discontinuity studies that provide the strongest evidence of causal validity.

• Meets Evidence Standards with Res- ervations for all quasi-experimental studies with no design flaws and ran- domized controlled trials that have problems with randomization, attri- tion, or disruption.

• Does Not Meet Evidence Screens for studies that do not provide strong evi- dence of causal validity.

Based on the recommendations and sug- gestions for their implementation, ap- pendix D presents more information on the research evidence supporting the recommendations.

The panel would like to thank Kelly Hay- mond for her contributions to the analy- sis, Mary Jo Taylor for her expert editorial assistance, the WWC reviewers for their contribution to the project, and Jo Ellen Kerr for her support of the intricate logis- tics of the project. We also would like to thank Scott Cody for his oversight of the analyses and the overall progress of the practice guide.

Dr. Russell Gersten Dr. Donald Compton Dr. Carol M. Connor

Dr. Joseph Dimino Dr. Lana Santoro

Dr. Sylvia Linan-Thompson Dr. W. David Tilly

INTRODUCTION

( 3 )

Table 1. Institute of Education Sciences levels of evidence for practice guides

Strong

In general, characterization of the evidence for a recommendation as strong requires both studies with high internal validity (i.e., studies whose designs can support causal conclusions) and studies with high external validity (i.e., studies that in total include enough of the range of participants and settings on which the recommendation is focused to support the conclu- sion that the results can be generalized to those participants and settings). Strong evidence for this practice guide is operationalized as: • A systematic review of research that generally meets the What Works Clearinghouse (WWC)

standards (see http://ies.ed.gov/ncee/wwc/) and supports the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR

• Several well designed, randomized controlled trials or well designed quasi-experiments that generally meet WWC standards and support the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR

• One large, well designed, randomized controlled, multisite trial that meets WWC standards and supports the effectiveness of a program, practice, or approach, with no contradictory evidence of similar quality; OR

• For assessments, evidence of reliability and validity that meets the Standards for Educa- tional and Psychological Testing.a

Moderate

In general, characterization of the evidence for a recommendation as moderate requires studies with high internal validity but moderate external validity, or studies with high external valid- ity but moderate internal validity. In other words, moderate evidence is derived from studies that support strong causal conclusions, but where generalization is uncertain, or studies that support the generality of a relationship, but where the causality is uncertain. Moderate evi- dence for this practice guide is operationalized as: • Experiments or quasi-experiments generally meeting WWC standards and supporting the

effectiveness of a program, practice, or approach with small sample sizes and/or other conditions of implementation or analysis that limit generalizability and no contrary evi- dence; OR

• Comparison group studies that do not demonstrate equivalence of groups at pretest and therefore do not meet WWC standards but that (a) consistently show enhanced outcomes for participants experiencing a particular program, practice, or approach and (b) have no major flaws related to internal validity other than lack of demonstrated equivalence at pretest (e.g., only one teacher or one class per condition, unequal amounts of instructional time, highly biased outcome measures); OR

• Correlational research with strong statistical controls for selection bias and for discern- ing influence of endogenous factors and no contrary evidence; OR

• For assessments, evidence of reliability that meets the Standards for Educational and Psy- chological Testingb but with evidence of validity from samples not adequately representa- tive of the population on which the recommendation is focused.

Low

In general, characterization of the evidence for a recommendation as low means that the rec- ommendation is based on expert opinion derived from strong findings or theories in related areas or expert opinion buttressed by direct evidence that does not rise to the moderate or strong levels. Low evidence is operationalized as evidence not meeting the standards for the moderate or high levels.

a. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999).

b. Ibid.

( 4 )

Assisting Students Struggling with Reading: Response to Intervention and Multi-Tier Intervention for Reading in the Primary Grades

Overview

Response to Intervention (RtI) is a compre- hensive early detection and prevention strat- egy that identifies struggling students and assists them before they fall behind. RtI sys- tems combine universal screening and high- quality instruction for all students with in- terventions targeted at struggling students.

RtI strategies are used in both reading and math instruction. For reading instruction in the primary grades (K–2), schools screen students at least once a year to identify students at risk for future reading failure.2 Students whose screening scores indicate potential difficulties with learning to read are provided with more intensive reading interventions. Student responses to the interventions are then measured to deter- mine whether they have made adequate progress and either (1) no longer need the intervention, (2) continue to need some intervention, or (3) need even more inten- sive intervention.

In RtI, the levels of interventions are conven- tionally referred to as “tiers.” RtI is typically thought of as having three tiers, with the first tier encompassing general classroom instruction.3 Some states and school dis- tricts, however, have implemented multi-tier intervention systems with more than three tiers. Within a three-tier RtI model, each tier is defined by specific characteristics:

2. Johnson, Jenkins, Petscher, and Catts (in press, pp. 3–4).

3. Fuchs, Fuchs, and Vaughn (2008) make the case for a three-tier RtI model.

• Tier 1 instruction is generally defined as reading instruction provided to all students in a class. Beyond this gen- eral definition, there is no clear con- sensus on the meaning of the term tier 1. Instead, it is variously referred to as “evidence-based reading instruction,”4 “high quality reading instruction,”5 or “an instructional program…with bal- anced, explicit, and systematic reading instruction that fosters both code-based and text-based strategies for word iden- tification and comprehension.”6

• Tier 2 interventions are provided only to students who demonstrate prob- lems based on screening measures or weak progress from regular classroom instruction. In addition to general classroom instruction, tier 2 students receive supplemental, small group reading instruction aimed at building foundational reading skills.

• Tier 3 interventions are provided to students who do not progress after a reasonable amount of time with the tier 2 intervention and require more intensive assistance. Tier 3 (or, in dis- tricts with more than three tiers, tiers 3 and above) usually entails one-on- one tutoring with a mix of instruc- tional interventions. Ongoing analysis of student performance data is critical in tier 3. Systematically collected data are used to identify successes and failures in instruction for individual students. If students still experience difficulty after receiving intensive ser- vices, they are evaluated for possible special education services.

Though a relatively new concept, RtI and multi-tier interventions are becoming in- creasingly common. This is attributed in

4. Vaughn and Fuchs (2006).

5. Division for Learning Disabilities (2007).

6. Vellutino, Scanlon, Small, Fanuele, and Sweeney (2007).

OVERVIEW