Learner Outcomes for English Language Learner Low Readers in an Early Intervention

First, highlyeffective teachers incorporate differentiated instruction, which provideslearners with the opportunity to use reading, writing, and oral language in multiple ways across mul

Learner Outcomes for English Language Learner Low Readers in an

Early Intervention

PATRICIA R KELLY

San Diego State University

San Diego, California, United States

FRANCISCO-XAVIER GÓMEZ-BELLENGÉ

The Ohio State University

Columbus, Ohio, United States

JING CHEN

Education Statistics Services Institute

Washington, District of Columbia, United States

The academic achievement of ELLs has become an important issuefor many school districts as larger numbers of children come toschool speaking a language other than English By 2000, 18% or nearly

47 million people over the age of five in the United States spoke alanguage other than English (Meyer, Madden, & McGrath, 2004) This

increase in ELLs has affected schools significantly The number of guage minority students in U.S public schools is projected to be 40% ofthe school-age population by the 2030s, and according to Thomas andCollier (2002), most schools are undereducating this group of students.Reading achievement for many ELLs has been below their NES peers(Donahue, Daane, & Jin, 2005) Grade 4 data from the 2003 NationalAssessment of Educational Progress indicated that only 44% of Latino

lan-fourth graders, many of whom are ELLs, scored at or above the basic

level, far below the 75% level of Anglo students By Grade 4 it is often toolate to catch students up to their peers (Juel, 1988) However, earlyinterventions are showing promising results for ELLs (Slavin, 2005) Oneearly intervention, RR, has demonstrated very good success rates withELLs (Ashdown & Simic, 2000; Escamilla, 1994; Neal & Kelly, 1999) Thepurpose of this article is to further examine the efficacy of RR with ELLs

in U.S schools The study examined the literacy programmatic outcomes

of Grade 1 ELLs compared with their NES Grade 1 peers, who also wereenrolled in RR during the 2002–2003 school year

RR is a large-scale intervention, serving about 140,000 students innearly 10,000 U.S elementary schools Students served by RR are likely

to come from marginalized groups; they are more likely to belong to aminority group, to receive free or reduced-price school lunches, and areall, by definition, among the lowest readers in their classes (Gómez-Bellengé & Rodgers, 2004) Additionally, the students who are the focus

of this study are also ELLs The purpose of this article is studying thetriple jeopardy of sociodemographic risk, low reader, and ELL status andthe outcomes for such students Their outcomes are important for us aspractitioners because some school administrators withhold the benefits

of this intervention from ELLs believing that outcomes for ELLs will beless favorable or the intervention will take too long

BACKGROUND

Much of the debate regarding the ideal education for ELLs has centered

on beginning reading instruction Some educators and researchers gue that ELLs should learn to read in their native language first, andthen in English (Ramírez, Yuen, Ramey, & Pasta, 1991; Slavin & Cheung,2003; Thomas & Collier, 1997, 2002) Other researchers indicate thatresearch does not support bilingual education (Baker, 1987; Baker & deKanter, 1983) Before 1990, very few large-scale federally funded studiesexamined bilingual instruction and how long an ELL should spend inbilingual education to become proficient enough in English to competeacademically with NESs Ramírez and his colleagues addressed this gap

ar-in the ELL research (Ramírez et al., 1991) when they compared guage-minority students’ performance in three program treatments inwhich the major difference was the amount of instruction conducted instudents’ first language, Spanish The children who received substantialamounts of instruction in Spanish and were gradually exposed to Englishfor instruction “realized the greatest growth in skills” (p 639) Ramírez

lan-et al concluded that instructional time spent in bilingual education maycontribute to later achievement in English

Thomas and Collier (1997) reached similar conclusions indicatingthat the use of learners’ native language in classroom settings for anextensive time period resulted in more overall English academic achieve-ment Slavin and Cheung (2003) reviewed experimental studies of read-ing programs for ELLs using a best-evidence synthesis, focusing on “com-parisons of bilingual and English-only programs and on specific, repli-cable models that have been evaluated with English language learners”(p v) They concluded that existing evidence favors bilingual ap-proaches, particularly paired bilingual strategies Additional support forwell-designed bilingual programs comes from Cummins (1981), Krashenand Biber (1988), Lambert and Taylor (1987), and Moll and Diaz(1985), based on the view that a child’s first language can become thefoundation for the acquisition of a second language (English)

However, bilingual instruction is not available to many ELL students;therefore, teaching must have certain characteristics to qualify as highlyeffective, expert teaching for this population of students First, highlyeffective teachers incorporate differentiated instruction, which provideslearners with the opportunity to use reading, writing, and oral language

in multiple ways across multiple academic content areas, and teachersscaffold the students’ literacy attempts to encourage student risk-takingand discovery (Boyle & Peregoy, 1998) Second, these teachers believethat all children have funds of knowledge to share (Moll & Greenburg,1990) and that incorporating students’ experiences within lessons in-creases students’ motivation to learn Third, effective teachers have au-tonomy in their teaching Moll (1988) explains, “This autonomy was notonly reflected in their teaching but in the children’s learning” (p 471).Fourth, ELLs benefit from instruction which uses systematic phonics,one-to-one tutoring, small-group instruction, programs that emphasizeextensive reading, and cooperative learning programs (Slavin & Cheung,2003)

In conclusion, instruction that is highly effective for ELLs when gual education is not available is produced by teachers who have a strongtheoretical understanding of effective teaching and learning and whooffer flexible instruction that is tailored to meet diverse learners’ indi-vidual academic needs

bilin-The Role of Intervention

Some researchers have concluded that early intervention, itself, may

be more important than the language of instruction for ELLs (August &Hakuta, 1997; Brisbois, 1995; Johnson, 1992) It has been argued thatearly literacy instruction for ELLs coincides with effective instruction forEnglish-dominant students (Slavin & Cheung, 2004) because the instruc-tion considers the learners’ diverse needs

Several early interventions have been associated with positive comes for ELLs (Gersten, 1985; Hurley, Chamberlain, Slavin, & Madden,2001) RR has been found to be a highly successful early intervention forstruggling Grade 1 students (Pinnell, Lyons, DeFord, Bryk, & Seltzer,1994) and in local studies with ELLs (Ashdown & Simic, 2000; Neal &Kelly, 1999) The remainder of this article reports the results of the firstnational U.S study examining the efficacy of RR with ELLs It is also thefirst study to investigate the impact of initial oral English proficiency on

out-RR outcomes by examining oral English proficiency in more detailedcategorizations than previously investigated

Reading Recovery

RR is an early intervention designed to help the lowest-achievingGrade 1 children learn to read and write Children receive daily indi-vidual 30-minute lessons from a specially trained teacher for 12–20weeks RR results show that it is successful in closing the achievement gapfor these children (Schmitt, Askew, Fountas, Lyons, & Pinnell, 2005)

RR was developed in the 1970s in New Zealand by Marie M Clay and

a team she recruited to determine which teaching activities worked withhard-to-teach 6-year-old children Along with teaching procedures, Claydeveloped the theoretical basis for RR It is different from the single-or-major-variable theoretical approaches often driving intervention instruc-tion RR is an effective model of literacy instruction based on a complexconstructive model of literacy learning (Clay, 2001) and on several the-oretical tenets including the following: (a) The role of oral language iscentral to the task of learning to read (Clay, 1998, 2001) (b) The learner

is seen as active and constructive Clay (2001) draws on the work ofSinger (1994) to explain how the child is at first “constructing verysimple actions systems, which become more complex” (p 224) (c) Read-ing and writing are understood as complex and reciprocal processes (d)The emphasis of instruction, carried out through reading and writingconnected texts, is on creating a broad foundation of cognitive compe-tencies which leads to the beginning of a self-extending system through

which learners learn more about reading every time they read, dent of instruction (Clay, 2001) (e) Quality interactions between anexpert teacher and young reader are critical; teachers adjust their scaf-folding to accommodate the growing competencies of the child (Clay &Cazden, 1990) (f) Acceleration of learning for a short period of time tocatch the student up to grade-level performance is the expected out-come of RR tutoring (Clay, 2005; Klenk & Kibby, 2000).

indepen-Best Practices in Early Intervention

RR fits with the best practices in early intervention Children who donot learn to read in the first few grades of school fall further and furtherbehind, and catching them up becomes more difficult with each succes-sive year of failure ( Juel, 1988) Early intervention has attracted increas-ing attention as an important prevention before the downward spiral ofreading failure becomes established (Slavin, Karweit, & Wasik, 1992)

RR has received considerable scrutiny by researchers outside of RRwith favorable conclusions about its efficacy as an early intervention.Wasik and Slavin (1993) showed that RR raised the reading achievement

of the lowest performing students up to average achievement levels, andthe effects of RR were impressive at the end of the intervention and heldfor 2 years

Hiebert (1994) reported that a high percentage of RR children wereable to read at least Grade 1 text at the end of Grade 1, and the results

of RR had considerable fidelity Shanahan and Barr (1995) came tosimilar conclusions about RR’s ability to bring lowest-achieving children

to performance levels of their average-achieving peers and its ity across sites They also acknowledged that RR had become a vital force

replicabil-in replicabil-influencreplicabil-ing the way we view early literacy development

Researchers affiliated with RR also have reported the effectiveness ofthis early intervention (Pinnell, 1989; Pinnell, Lyons, DeFord, Bryk, &Seltzer, 1994) In a recent randomized trial involving 148 Grade 1 stu-dents, Schwartz (2005) found that students who received RR services

“showed significantly higher performance compared with the randomsample control group” (p 257) A recent meta-analysis of RR found that

“the bulk of available evidence indicated that RR has had positive effects

on participating students across outcomes designed for the program andexternal to it” (D’Agostino & Murphy, 2004, p 35–36) Klenk and Kibby(2000) examined the field of remedial reading They noted the influ-ence of Clay and RR, which operationalized the ideas that reading de-velopment in young children can be accelerated to prevent readingfailure and that success should be defined as reading at grade level

Pressley (2002) addressed the positive effects of RR in the affective main: “Experiencing success in Reading Recovery lessons seems to be asource of joy for students” (p 212) This may be an especially importantaspect of RR instruction for ELLs in light of the evidence offered byKrashen (1981) that affective variables are related to success in learning

do-a second ldo-angudo-age

RR in ELL Research

The effectiveness of RR with ELLs has been reported in England, NewZealand, and the United States Hobsbaum (1995) in England and Smith(1994) in New Zealand reported similar rates of success for ELLs as forNES children

In the United States, studies in New York (Ashdown & Simic, 2000)and California (Kelly, Gomez-Valdez, Klein, & Neal, 1995; Neal & Kelly,1999) described similar results They concluded that ELLs reach similaron-grade reading levels as their NES peers Neal and Kelly (1999) notedalso that ELLs needed only a few more days of instruction than their NESpeers to reach these reading levels Their study did not address theimpact of initial oral English proficiency on learning outcomes for ELLs.Two of the previously mentioned studies did address English languageproficiency In the Hobsbaum (1995) study, teachers rated students’

fluency according to a four-stage rubric, from Stage 1, Beginner, to Stage

4, Fluent She reported that the discontinuing or successful completion

rate for ELLs increased as their level of English fluency increased That

is, the higher the learner’s level of fluency, the greater his or her thesuccess rate However, Hobsbaum (1995) cautioned that the teachers inher study may not have been using fluency ratings accurately Ashdownand Simic (2000) also addressed fluency levels They found that all ELLshad high success rates and, it is interesting to note, fluent ELLs weremore likely to be successful in RR than native speakers However, their

ELLs were classified into only two broad categories: limited English

profi-cient (LEP) or fluent A further limitation of the previous research is that

none of the studies were national in scope Though some were relativelylarge scale, they used convenience samples

Instructional Elements in RR That Support ELLs

Several elements of the design of RR reflect the previously discussedcharacteristics described as supportive of instruction for ELLs (Boyle &Peregoy, 1998; Moll, 1988; Moll & Greenburg, 1990; Slavin & Cheung,2003) RR teachers build close relationships with students, and they

provide a safe learning environment (McDowall, Boyd, Hodgen, & vanVliet, 2005) Teachers have autonomy in designing instruction for indi-vidual children They recognize each learner’s individual strengths andneeds and provide personalized and individualized instruction based onthese strengths (Clay, 1993, 1998) RR lessons include reading, writing,and oral language in multiple ways It is a powerful resource for ELLs’language development According to Clay (2001), oral language sup-ports reading and writing acquisition and is both a resource to and abeneficiary of learning to read and write RR teachers support ELLs’ orallanguage development in their daily interactions throughout the lesson;they model English in their conversations, select texts that extend thelearner’s control of English structures, and scaffold the learner’s con-struction of new language during writing and reading.

The lesson design provides both structure and flexibility At the ginning of each RR lesson, students benefit from the repeated practice

be-of reading familiar books that provide the learner with an opportunity tostrengthen the decision-making processes (Clay, 2005) Later in the les-son, when the child is engaged in writing, they have many opportunities

to incorporate their own experiences as they construct their own sages Over time, more and more complex language structures are writ-ten with the support of the expert teacher The last part of the lessonincludes the introduction (by the teacher) and the reading (by the stu-dent) of a text specially chosen for the student Here the teacher has theopportunity to negotiate the meanings and vocabulary in the text based

mes-on the student’s experiences, and to orient the child to new languagestructures he or she will encounter in the book This scaffolding allowsELLs to understand concepts and ideas in the text and to hear and usenew language prior to reading the new book Each RR learner has theopportunity to further develop their independent problem-solving skillsusing new and interesting texts; the teacher encourages independentproblem-solving while providing appropriate instruction and scaffoldingfor new learning (Clay, 2005)

Other elements of RR also contribute to its success RR teachers areexpertly trained through extensive theoretical and clinical coursework,and they receive ongoing professional development following their ini-tial year of training Teachers learn how to inform their instruction bycollecting and analyzing data on students’ reading and writing each day.Additionally, RR employs vigilant national data evaluation using a stan-dardized methodology

The study we describe in this article addresses some of the limitations

of previous research and extends findings of earlier research about RRand ELLs The data presented in the current study are unique in thatthey cover the entirety of available U.S data on ELL students served bythis very large-scale early intervention We wanted to see if our results

based on national data of ELLs in RR during one academic year wouldyield results similar to those found in earlier smaller studies (Ashdown &Simic, 2000; Neal & Kelly, 1999) in which ELLs in RR performed as well

as NESs on end-of-year tests We also wanted to extend our ings about the relationship between initial oral English proficiency andELLs’ success on text reading and phonemic awareness tasks, extendingearlier research by Hobsbaum (1995)

understand-Research Questions

The following research questions guided this study:

1a Is the rate of students who successfully discontinue their series oflessons comparable between ELLs and NESs?

1b Do both groups have similar outcomes on the text-reading and nemic-awareness tasks?

pho-2a Are the lengths of the interventions comparable between ELLs andNESs?

2b Do the lengths of the interventions vary for ELLs by their fall oralEnglish proficiency level?

3 For ELLs, is the fall oral English proficiency level related to springoutcomes on the text reading and phonemic awareness tasks?

4 For ELLs, is the fall oral English proficiency level associated with

end-of-program status outcomes? (End-of-program status outcomes are

whether the learner successfully completed the intervention and was

at average reading and writing levels.)

METHOD

In this study, we examined the literacy outcomes of ELLs served by RRand compared the outcomes with those of NES Grade 1 students whowere also served by RR RR was implemented in 52 states and federalsubdivisions during the 2002–2003 school year, with 17,000 teachersserving about 140,000 learners in more than 3,000 school districts andnearly 10,000 schools This study used data collected as part of the na-tional program evaluation of RR Data are collected and reported forevery learner in the program (Gómez-Bellengé & Rodgers, 2004)

Procedure

As a condition of being granted a royalty-free trademark from RR,participating schools send program evaluation data to the National Data

Evaluation Center (NDEC) at the Ohio State University Although dataare collected for all students who participate in RR, not all data fields arecollected for all students A core data collection of required fields iscollected for all students These data are complemented by an optionalcollection of data from participating school districts.

The data used for this study were therefore collected as part of anongoing annual program evaluation Because some data fields are op-tional, they are not available for all students For example, the measure

of oral English proficiency was available for only 55% of the nonnativespeakers of English (NNESs) This means 55% of the schools implement-ing RR voluntarily participated in a supplemental data collection, whichincluded measuring oral English proficiency Data on oral English pro-ficiency were available for three-fourths of the schools serving ELLs.The NDEC collects a variety of information on each RR student Inthis study, the following data were used: background, intervention statusoutcomes, and the length of the intervention Oral English proficiencyand literacy measures data were collected in fall (preintervention) as well

as in spring (postintervention)

Participants

RR is aimed at bringing struggling readers to average reading levels

As a result, all of the children in this study were identified as beingamong the lowest 20% of readers in their schools in the fall of Grade 1.Participants in this study include 8,581 ELLs and 121,961 NESs who wereserved by the RR program in the United States during the 2002–2003school year

ELLs are defined as having a native language other than English and

as having an oral English proficiency level less than articulate and proficient

based on school tests; we excluded children whose native language is notEnglish but who were found to be proficient speakers of English on testsadministered in their schools RR is an intervention that serves signifi-cant numbers of ELLs In 2002–2003, 13% of all the children served by

RR were ELLs These ELLs were served by 3,837 different teachers in2,537 schools and 907 school districts The proportion of ELLs served by

RR varied from 0.15% to 50.56% of the children served locally

ELLs and NESs served by RR during the 2002–2003 school year werequite different demographically (see Table 1) Most (74%) ELLs wereSpanish speakers As compared with NESs, ELLs were less likely to beboys (56.2% vs 58.8%), much more likely to have free or reduced-pricelunch (60.9% vs 37.4%), much less likely to be white, non-Hispanic(9.3% vs 66.2%), and more likely to be Hispanic (74.0% vs 7.9%) orAsian (12.1% vs 0.6%)

Oral English Proficiency Assessment

Oral English proficiency was measured in fall of the Grade 1 Because

it is an optional data field, data were available for 55% of the ELLsserved School districts participating in RR use several different assess-ments, including bilingual syntax measure, language assessment scales,idea proficiency test, student oral language observation matrix, languageassessment battery, and basic inventory of natural language Most of theassessments follow a six-level scale rubric If none of these assessmentswere used, teachers were instructed to use a six-level scale rubric (seeTable 2) developed by Rodriguez (2003) for this purpose Although weunderstand that the rubrics are not strictly comparable to each other, we

TABLE 1 Characteristics of Children Served by the Reading Recovery Program, 2002–2003

0 Child unable to respond.

1 Isolated words and expressions.

2 Isolated phrases and fragmented or very simple sentences.

3 Complete sentences, often with systematic errors in syntax (i.e., errors in the use of

articles, verb endings, and pronouns).

4 Coherent sentences with native-like fluency (occasional errors in either syntax or vocabulary; controls syntactic structures that include plurals, articles, pronouns, and some verb endings).

5 Complete sentences that are coherent and syntactically correct (articulate and

proficient).

Note Students at Proficiency Level 5 were excluded from the study.

are treating them as having equivalent psychometric properties (e.g.,validity and reliability) This is a limitation of the study.

Early English Literacy Measures

Students served by RR are administered the six tasks from An

Obser-vation Survey of Early Literacy Achievement (Clay, 2002), an assessment

ad-ministered by a trained teacher one-on-one This assessment is designed

to identify struggling readers and evaluate their progress in the period oftime when English literacy skills are emergent Two of these six measures

were used in this study: text reading level (TRL) and hearing and recording

sounds in words (HRSW) Together these measures provide an indication

of a student’s ability to do actual reading and writing The first is ameasure of oral reading, and the second assesses phonemic awareness.Both are considered among the essential elements of early reading de-scribed by the National Institute of Child Health and Human Develop-ment (2000)

While all six tasks of Clay’s (2002) survey are used for identifying thelowest readers in the fall, the two tasks we chose, TRL and HRSW, arewell suited for assessing student outcomes postintervention in the spring

of Grade 1 because TRL is the broadest measure of reading ability andHRSW provides a measure of phonemic awareness, which is important inreading and oral English proficiency Measure descriptions and reliabil-ity estimates are listed in Table 3

Data Analysis

Variables

The six variables involved in data analyses of this study were

1 ELL status (ELL vs NES): This variable is determined by examining

a each child’s school record to find out if the child is an NES ornot, and

b if a child is not an NES, this child’s fall oral English proficiencylevel NNESs who have fall oral English proficiency levels from0–4 are defined as ELLs in this study

2 Fall oral English proficiency level: This variable is reported on a six-point

scale (for most of the assessments), with 0 as the lowest level and 5

as the highest Students at the highest level are excluded from the

analysis They are articulate and proficient in English and cannot be

considered ELLs

3 Spring text reading level: The participant’s text reading level in spring

2003 was measured and reported, with a permissible score range of0–30

4 Spring hearing and recording sounds in words: The participant’s

phone-mic awareness level in spring 2003 was measured and reported, with

a permissible score range of 0–37

5 End of program status (discontinued vs others): RR children who achieve

average reading levels as determined by rigorous exit criteria are

designated as discontinued The intervention is discontinued when

children have reached a reading level comparable to that of theirclassroom peers and have developed “a system of strategies whichwork in such a way that the child learns from his own attempts toread” (Clay, 1993, p 58) These children represent 59% of all chil-dren served and 77% of children completing the full series of lessons(Gómez-Bellengé & Rodgers, 2004)

6 Length of the intervention (number of weeks): The length of RR

interven-tions is the final variable used in this study The annual evaluationmethodology measures this variable in two ways: calendar weeks ofinstruction and total number of instructional sessions As one wouldimagine, the two measures are highly correlated (Pearson r = 0.873,

p < 0.01) The measure of weeks was used in this study because it

TABLE 3 Literacy Measures on Hearing and Recording Sounds in Words (HRSW) and Text Level

Reading (TRL)

Purpose To assess phonemic awareness by

determining how well the child

represents the sounds of letters

and clusters of letters in graphic

form.

To determine an appropriate level of text difficulty and to record, using a running record, what the child does when reading continuous text.

Task To write a dictated sentence, with

credit for every sound correctly

represented.

To read texts representing a gradient of difficulty until the highest text level with 90% accuracy or better is determined with teacher recording text reading behaviors during the oral reading task; texts were drawn from established basal systems and have, over the years, proved

to be a stable measure of reading performance.

Range of

scores

Reliability Cronbach’s ␣ = 0.96 (Clay, 2002),

based on data collected from

104 urban children in fall 1990.

Rasch (Wright, Linacre, & Schultz, 1989) item separation reliability (equivalent to Cronbach’s ␣) = 0.9896 (Clay, 2002), based on data from 96 urban children

in fall 1990.

more directly relates to policy implications of RR instruction of ELLstudents.

Statistical Procedures

To compare the rates of students who discontinued successfully

be-tween ELLs and NESs (Research Question 1a), a z test on the difference

in discontinuing rates was conducted To examine differences in springliteracy level indicated by TRL and HRSW (a phonemic awareness as-sessment, Research Question 1b), a between-subject multivariate analysis

of variance (MANOVA) was performed Two literacy measures in spring(TRL and HRSW) served as dependent variables and ELL status was theindependent variable The MANOVA assumption of homogeneity ofvariance-covariance matrices was checked

A t test was used to compare the lengths of interventions of ELL and

NES students who discontinued successfully (Research Question 2a) and

a univariate analysis of variance (ANOVA) was carried out to find out ifthe lengths of interventions (dependent variable) varied for successfullydiscontinued ELL students based on their fall oral English proficiencylevel (Research Question 2b) We did not conduct a regression analysisbecause we do not have evidence that the fall oral English proficiencymeasure (independent variable) is an equal interval variable This reasonalso applies to Research Question 3

To answer Research Question 3 (spring TRL), we examined ance matrices

covari-A chi-square test for independence was used to determine the tionship between the end-of-program status and the fall oral Englishproficiency (Research Question 4) We used this statistical procedurebecause the end-of-program status is a nominal variable and the fall oralEnglish proficiency is an ordinal variable but treated as nominal as wedid in Research Questions 2 and 3

rela-RESULTS

Research Questions 1a and 1b

1a Is the rate of students who discontinue successfully their series oflessons comparable between ELLs and NESs? 1b Do both groups havesimilar outcomes on the text reading and phonemic awareness tasks?For research question 1, we specifically wanted to know (a) if thediscontinuing rates for ELLS and NESs served by the RR program are