The Role of Districts in Fostering Instructional Improvement pptx

Zimmer, Heather Barney Supported by The William and Flora Hewlett Foundation The Role of Districts in Fostering Instructional Improvement Lessons from Three Urban Districts Partnered wi

This document and trademark(s) contained herein are protected by law

as indicated in a notice appearing later in this work This electronic representation of RAND intellectual property is provided for non- commercial use only Permission is required from RAND to reproduce, or reuse in another form, any of our research documents.

Limited Electronic Distribution Rights

Visit RAND at www.rand.org

Explore RAND Education

View document details

For More Information

This PDF document was made available from www.rand.org as a public service of the RAND Corporation

6Jump down to document

THE ARTS CHILD POLICY

CIVIL JUSTICE

EDUCATION

ENERGY AND ENVIRONMENT

HEALTH AND HEALTH CARE

WORKFORCE AND WORKPLACE

The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world.

Purchase this documentBrowse Books & PublicationsMake a charitable contribution

Support RAND

RAND monographs present major research findings that address the challenges facing the public and private sectors All RAND mono-graphs undergo rigorous peer review to ensure high standards for research quality and objectivity.

Julie A Marsh, Kerri A Kerr, Gina S Ikemoto, Hilary Darilek,

Marika Suttorp, Ron W Zimmer, Heather Barney

Supported by The William and Flora Hewlett Foundation

The Role of Districts in Fostering Instructional Improvement

Lessons from Three Urban Districts

Partnered with the Institute for Learning

The RAND Corporation is a nonprofit research organization providing objective analysis and effective solutions that address the challenges facing the public and private sectors around the world RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

© Copyright 2005 RAND Corporation

All rights reserved No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from RAND.

Published 2005 by the RAND Corporation

1776 Main Street, P.O Box 2138, Santa Monica, CA 90407-2138

1200 South Hayes Street, Arlington, VA 22202-5050

201 North Craig Street, Suite 202, Pittsburgh, PA 15213-1516

RAND URL: http://www.rand.org/

To order RAND documents or to obtain additional information, contact

Distribution Services: Telephone: (310) 451-7002;

Fax: (310) 451-6915; Email: order@rand.org

Education and supported by The William and Flora Hewlett Foundation.

Library of Congress Cataloging-in-Publication Data

The role of districts in fostering instructional improvement : lessons from three urban districts partnered with the Institute for Learning / Julie A Marsh [et al.].

p cm.

“MG-361.”

Includes bibliographical references.

ISBN 0-8330-3853-2 (pbk : alk paper)

1 School improvement programs—United States—Case studies 2 Instructional systems—United States—Case studies 3 School districts—United States—Case studies 4 Educational change—United States—Case studies I Marsh, Julie A II Institute for Learning.

In fall 2002, the RAND Corporation initiated a study to analyzethree urban districts’ efforts to face these challenges and improve theinstructional quality and performance of their schools The study alsosought to assess the contribution to these efforts made by an interme-diary organization, the Institute for Learning (IFL) We closely ex-amined district reform efforts in four areas: promoting theinstructional leadership of principals; supporting the professionallearning of teachers, in particular through school-based coachingmodels; specifying curriculum; and promoting data-based decision-making for planning and instructional improvement We also exam-ined the impact of the IFL on these instructional improvementefforts.

This monograph presents findings from that three-year study Itdescribes the districts’ work in each area of reform, identifies com-mon constraints and enablers of district success, assesses the nature

and impact of district-intermediary partnerships, and makes mendations for districts undertaking similar instructional reforms.The report should interest policymakers, researchers, and practi-tioners involved in designing, implementing, assisting, or studyingschool districts’ efforts to improve the instructional quality and per-formance of all schools.

recom-This research was undertaken within RAND Education, a unit

of the RAND Corporation Funding to carry out the work was vided by The William and Flora Hewlett Foundation

Dedication

We dedicate this report to the memory of RAND’s Tom Glennan, adear colleague, friend, and mentor to all of us involved in this study.Tom initiated this research out of a profound commitment to betterunderstanding and supporting the work of urban school districts

Contents

Preface iii

Dedication v

Figures xiii

Tables xv

Summary xvii

Acknowledgments xxvii

Abbreviations xxix

CHAPTER ONE Introduction 1

Study Purpose 3

Methodology 4

Organization of the Report 5

CHAPTER TWO Research Background, Framework, and Methods 7

What We Know from Prior Research 7

School Districts and Instructional Improvement 7

Intermediary Organizations and District Reform 9

Conceptual Framework 11

Methods 17

Sample 18

Data Sources 18

Data Analysis 22

Study Limitations 22

Formative Feedback 23

CHAPTER THREE Setting the Stage: Overview of Study Districts and the IFL 25

The Three Study Districts: Characteristics and Context 25

Institute for Learning: Background and History 27

Stage One: Early History and Evolution 27

Stage Two: Shift to On-Site Support and Articulated Notions of High-Performing Districts 29

Current Status and Scope of IFL Work 30

IFL-District Partnerships in the Case Study Districts 32

Monroe 33

Roosevelt 34

Jefferson 35

Summary 36

CHAPTER FOUR District Strategies to Improve Instruction: Implementation and Outcomes 39

Principals’ Instructional Leadership 40

Consistent Emphasis on Professional Development and Supervision of Principals 41

Greater Alignment of District Actions in Monroe and Roosevelt 43

Principals Varied in Degree of Reported Instructional Leadership Actions 43

Factors Affecting District Efforts to Support Principals’ Instructional Leadership 45

IFL’s Role in Supporting Instructional Leadership Was Consistent and Strong 49

School-Based Coaches to Support the Professional Learning of Teachers 50

Districts Implemented Different Coaching Models: Curriculum-Versus School-Centered Approaches 51

Contents ix

Coaching Role Valued in General, Yet Teachers Reported Strong Preferences for Individualized Interactions 52

Factors Affecting the Implementation of School-Based Coaches 55

IFL’s Role in Influencing School-Based Coaching Models Varied 57

Curriculum Specification 58

Curriculum Guides Were a Driving Force for Improving Instruction in Two Districts 58

Curriculum Guides Useful for the System, but Challenges Existed at the Classroom Level 60

Factors Affecting Implementation and Perceived Usefulness of Curriculum Guides 62

IFL Role in Affecting District Curricular Reforms Varied 64

Data Use 66

Strong Focus on Data in Jefferson and Monroe 66

Factors Affecting Data Use 72

IFL Role in District Use of Data to Inform Instruction Was Limited 75

Summary 75

CHAPTER FIVE Overarching Findings About District Instructional Improvement: Common Constraints and Enablers 79

A Comprehensive Set of Strategies Was Important for Addressing All Facets of Instruction 80

Focus on a Limited Number of Initiatives Assisted in Implementing Reforms, but Tradeoffs Resulted 82

Insufficient Capacity Was a Significant Obstacle to Instructional Improvement 84

On-Site Assistance for Teachers and Principals Enhanced Instructional Capacity at the School Level 86

Strategies That Were Aligned and Mutually Supportive Facilitated Reform; Misalignment Greatly Constrained Efforts 87

Districts Struggled to Design Reform Strategies That Enabled Multiple Stakeholders to Engage in Instructional Improvement 89

Achieving a Balance Between Standardization and Flexibility Proved

Difficult for Districts 90

Local Accountability Policies Created Incentives and Disincentives That Affected the Quality of Implementation of Reform Strategies 92

Policy Decisions at Higher Levels Influenced Policy Decisions and Actions at the District Level, Often with Unintended Consequences 94

Summary 95

CHAPTER SIX Impact of the Institute for Learning 99

IFL Contribution to the Four Main Areas of Instructional Reform 99

IFL Made Greatest Contribution to District Instructional Leadership Strategies 100

IFL Had Less Influence on Other Areas of Reform 101

IFL Resources: The Most Influential Ideas and Tools 102

Learning Walks Supported Multiple Instructional Improvement Efforts 104

Principles of Learning Provided a Common Language 106

IFL’s Overall Impact on Districts: The Bottom Line 107

IFL Had a Strong Reported Impact on Organizational Culture 107

IFL Was Reported to Affect Administrators’ Capacity 109

Less Evidence to Suggest IFL’s Impact on Teachers 112

Factors Affecting the IFL’s Reported Impact on Districts 114

The IFL Had Limited Capacity—and Possibly Limited Intentions— to Assist Districts with the Full Range of Instructional Improvement Efforts 115

Leadership Buy-In at All Levels Enabled Partnership Efforts and Impact 116

The Perception of IFL as a Vendor Hindered Its Effect on District Reform 117

Trust in IFL Staff, Ideas, and Tools, and Their Perceived Credibility, Was Important for Building Teacher, Principal, and District Leader Support 119

Contents xi

Practical Tools Supported Application of IFL Ideas but Raised Concerns About Superficial Implementation 120

Turnover Challenged IFL Efforts to Sustain and Deepen Reform 121

Summary 122

CHAPTER SEVEN Conclusions and Lessons Learned 125

Summary of Findings 125

Lessons Learned for Policy and Practice 127

Lessons for Instructional Improvement 128

Lessons for District-Intermediary Partnerships 131

Conclusion 134

APPENDIX A Survey Instruments 137

B Technical Notes on Research Methods 163

C Student Achievement Trends 171

D Principles of Learning 181

Bibliography 183

Figures

2.1 Conceptual Framework 12 4.1 Emphasis of District Instructional Improvement Actions 40 6.1 Reported Role of the IFL in Influencing District Reform

Efforts 100 6.2 Principals’ Reports on the Impact of IFL-Related Professional Development 110

Tables

2.1 Mapping of Actions and Intermediate Outcomes 16

2.2 District Site Visit Interviews and Focus Groups (2003 and 2004) 19

2.3 Survey Response Rates, Spring 2004 21

3.1 Characteristics of Study Districts, 2003–04 26

3.2 Examples of Principles of Learning 28

4.1 Percentage of Teachers Agreeing or Strongly Agreeing About Support Provided by Their Principals 44

4.2 Percentage of Principals Reporting Time Spent on and Value of Reviewing Student Achievement Data 45

4.3 Percentage of Teachers Agreeing or Strongly Agreeing About Support Provided by Coaches 53

4.4 Percentage of Teachers Agreeing or Strongly Agreeing About Usefulness of Curriculum Guides 60

4.5 Percentage of Teachers and Principals Reporting Moderate to Major Emphasis on Interpreting and Using Student Test Results to Guide Instruction in Professional Development Activities 67

4.6 Percentage of Teachers Reporting That Various Types of Data Were Moderately or Very Useful for Guiding Instruction 68

B.1 Breakdown of Schools by Year of Data Collection 163

B.2 Outline of Strata Used to Create Survey Sampling Framework, Monroe 164

B.3 Survey Sample of Schools Versus Total Number, Monroe 165

C.1 District Changes in Percentages of Proficient and

Low-Performing Students, 1997–98 Through 2003–04 172 C.2 Difference Between State and District Averages on the

Percentage Scoring Proficient in ELA and Mathematics,

1997–98 to 2003–04 176 C.3 Difference Between State and District Averages on the

Percentage of Low-Performing Students in ELA and

Mathematics, 1997–98 to 2003–04 178

Summary

Improving school systems is critical to bridging the achievement gapbetween students of different racial and socioeconomic backgroundsand to achieving the goals of the No Child Left Behind Act (NCLB)

In fall 2002, the RAND Corporation initiated a formative assessment

of three urban districts’ efforts to improve instructional quality andschool performance The study explored ways to improve teachingand learning in urban school districts It also examined the contribu-tions of one intermediary organization, the Institute for Learning(IFL), to efforts to introduce systemic change in the three districts.The study sought to answer four broad questions:

• What strategies did districts employ to promote instructionalimprovement? How did these strategies work?

• What were the constraints and enablers of district instructionalimprovement efforts?

• What was the impact of the IFL? What were the constraints andenablers of the district-IFL partnerships?

• What are the implications for district instructional improvementand district-intermediary partnerships?

We used a comparative case study design and mixed methods to swer these questions Districts were selected for experience workingwith the IFL (more than three years) and for variation in district size,union environment, and state context We collected and analyzeddata from extensive field interviews and focus groups conducted over

an-a two-yean-ar period; from RAND-developed surveys of elementan-ary,middle, and high school principals and teachers; from district andIFL documents; and from demographic and student achievement da-tabases

Findings

Our evidence yielded the following findings

District Instructional Improvement Strategies

In the three districts, instructional reform efforts revolved aroundfour common areas of focus: building the instructional leadershipskills of principals; supporting the professional learning of teachers,with a particular focus on school-based coaching; providing greaterspecification of and support for standards-aligned curriculum; andpromoting the use of data to guide instructional decisions While alldistricts pursued strategies within each area, each tended to focus ontwo key areas to change the system In addition, districts had varyingdegrees of success in attaining the intermediate reform goals (i.e.,outcomes expected to ultimately contribute to improved teaching andlearning) Our findings in the four areas of reform are as follows

Instructional Leadership All districts attempted to increaseprincipals’ instructional leadership capacity, giving principals profes-sional development and expecting principals’ supervisors (who typi-cally had titles such as area or assistant superintendent) to focusschool visits and meetings with principals on matters related to im-proving instruction

Summary xix

Despite a relatively consistent focus on instructional leadership, principals varied greatly in the extent to which they acted as instructional

leaders While our data do not definitively explain this variation,

sev-eral factors enabled district efforts: high-quality professional opment and supportive supervisors who helped principals developinstructional leadership skills and implement them daily Other fac-tors limited this ability: lack of time and lack of credibility—that is,teachers did not view their principals as knowledgeable about instruc-tion

devel-School-Based Coaching. Two districts invested in school-based

instructional coaches as a means of providing ongoing, job-embeddedprofessional development for teachers, but each implemented a dif-ferent model Although both models were intended to build the in-

structional capacity of schools and support district initiatives, teachers tended to prefer the more flexible, school-centered approach to coaching rather than the relatively standardized curriculum-centered one The

perceived value and effectiveness of coaches by teachers was greaterwhen (1) coaches tailored their work to school and teacher needs, (2)coaches advised teachers about instruction, (3) time was available tomeet with teachers, and (4) roles were clearly defined

Curriculum Specification. All districts developed and

imple-mented curriculum guidance documents that were intended to prove alignment of instruction with state standards and assessmentsand to increase consistency of instruction across classrooms andschools by specifying districtwide guidelines for the scope, pacing,and content of curriculum Two districts invested significant re-sources into developing and monitoring teachers’ use of the docu-ments

im-While district and school staff generally viewed the curriculum guides as useful for planning, promoting consistency of instruction, and helping principals observe and monitor teachers, teachers reported a lim- ited effect on pedagogy That is, teachers reported that guides influ-

enced “when” and “what” they taught, but they did not make majorshifts in “how” they taught the curriculum Teachers were apt tovalue and use the guides when they perceived them to be aligned withstate tests, received them in a timely manner, and participated in the

development process However, many teachers in all districts scribed the pacing and content of the guides as conflicting with theirneed to tailor instruction to individual students.

de-Data Use. The study districts invested to varying degrees in

multiple strategies promoting the use of data to guide instructionaldecisions, such as providing professional development on how to in-terpret test results and encouraging structured reviews of studentwork However, two districts focused much more on use of data Oneemphasized the school improvement planning (SIP) process The sec-ond district focused on interim assessments, designed to provide an

“early warning system on progress being made” toward meeting statestandards

Teachers and principals in both districts generally found the various sources of data useful and reported using them regularly to identify areas

of weakness and to guide instructional decisions Principals and teachers

in the district that focused on the SIP process, however, described theprocess as overly labor-intensive Furthermore, teachers in the districtthat focused on interim assessments were less enthusiastic about theseassessments than principals, preferring more timely, regular classroomassessment data The efforts of both districts to focus on data wereenabled by long-standing state accountability systems, accessibilityand timeliness of data, teachers’ views of the assessment results asvalid measures of students’ knowledge and ability, and the degree towhich school staff received training and support for analyzing andinterpreting data

Constraints and Enablers of Instructional Improvement

Once district leaders had designed their reform strategies and putthem into place, a number of common factors affected districts’ suc-cess in bringing about the intermediate outcomes they intended foreach set of strategies Taken as a whole, these factors led to severalcross-cutting findings:

• Although it was important for districts to implement hensive reform, they benefited from focusing on a small num- ber of initiatives While seemingly counter-intuitive, the com-

compre-Summary xxi

bination of comprehensiveness—a systemic approach, strategiesaddressing all dimensions of instruction, and a dual focus on in-frastructure and direct support—and focus on two key areas ofreform proved to be important for instructional reform in allthree districts

• District and school capacity greatly affected reform efforts.

While focusing on a few priority initiatives may have helpedconserve limited resources to some extent, all districts nonethe-less faced significant capacity gaps that hindered instructionalimprovement According to district and school staff across the

sites, capacity gaps that were most detrimental were insufficient time (e.g., for planning, to act as instructional leaders), lack and/or instability of fiscal or physical resources (e.g., instructional materials, funding), and limited capacity of central office staff

(e.g., inadequate numbers, lack of expertise)

• The broader policy context created both enabling and straining conditions for district reform For example, some

con-union policies hindered reform in two districts, and state andfederal accountability policies shaped much of the districts’ workwith curriculum and data use

• Districts’ success also was tied to several key dimensions and characteristics of the policies they developed District progress

at achieving intermediate instructional improvement goalshinged in large part on the degree to which strategies

—were aligned and mutually supportive

—enabled multiple stakeholders to engage in reform

—balanced standardization and flexibility

—used local accountability policies to provide incentivesfor meaningful change

Overall, districts generally struggled to achieve these policy tures, which might be better characterized as common challenges ortensions that districts faced in achieving systemwide change

fea-Effect of IFL on District Instructional Reform

Partnerships with the IFL contributed to district reform in all threedistricts In exchange for a fee, the IFL provided districts with a vari-ety of resources, including on-site technical assistance from IFL resi-dent fellows, opportunities to attend national meetings with otherpartner districts, advice from IFL leaders, and access to research, ma-terials, and other tools The IFL did not present an intervention ormodel for districts to implement as such but instead acted as a

“coach,” assisting districts with various aspects of instructional provement appropriate to each local context

im-In examining the IFL role in supporting district work in the fourareas of instructional reform, we found that the strongest reportedcontributions of IFL were to systemwide efforts to build the instruc-tional leadership of administrators At all three sites, the IFL influ-enced the design and implementation of professional developmentopportunities for principals and central office staff, frequently deliv-ering monthly training sessions and providing supporting materialsthat elucidated what it meant to be an instructional leader According

to district and IFL leaders, the IFL’s contribution to other areas ofreform—data use, coaching, and curriculum specification—was not

as strong or as consistent across sites

Overall, two findings emerged from our analysis of the IFL tributions to district reform

con-1 District and school leaders reported that the IFL affected the organizational culture, norms, and beliefs about instruction.

District leaders reported shifts in beliefs and norms around a set ofideas emphasized in IFL materials, professional development, andtechnical assistance These included effort-based intelligence, two-way accountability, a focus on instruction and learning, the ideathat everyone is a learner, and instruction as a public endeavor

2 The IFL was credited with helping develop the knowledge and skills of central office and school administrators The majority of

principals in all three districts reported that professional ment opportunities organized by the IFL and the districts im-proved their skills as instructional leaders, deepened their

develop-Summary xxiii

knowledge about learning, and provided them with a commonlanguage facilitating dialogue Similarly, central office leaders andstaff reported that IFL staff pushed them to focus on instructionand system-level structures and policies that enabled high-qualityinstruction They also reported that the IFL helped them becomemore knowledgeable about instruction and more skilled at super-vising and supporting principals

Both of these reported effects address key challenges facing tricts undertaking systemic reform—namely, a lack of alignmentamong district initiatives and limited capacity to undertake reform

dis-By providing a common set of ideas concerning teaching and ing, the IFL may have helped districts build mutually supportive re-form strategies around a common vision of high-quality instruction

learn-By enhancing principals’ and central office administrators’ knowledgeand skills, the IFL also may have helped build the overall capacity ofthe district to lead instructional change across a system of schools.Finally, several common factors appeared to influence IFL part-nerships with the districts and its impact on them The effect of theIFL was particularly strong when

• district and school leaders (e.g., superintendent, mid-level agers, principals) bought into the IFL’s work

man-• IFL staff were viewed as trustworthy, credible, and having tise that matched a particular district need

exper-• the IFL offered practical tools to support implementation oftheoretical ideas

In some cases, however, the IFL’s influence was constrained by

• the perception of IFL as a vendor brought in to provide ticular services without much coordination and support fromdistrict leaders

par-• the IFL’s limited capacity to support districts in all areas ofreform

• turnover within the districts and the IFL

Lessons Learned

The experiences of these three urban districts and their partnershipswith the IFL provide evidence of promising results from systemwideinstructional improvement efforts, yet they also raise warnings fordistricts and intermediary organizations about several importantchallenges they might face when attempting similar reforms Our casestudies also show that an intermediary organization can help districtsaddress persistent constraints on reform by building the capacity ofdistrict staff to engage in instructional change and by facilitating pol-icy alignment

Lessons for Instructional Improvement

Based on the reform experiences of the three study districts, we offerthe following lessons learned:

• Investing in the professional development of central office staffcan enhance capacity to lead instructional reform

• Instituting local accountability policies that create incentives formeaningful change can promote implementation

• Aligning and developing a comprehensive set of strategies canreinforce overarching instructional improvement goals

Lessons for District-Intermediary Partnerships

Although the specific characteristics of the IFL set it apart from someother types of third-party organizations, its experiences in these threedistricts nevertheless offer potentially useful insights for similar orga-nizations as well as for districts considering similar partnerships Wepresent the following observations:

• Buy-in and support from top-level leaders can affect partnershipviability

• Preexisting reform initiatives and partnerships are important toconsider when forming new partnerships

Acknowledgments

Many individuals contributed to this report First, we are grateful toleaders in the three study districts and the Institute for Learning forallowing us to conduct this research We deeply appreciate the morethan 4,500 administrators, staff members, principals, assistant princi-pals, and teachers who participated in interviews and surveys andshared their valuable time and insights with us

We acknowledge the important contribution of our reviewers

We thank Jim Spillane of Northwestern University and Mark ends of Vanderbilt University for their thoughtful reviews and com-ments We also appreciate the assistance given by Pearson NCS withsurvey development, administration, and processing The projectcould not have been completed without significant support from ourRAND colleagues, including Sheila Kirby, Sue Bodilly, Janet Han-sen, and Laura Hamilton, who offered valuable input at various stages

Ber-of the study We also thank Nancy Rizor for her valued assistance onthis report and throughout the project

Finally, we thank Mike Smith and The William and FloraHewlett Foundation for their generous support of this research

NAEP National Assessment of Educational Progress

Introduction

In the past decade, the responsibilities facing school district centraloffices have greatly increased They now include not only manage-ment and personnel duties, but also oversight of school improvement,facilitation of community engagement, and provision of professionaldevelopment The current high-stakes accountability environmentbrought on by the federal No Child Left Behind Act (NCLB) addsenormous pressure on districts to perform these roles well and todemonstrate success The threat of sanctions for districts and schoolsfailing to demonstrate improved student achievement places evengreater demands on central office administrators to provide teachersand administrators with the skills, knowledge, and resources needed

to help all students meet high academic standards These pressures areescalating rapidly as states ratchet up their progress targets to meet thefederal goal of academic proficiency for all students by 2014 Unlessthe federal government or states alter NCLB rules or targets—andmany have this past year—it will become even more difficult for dis-tricts and their schools to meet these accountability expectations overtime

In the national drive to raise student achievement, urban schooldistricts face major challenges:

• Despite some improvement in recent years, achievement levelsremain relatively low in urban districts, even when controllingfor their level of poverty.1

• Most urban districts struggle to attract and retain a qualified teaching force For example, research shows that high-poverty public schools have a significantly higher teacher turn-over rate (20 percent) than more-affluent public schools (12percent) (Ingersoll, 2001, 2003)

well-• There is rapid leadership turnover in these settings For example,

in 2003, the average tenure of superintendents in urban districtsbelonging to the Council of the Great City Schools (CGCS) was2.75 years (CGCS, 2003)

• Large urban districts enroll the majority of poor, minority, andimmigrant children in the country.2

• Despite serving a more disadvantaged population, urban schoolsspend close to the national average per pupil, and their expendi-tures have been increasing less rapidly than average expendituresnationally (CGCS, 2000, 2004; Quality Counts, 1998)

• Many central offices lack the personnel and staff expertise andskills needed to bring about systemwide improvement

Given their limited capacity, many districts look to outside ganizations for assistance In recent years, an increasing number oforganizations have emerged to address this need, ranging from orga-

or-1 For example, on the 1994 National Assessment of Educational Progress (NAEP) reading test, only 23 percent of fourth graders in high-poverty urban schools achieved at the basic level or above This statistic compares with 46 percent of students in high-poverty schools in nonurban areas; in nonpoverty schools, 69 percent of fourth graders were ranked at the basic level and above (Quality Counts, 1998) Similarly, although a more recent analysis of state assessment results in 61 city school systems found gains in math and reading performance and some signs of reduction in racially identifiable achievement gaps, urban schools as a group still scored below state and national averages (CGCS, 2004).

2 The 100 largest urban districts, representing less than 1 percent of all districts in the try, educate 23 percent of all public school children, approximately 40 percent of all non- white students, and 30 percent of students receiving free and reduced-price lunches (MDRC, 2003) In these 100 districts, 69 percent of students were nonwhite, compared to 41 percent

coun-in all school districts; 54 percent were eligible for free and reduced-price lunches, compared

to 40 percent of students in all districts (National Center for Education Statistics, 2003).

Introduction 3

nizations working with districts nationally to those working on a locallevel Many of these organizations have gained prominence and sup-port from private foundations; most notably, the Annenberg Chal-lenge grant was a catalyst for many (Kronley and Handley, 2003).These external organizations—sometimes called “nonsystem actors”(Cohen, 1995), intermediaries (Bodilly, 2001; Honig, 2004), or re-form support organizations (Kronley and Handley, 2003)—generallyseek to support “system reform.” As opposed to technical assistance orprofessional development providers supporting one facet of an orga-nization, these intermediaries envision a more comprehensive trans-formation of the organization and seek to build the capacity of schooland central office staff to support improvements in teaching andlearning

The immediate purpose of this research was to provide feedback

to the three districts and the IFL to improve their reform efforts.While a sample of three districts limits the generalizability of our

findings, the study nonetheless offers other policymakers, funders,and administrators important insights about how to improve teachingand learning in urban districts Such improvement is critical to anyattempt to erase the achievement gap between students of differentracial and socioeconomic backgrounds and to achieving the goals ofNCLB As such, the efforts of these three urban districts, as well asthe IFL’s conception of a strategy for improving schools in urban dis-tricts, shed light on strategies for improvement, outcomes associatedwith implementation, and challenges urban districts and intermedi-aries face in attempting to bring about systemwide change.

Overall, the study addressed the following questions:

1 What strategies did districts employ to promote instructional provement? How did these strategies work?

im-2 What were the constraints and enablers of district instructionalimprovement efforts?

3 What was the impact of the IFL? What were the constraints andenablers of the district-IFL partnerships?

4 What are the implications for district instructional improvementand district-intermediary partnerships?

Introduction 5

Organization of the Report

In Chapter Two, we present a brief review of the literature on schooldistrict reform and intermediary organizations, along with the con-ceptual framework and methodology of the study Chapter Threeprovides an overview and background on the three study districts andthe Institute for Learning Chapter Four analyzes and describesthe design, implementation, and selected intermediate outcomes

of key instructional improvement strategies pursued in the threedistricts—laying the groundwork for a broader analysis of cross-caseand cross-reform-strategy findings presented in the subsequent chap-ter Chapter Five identifies cross-cutting themes and key factors thatconstrained and enabled district reform, and Chapter Six providesevidence on the impact of the IFL on district instructional reformefforts and the factors influencing the effectiveness of the partner-ships The final chapter summarizes the overarching findings of thestudy and provides tentative lessons for policy and practice

Research Background, Framework, and Methods

In this chapter, we review the literature in which we grounded thestudy and its conceptual framework We then describe in detail ourframework and methodology for collecting and analyzing data on dis-trict reform and partnership efforts

What We Know from Prior Research

The framework for this study is grounded in the research on schooldistrict instructional change and on intermediary organizations Wereviewed these two literature bases with two specific goals in mind: tosituate the particular organizations we studied in a broader contextand to help develop a framework to guide our data collection andanalysis

School Districts and Instructional Improvement

A growing body of research has documented the key roles thatdistricts play in supporting improvements in teaching andlearning—building a strong case that school district central officescould and should be instruments for significant reform on a widescale (David, 1990; Massell and Goertz, 1999; Rosenholtz, 1989;Spillane, 1996, 1997) The work of New York City’s CommunityDistrict 2 is often cited as proof that districts have the capacity to beagents of instructional improvement (Elmore and Burney, 1999).This research has also identified preconditions and strategies associ-

ated with “success” (Elmore and Burney, 1999; Fullan, 2000; tower et al., 2002; Massell, 2000; Massell and Goertz, 1999;McLaughlin and Talbert, 2002; Rosenholtz, 1989; Snipes et al.,2002; Togneri and Anderson, 2003; for a review of research, seeMarsh, 2002).1 Collectively, this literature suggests the following at-tributes of reforming districts:

High-• A strong focus on teaching, learning, and instructional provement, which is supported by clear expectations and sus-tained over time

im-• A systemwide approach to reform: conceptualizing strategieswith all aspects of the system in mind and with an understand-ing that individuals at all levels are responsible for change

• Alignment and coherence of policies (e.g., alignment of riculum with standards and instruction)

cur-• Strong support for teacher learning and professional ment, including the pursuit of cutting-edge or new approaches

sup-1 The authors cited herein vary in their definitions of success Some base success on student outcomes, such as Snipes et al (2002), who examined districts demonstrating trends of im- proved student achievement and improvement that outpaced statewide gains, and Togneri and Anderson (2003), who analyzed districts that exhibited at least three years of improved student achievement across grades, subjects, and racial/ethnic groups Others focused on districts that were reform-oriented—for example, Rosenholtz (1989), who compared “mov- ing” districts (those that provided a clear focus on instruction and encouraged educators to take risks) with “stuck” districts (those with fragmented instructional goals and policies); and Massell (2000), who examined the capacity-building activities of 22 districts and focused on districts that “embraced these activities in a more comprehensive way and use them as major mechanisms for enacting improvement” (p 1) The studies cited also vary in their methods and rigor Thus, there is no solid evidence base proving that a certain set of district-level factors leads to improved student outcomes.