Teacher Impact on Student Proficiency and Growth NCEA

Teacher Impact on Student Proficiency and GrowthCatholic educators are called now more than ever to be able to demonstrate that students are proficient, and achieve more than a year’s gr

Teacher Impact on Student Proficiency and Growth

Catholic educators are called now more than ever to be able to demonstrate that students are proficient, and achieve more than a year’s growth during a school year, because they attend Catholic schools

Ronald W Costello, Barbara Shuey, Peggy Elson and Kathy Mears

Catholic schools are committed to providing strong academic environments, which enable students to grow to their full potential We must demonstrate that we are constantly making this difference Through more than 20 years of research, William Sanders found that classroom teachers are the most important factor in achieving gains in student achievement Reports by education researchers John Kain, Eric Hanushek, William Sanders and others have demonstrated that good instruction is 15 to 20 times more powerful than family background and income, race, gender and other explanatory variables To be successful, Catholic educators are called now more than ever to be able to demonstrate that our students are proficient and achieve more than a year’s growth during a school year

Through a school improvement grant from Lilly Endowment Inc., the Archdiocese of

Indianapolis used the SAS-EVAAS (William Sanders) value-added analysis for the last four years The analysis determined whether the school and classroom teacher achieved a year’s academic growth with the students they serve This article will explain the value-added model utilized to report teacher impact and discuss the findings on student achievement In order to do this, it is important to understand how both proficiency and growth influence student

achievement

Possible School Outcomes

Chart 1 Theodore Hershberg,

Writing about

“Value-Added Assessment and

System Reform” in an

article in the Phi Delta

Kappan (2005), describes

the possible school and

teacher outcomes on

student achievement This

effect is a balance

between two factors –

proficiency and growth

(Chart 1) How the teacher

balances the two in the

classroom can determine

whether the students reach

their full potential

Proficiency is the ability

Possible School Outcomes

Low Proficiency Low Growth

High Proficiency Low Growth

High Proficiency High Growth

Low Proficiency High Growth

P

r

o

f

i

c

i e n c y

Growth

Theodore Hershberg Value-Added Assessment and Systemic Reform, PDK, Dec 2005

of students to meet an expected level of academic performance based upon standards Growth is the gain in student achievement from one year to the next

Students in the Archdiocese of Indianapolis take the Indiana Statewide Test of Educational Progress (ISTEP) in grades 3 through 10 The ISTEP was developed by the Indiana Department

of Education to report student progress in meeting the Indiana academic standards It is the primary measure of student growth and progress in the state Because all of the archdiocesan schools are accredited by the Indiana Department of Education, the archdiocese has been able to report that our students are proficient in meeting the Indiana state standards But this tells little about the impact that the teacher has on student achievement from one year to the next In order

to do this, the archdiocese needs to measure and report annual growth Value-added is our model for this purpose

Why Value-Added?

It is vital to understand why a value-added analysis is informative The first step is to be able to look at student progress over time Chart 2 shows the average number of students passing both the English/language arts and mathematics ISTEP assessments at grades 3 through 10 for school year 2006-2007 The three groups compared are archdiocese, nonpublic schools and state average

Chart 2

The chart displays a trajectory using the average number of students passing the

English/language arts and mathematics sections for the archdiocese (13,186 students), non-public

the lack of growth at the upper grades, yet this is not a value-added comparison because there is

no determination about expected level of growth from one year to the next We could be seeing from the chart the difference in student ability versus the impact that the school or teacher has had on student achievement In order to determine the impact of the school or teacher, we need to match individual student achievement data from one year to the next and determine an expected level of growth

ISTEP Value-Added Results

For the last four years, the archdiocese has compared test results from year-to-year using the value-added model developed by Sanders and now administered by the SAS®

EVAAS® Institute in North Carolina The assumption of this model is that for a student to show growth, the student must achieve a “gain score” that is greater than one-year’s growth

For the first three years, the archdiocese used the Terra Nova test by CTB McGraw-Hill for this comparison because ISTEP did not test students at all grades During the 2005-06 school years, with two years of ISTEP results from grades 3 through 10, we were able to get value-added results for 29 schools using the ISTEP scores

The data was collected from eight schools participating in the improvement efforts An

additional 21 schools were part of the control group, which provided needed data to make the value-added predictions for the eight schools As Chart 3 shows, of the 29 schools, 13 schools demonstrated a year’s growth, which is a three (3) or No Discernible Difference Eight schools had value-added growth and another eight had no value-added growth at a level to be considered significant This analysis assisted us in demonstrating both proficiency and growth for the 29 schools

Chart 3

Proficiency and Growth Results

For the 29 schools for which we had value-added scores, we created a chart to display

proficiency and growth (Chart 4) Based upon the five-point value-added score, each school was placed into one of three categories No growth was a one or two value-added score, average growth was a three, and high growth was a four or five added score Along with the value-added score we used the combined average score in passing both English/language arts and mathematics subject areas for the school The student must pass both parts to be considered passing, which is a higher expectation than the average passing used in the initial chart For the state of Indiana, the combined score is about 10 percentage points lower than the average score

at each grade level The archdiocese used the combined average because we wanted higher expectations for our schools than was required by the state of Indiana

Chart 4

In analyzing the results, seven of the 29 schools had both high proficiency and growth Based upon possible school outcomes in the Hershberg chart, we expect all schools to be proficient and show growth In order to accomplish this, the teacher needs to understand what is expected, analyze the data and modify instruction in the school year in order to achieve both high

proficiency and growth with all students

This year we will provide growth and proficiency reports for all 67 archdiocesan schools and not just the pilot schools in our original improvement effort and we will release proficiency and growth data to all archdiocesan schools so teachers can plan to modify instruction

Battelle for Kids, a nonprofit organization in Columbus, Ohio, was established to “enhance student learning by bringing clarity to school improvement.” This improvement in teaching and learning is to be achieved through The Power of Two: Progress and Achievement Progress is defined as student growth and performance from one year to the next, and achievement is

meeting the expected level of student performance If we use the terms from Hershberg,

progress is growth and achievement is proficiency; how Catholic schools address proficiency and growth will determine our success in this era of teach accountability

When we are able to show that our Catholic teachers are effective because of the high percentage

of students passing a standards-based assessment such as the ISTEP, and students in our

classroom are gaining a year’s growth in a school year, then we will have met our goals

Providing teachers with information about student proficiency and growth, and measuring the impact teachers have on student learning help identify best practices that can be replicated and rewarded The goal of all Catholic educators is to demonstrate that they have achieved growth and proficiency for all students This should convince all about the value of excellent teachers in our Catholic schools

References

A Toolkit for School Leaders: Understanding & Using Value-Added Analysis Battelle, for Kids, 2006.

Costello, R., Shuey, B & Elson, P Performance pay − a case for corporate and foundation support Momentum, 37,

34-37.

Dr Ronald W Costello has been superintendent of Catholic education for the Archdiocese of Indianapolis for the

past five years The archdiocese has 68 schools with almost 23,000 students in kindergarten through 12 th grade Prior to working in Catholic education, he worked for more than 30 years in public education as a teacher,

counselor and administrator at both the building and central office level ( rcostello@archindy.org )

Barbara Shuey is director of Catholic Education Services for Teachscape, Inc Prior to this position, she served in

the Archdiocese of Indianapolis for 25 years as elementary principal, high school president and, most recently, as associate director of the Center for Catholic School Excellence in the Office of Catholic Education She worked for seven years as school administrator of the Indiana United Methodist Children’s Home She currently is a lecturer in education at Marian College in Indianapolis ( bshuey@sbcglobal.net )

Peggy Elson is in her third year as TAP program director for the Archdiocese of Indianapolis She taught for 25

years at Saint Matthew School followed by two years at Saint Lawrence School as TAP master teacher

( pelson@archindy.org )

Kathy Mears is associate director for school learning resources for the Archdiocese of Indianapolis During her 25

years in Catholic education, she has served as a principal and as an elementary and high school special needs teacher She is a lecturer in education at Marian College in Indianapolis ( kmears@archindy.org )

Graphical Summary of Educational Findings from the Tennessee Value-Added Assessment System (1996)

Knoxville, TN: University of Tennessee Value-Added Research and Assessment Center.

Hershberg, T., (December 2005) Value-added assessment and system reform

Phi Delta Kappan.

Kain, J.F (November 17, 1998) The impact of individual teachers and peers on individual student achievement Association for Public Analysis and Management.

Mahoney, J.W (March/April 2006) How Value-Added Assessments Helps Improve Schools.

Phi Delta Kappa Edge.

Sanders, W L (2003) Beyond No Child Left Behind Presented at the 2003 Annual Meeting American Educational Research Association, Value-Added Approaches to School Accountability: Results and Lessons from an Evaluation, Chicago, IL.

Sanders, W & Rivers, J (1996) The Cumulative and Residual Effects of Teachers on Future Student Academic Achievement University of Tennessee Value-added Research and Assessment Center.

Adapted from an article in Momentum, the Official Journal of the National Catholic Educational Association,

April/May 2007 Used with permission Contact: Barbara Keebler, NCEA, (202)378-5762.