The academic impact of enrollment in international baccalaureate diploma programs, a case study of chicago public schools

... 48106-1346 The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools Anna Rosefsky Saavedra Ad hoc committee Meira Levinson Richard Murnane... that the IB Diploma Program is a cost-effective way to increase high-school graduation rates 1 The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of. .. of Contents Abstract v The International Baccalaureate Diploma Program The IB Diploma Program in Chicago Public Schools Research Questions Research Design DATASET SAMPLE MEASURES ANALYTIC METHODS

The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools

Anna Rosefsky Saavedra

Advisor

Meira Levinson

A Thesis Presented to the Faculty

of the Graduate School of Education of Harvard University

in Partial Fulfillment of the Requirements for the Degree of Doctor of Education

2011

All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted

In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted Also, if material had to be removed,

a note will indicate the deletion

UMI

Dissertation Publishing

UMI 3485998 Copyright 2011 by ProQuest LLC

All rights reserved This edition of the work is protected against

unauthorized copying under Title 17, United States Code

ProQuest LLC

789 East Eisenhower Parkway

P.O Box 1346 Ann Arbor, Ml 48106-1346

The Academic Impact of Enrollment in International Baccalaureate Diploma Programs: A Case Study of Chicago Public Schools

Anna Rosefsky Saavedra

Ad hoc committee

Meira Levinson Richard Murnane John Willett

A Thesis Presented to the Faculty

of the Graduate School of Education of Harvard University

in Partial Fulfillment of the Requirements for the Degree of Doctor of Education

2011

Anna Rosefsky Saavedra

1994-98 Yale University B.A

New Haven, CT June 1998 1998-99 High School Teacher, History

Pinewood School Los Altos, CA 1999-05 Educational Programs Manager

EF Educational Tours Boston, MA 2005-06 Graduate School of Education Ed.M

Harvard University May 2006 2006-11 Doctor of Education candidate

Graduate School of Education Harvard University 2006-09 Teaching Fellow

Graduate School of Education Harvard University 2009-10 Fulbright Student Scholar

Bogota, Colombia

© Anna Rosefsky Saavedra

2011

All Rights Reserved

Without the support of many people, I would not have been able to begin or finish this project I will always be thankful to every one of them for the roles they played in enabling me to write this dissertation Even after studying the impact of enrollment in the International Baccalaureate Diploma Program for past three years, I still thoroughly enjoy research of this topic and am keen to continue it as part of my future research agenda!

Specifically, I am grateful to Sara Leven from Chicago Public Schools (CPS) for providing the support necessary to begin this project in the first place and for addressing

my ongoing questions throughout its duration Thank you as well to Amy Novell for approving my use of CPS data Many thanks to Sue Sporte and Vanessa Coca from the Consortium on Chicago School Research for providing me with the data I use in this project and for also addressing my ongoing questions

I am grateful for generous financial support from the Center for the Advancement and Study of International Education, the Fulbright Foundation, the Harvard Frederick Sheldon Fund Travel Fellowship and the Harvard Graduate School of Education (HGSE) Financial resources from these organizations made it possible for me to dedicate a substantial portion of my time over the past two years to this dissertation

Several of my doctoral colleagues and friends offered valuable feedback to previous versions of this dissertation, including Liz Dawes Duraisingh, Rebecca Holcombe, Qian Guo, Maria Elena Ortega and Justin Reich I also feel gratitude toward particularly supportive HGSE professors David Perkins and Eileen McGowan, who, over

my years at HGSE, became friends as well as people for whom I have the utmost respect

To my dear friend Leila Morsy Eckert, I will forever be thankful the fates threw us

ii together as statistics partners I am so fortunate to have gone through HGSE with Leila and look forward to our annual "road race," no matter where we both happen to be living

I especially thank my advisors Meira Levinson, Richard Murnane and John Willett for supporting me through the dissertation, from beginning to end Individually, each helped me with specific parts of the process and, in sum, their advice, time and support were critical to my completion of the doctoral program and this dissertation I will miss each of them but will stay in touch and will keep their advice with me wherever

I go To Meira in particular, I will always be grateful for taking me on as her first doctoral student and then for guiding me through the doctoral program with enthusiasm and style I could not have found a more encouraging, loyal and caring advisor, who will always by my role model, mentor and friend

I am also so thankful to my parents and sister It was only by modeling my father's work ethic and determination that I was able to successfully finish my dissertation For his example and for his support in so many other ways I will always be thankful I thank my mother for many, many hours of listening, emphasizing and understanding She boosted me up when I was down and her face shone with happiness and pride when I shared good news I thank my sister Heather for her unwavering confidence in my ability to finish my dissertation She is and has always been there for encouragement or a celebration To each one of you, I am thankful for your love and support, I am so fortunate to have you as my family

And most of all, I am thankful to my dearly beloved husband, Juan Esteban, who has played all of the roles I describe above and is my best friend and lifelong partner in all things In addition to editing drafts, writing code, and otherwise helping me with the

technical aspects of writing a dissertation, his steadfast love and confidence in me provided me with the foundation from which I was able to progress through my doctoral studies and eventually complete my dissertation He writes at the end of his dissertation that he is thankful to me for conquering every bit of his dissertation milestone with me I could not feel more similarly with regards to my dissertation and all we have experienced, enjoyed and conquered together

iv

Table of Contents Abstract v The International Baccalaureate Diploma Program 1

The IB Diploma Program in Chicago Public Schools 4

Research Questions 7

Research Design 8

DATASET 8 SAMPLE 9 MEASURES 9

ANALYTIC METHODS 13

Results 17

THE IB ENROLLMENT PROCESS 17

RQ #1: THE IMPACT OF IB ENROLLMENT ON STUDENTS' ACADEMIC ACHIEVEMENT 21

RQ #2: THE IMPACT OF IB ENROLLMENT ON STUDENTS' PROBABILITY OF HIGH-SCHOOL

CHALLENGE #1: SELECTION BIAS 24

CHALLENGE #2: IB ENROLLMENT INCREASES THE PROBABILITY THAT STUDENTS TAKE THE

ACT EXAMINATION 29

CHALLENGE #3: IB ENROLLMENT INCREASES THE PROBABILITY THAT STUDENTS GRADUATE

FROM HIGH SCHOOL 30

CHALLENGE #4: DIFFERENTIAL PROBABILITY OF MISSING SEVENTH-GRADE ITBS SCORES 31

CHALLENGE #5: EXTERNAL VALIDITY 32

Discussion 32 Cost of the IB Diploma Program Effects 36

Conclusion 38

References 40 Tables and Figures 44

Vita 52

Abstract

In this study, I examine whether eleventh-grade students' enrollment in the Chicago Public Schools (CPS) International Baccalaureate (IB) Diploma Program improves their academic achievement as measured by their ACT examination scores, probability of high-school graduation and probability of college enrollment CPS offers the IB Diploma Program in thirteen high schools, more than twice any other U.S school district Using data on the IB enrollment status of 20,422 students attending these thirteen high schools from 2002-2008,1 estimate that IB enrollment increases students' academic achievement

by as much as 0.5 standard deviations and their probability of high-school graduation and college enrollment by as much as 17 and 22 percentage points respectively All of my estimates are highly robust to validity threats posed by self-selection into IB enrollment All estimates are greater for boys than for girls I also calculate that the IB Diploma Program is a cost-effective way to increase high-school graduation rates

1 The Academic Impact of Enrollment in International Baccalaureate

Diploma Programs: A Case Study of Chicago Public Schools

Originally founded as a private means for diplomats' children to earn an internationally recognized high-school diploma, today the International Baccalaureate (IB) Diploma Program is one of the fastest growing curricular innovations in U.S public-school districts In schools accredited as "IB World Schools" by the International Baccalaureate Organization (IBO), teachers use IB curriculum and pedagogy to teach a range of courses that are intended to prepare IB-enrolled students for college In this study, I examine whether IB enrollment does in fact prepare students for college Specifically, I ask whether enrollment in the IB Diploma Program increases students' academic achievement as measured by ACT college admissions examination scores,1

probability of high-school graduation and probability of college enrollment and whether

my estimates differ by gender

The International Baccalaureate Diploma Program

Over the past fifteen years, the number of U.S schools that implement IB programs has increased nearly tenfold, from 133 in 1994 to 1,218 in 2011 By 2009, public schools offered over 90% of U.S high-school level IB programs (IBO, 2010) What began as a program that a few private schools offered to wealthy children, today reaches a much broader student audience

This growth in IB implementation is based predominantly on the program's perceived academic rigor and success as a college-preparatory intervention (Byrd, Ellington, Gross, Jago, & Stern, 2007; Cech, 2008; Mathews & Hill, 2005) This

1 The ACT organization administers the ACT college admissions examination, which includes English, mathematics, reading and science sections (ACT, 2010)

reputation, augmented through recent governmental support, suggests that the growth

trend will continue For example, through the reauthorization of the Elementary and Secondary Schools Act, the U.S Department of Education has proposed to fund IB

programs as part of its objective to "increase access to accelerated learning opportunities" (U.S Department of Education, 2011a, 29) State-level education departments are also promoting IB implementation as part of their "Race to the Top" strategies (e.g FL, MA,

ME, NH; U.S Department of Education, 201 lb)

The IB school-level accreditation process and subsequent requirements for students and teachers are rigorous (Byrd et al., 2007; Mathews & Hill, 2005; U.S Department of Education, 2010) To earn accreditation from the IBO, schools must demonstrate adherence to IBO's curricular, pedagogical, mission-based and ongoing professional-development requirements as stated in the most current version of the

"Handbook of procedures for the Diploma Program" (IBO, 2009) Typically, the IB accreditation process requires extensive faculty and staff participation, takes several years

to complete and, if successful, results in authorized "IB World School" status IB School teachers must use IB curricular and pedagogical materials to teach IB courses, and participate in ongoing professional development and self-review processes

World-From a financial perspective, as of 2010, IB World-School accreditation costs schools approximately $9,500 per year during the two years of candidacy Thereafter, accredited schools pay annual fees—of $9,6000 USD in 2010—to access IB curriculum and support These fees do not differ according to the number of students estimated to enroll in IB courses, once the school is accredited IB World Schools offering the Diploma Program must also provide a way for students to take the IB Diploma

3 examinations at the end of their twelfth grade Per-student, as of 2010, registration to take

IB examinations costs $135 and the examinations themselves cost $92 per subject (IBO, 2010).2 In some schools, the students pay for the examination fees, while in others, the schools themselves pay the fees

Students enroll in the official IB Diploma Program in eleventh grade Over the course of their final two years of high school, to be eligible to earn the IB Diploma,

students must: enroll in six core IB courses and the IB Theory of Knowledge (TOK) epistemology course; participate in the weekly Creativity, Action and Service (CAS)

requirement; and write a 4,000-word "extended essay." They must also score above a defined threshold on IBO-created and administered examinations in six subjects including languages, social studies, experimental sciences and mathematics 9,000 IB-certified examiners, in 121 countries worldwide, assess the examinations (IBO, 2010)

Despite the IB program's rapid expansion, little is known about whether enrollment in the IB Diploma Program improves students' academic outcomes, including their high-school academic achievement, probability of high-school graduation and/or subsequent probability of college enrollment (i.e.: Foust, Hertberg-Davis & Callahan, 2009; Jackson, 2010; Kyberg, Davis & Callahan, 2007; Roderick, Nagaoka, Coca & Moeller, 2009) To date, only one study has examined the relationship between IB Diploma Program enrollment and students' high-school academic achievement and subsequent college entry (Roderick et al, 2009) The purpose of that study, however, was

to examine whether Chicago Public Schools' (CPS) students who enrolled in the IB Diploma Program, Advanced Placement or other honors-level courses, then enrolled in

2 The IBO tends to increase its fees on an annual basis

colleges that were appropriately competitive In contrast to the present study, according

to Roderick et al, "[their] report is not intended to be a rigorous evaluation of selective enrollment schools, IB programs or AP initiatives (2009, 2)."

The IB Diploma Program in Chicago Public Schools

In this study, I examine whether eleventh-grade CPS students' enrollment in the

IB program improves their academic achievement, probability of high-school graduation and probability of college enrollment CPS is an ideal setting in which to address this issue for three reasons First, CPS, at the forefront of IB implementation nationwide, offers the IB Diploma Program in thirteen high schools, more than twice any other U.S school district.3 Next, the system retains complete historical records of its students' IB enrollment status, achievement test scores, high-school graduation status and college-enrollment information Finally, since the average CPS student is low income and minority, evidence documenting the IB enrollment impact in this setting would enhance our knowledge of ways to improve secondary education for disadvantaged urban youth

In the early 1990s, levels of academic achievement in most CPS high schools were dismal Ninth- and eleventh-grade students' achievement scores revealed academic performance that was, on average, more than a year behind grade level when compared to national averages (Chicago School Reform Board of Trustees, 1999) The district's high-school graduation rate in 1995 was less than 50% of an entering cohort, relative to the national average of 76% (Heckman & LaFontaine, 2010).4 Among Illinois students who

3 www.ibo.org retrieved 5-30-09 Indianapolis currently has six high school level IB programs, followed by five in Philadelphia and four in St Paul, Minnesota

See Heckman & LaFontaine (2010) for a discussion of U.S high school graduation rates

5 took the 1994-1995 ACT college-entrance examination—50% of CPS seniors and 57% statewide—CPS students lagged behind their peers statewide by nearly five points out of

a maximum score of thirty-six (Rice, 1995)

Boys who attended CPS neighborhood schools fared particularly poorly For example, in the population of CPS students who were thirteen years old in 1998, only 39%o of African-American, 51% of Latino and 55% of White boys had graduated from high school by 2004, when this cohort was nineteen years old In comparison, 57% of African-American girls, 65% of Latino girls and 71% of White girls in the 1998 cohort graduated by 2004 (Allensworth, 2005) Arguably, the need for school reform in CPS was even greater for boys than for girls

From 1981 through 1997, Lincoln Park was the only public, non-magnet high school, out of more than 90 in CPS, to offer the IB Diploma Program From its inception, Lincoln Park IB students' academic achievement far surpassed that of students from other neighborhood high schools and even from most of the CPS selective-enrollment high schools (Roderick et al, 2009)

Driven by Lincoln Park's IB success and support from the Chicago mayor and business community, beginning in 1997, CPS increased rapidly the number of IB-accredited programs offered in Chicago neighborhood high schools CPS's primary goal for widely implementing the IB Diploma Program was to increase predominantly low-income and minority students' access to academically rigorous curricula in neighborhood high schools and, thereby, to increase their academic achievement and college preparedness (Spittle, Leven, & Roderick, 2008)

The thirteen CPS high-school level IB programs serve primarily low-income, minority students: CPS administrative data show that 65% of CPS students enrolled in IB are eligible to receive free or reduced price lunch and 74% are racial minorities Among non-IB (eleventh grade) students in the district, in comparison, 78% are eligible to receive free- or reduced-price lunch and 85% are racial minorities Therefore, both groups are majority-minority and low-income, but compared to non-IB CPS students, the probability is greater that CPS IB students will be White and higher income

The IB Organization (IBO) does not dictate how schools determine student eligibility for enrollment in IB programs The CPS policy is to require all students who think they would like to enroll in the official eleventh- through twelfth-grade IB Diploma Program in their geographically determined neighborhood high school to apply in their spring of eighth grade so that they can participate in an IB-preparation track during ninth and tenth grade The nature of the preparation track differs by CPS high school Several offer accredited IB-preparation programs (the IB Middle Years Program), while others offer AP courses or other honors classes

CPS requires that students submit their seventh-grade Illinois Test of Basic Skills (ITBS) mathematics and reading scores as part of the IB application requirement For the first few years of IB Diploma Program implementation, the stated CPS policy was to admit only students scoring above the sixtieth percentile on both tests This policy has evolved such that today the "unofficial" school-level policy is to admit only students scoring above the fiftieth percentiles on both tests

Research Questions

Currently, policy makers, administrators, teachers, parents and students lack credible evidence on whether enrollment in the IB Diploma Program increases students' academic achievement, probability of high-school graduation and probability of college enrollment and whether effects might differ by gender Such evidence could inform decisions about future investments in the IB Diploma Program throughout Chicago, and more broadly, in U.S public schools This deficiency motivates my research questions:

• RQ#1:

A) Does CPS students' enrollment in the IB Diploma Program in eleventh grade increase the probability that they will take the ACT college entrance examination? B) Does CPS students' enrollment in the IB Diploma Program in eleventh grade improve their high-school achievement, as measured by their ACT scores?

• RQ #2: Does CPS students' enrollment in the IB Diploma Program in eleventh

grade increase the probability that they will graduate from high school?

• RQ #3: Does CPS students' enrollment in the IB program in eleventh grade

increase the probability that they will enroll in a two- or four-year college at some point during the first two years after their graduation from high school?

• RQ #4: Do potential effects of IB enrollment on students' probability of taking the

ACT examination, academic achievement as measured by ACT scores, probability of graduation from high school and probability of enrollment in college, respectively, differ by gender?

Research Design

Dataset

The data that I analyze here contain information merged from two sources, from

the Chicago Public Schools (CPS) themselves and from the National Student Clearinghouse (NSC), a non-profit organization that verifies students' enrollment in, and

graduation from, 92 percent of U.S colleges (National Student Clearinghouse, 2011).5

The CPS dataset contains records for the 20,422 students who attended one of the thirteen CPS high schools that offered the IB Diploma Program to the 2002-2003, 2003-2004, 2004-2005 and 2006-2007 eleventh-grade cohorts.6 It contains measures of students' IB enrollment status, previous academic performance, gender, race/ethnicity and family income, as well as students' mathematics, English, reading and science scores on the ACT test, and indicators of whether they graduated from high school

Data on students are included in the NSC dataset if they enrolled in a college that

is included among the ninety-three percent of colleges whose admissions data is collated

5 The majority of CPS students who enroll in college attend postsecondary institutions located in Illinois (Roderick, 2006) According to the NSC's Assistant Director of Research Services, "the NSC post-secondary enrollment data coverage for students in Illinois is higher than the national average of 92%" (personal correspondence, February

23, 2011) By cross-referencing NSC data with CPS exit survey data, Roderick, Nagaoka, Allensworth, Stoker, Correa, & Coca (2006) estimate that the NSC data "misses approximately 5 percent of CPS graduates who may be enrolled in college" (cited in Roderick, 2006, footnote 56) Roderick et al also characterize the Illinois colleges for which the NSC does not verify enrollment as "primarily local proprietary and technical institutions" (2006) Contingent upon the assumption that compared to IB students, the probability is greater that non-IB students will attend local proprietary and technical institutions, missing NSC enrollment data could contribute to a slight understatement of the percentage of non-IB students who enroll in college

I do not have access to records for the 2005-2006 eleventh grade cohort I have no reason to believe that the results of my analysis would differ if I could include those records in my full sample

9

in the National Student Clearinghouse at some point during the first four semesters graduation from high school I located fifty percent of the 20,422 students in the CPS sample in the NSC dataset, indicating that at least half of CPS students in my full sample enrolled in two- or four-year college for at least one semester during the first four semesters after their graduation from high school

post-Sample

My full sample of CPS students (n=20,422) contains every eleventh-grade student

in each of the thirteen CPS high schools that offered the IB program to the eleventh-grade cohort in academic years 2002-2003 (n=5042), 2003-2004 (n=5,097), 2004-2005 (n=5,181) and 2006-2007 (n=5,102) I use data from these years because college enrollment data is available on each of these cohorts I combine four annual cohorts of students to improve the statistical power of my analyses and enhance the external validity

ACT, and which I formed by conducting a principal-components analysis (PCA) of

eleventh-grade students' mathematics, English, reading and science scores on the ACT college entrance examination The resulting continuous variable is a symmetrically distributed, weighted linear composite with an estimated Cronbach's alpha internal-

consistency reliability coefficient of 0.89 In total, 2,667 students—1 percent of IB students and 14 percent of non-IB students—are missing on at least one of the four constituent achievement tests and thus are also missing on the academic achievement outcome itself After compositing, for ease of interpretation, I standardized the obtained first principal component score to a mean of zero and a standard deviation (S.D.) of unity

in my analytic sample of students (n=14,368) who were not missing seventh-grade mathematics or reading scores nor one of the ACT subject-level tests Students take the constituent achievement tests during the late spring of their junior year, and so my estimated "treatment effects" represent the impact of only one year of official IB enrollment on the academic-achievement outcome

«Insert Table 1 h e r e »

My second outcome, HSGRAD, is a dichotomous indicator that I coded 1 if

students graduated on time from high school with a regular CPS degree or an alternate Illinois degree (0 otherwise).7 I possess this information for every student in the full sample, of which 99 percent of IB and 78 percent of non-IB students graduated from high school.8

My third outcome, COLLENROLL, is a dichotomous indicator that I coded 1 for

students who enrolled in either two- or four-year college for at least one semester during one of the first four semesters post-graduation from high school (0 otherwise) The NSC only matches CPS high-school graduates to its college enrollment database Therefore,

7 The requirements for the CPS high school degree are more demanding than the requirements for an Illinois degree My data does not indicate whether students later earned the General Education Development equivalency degree (GED)

8 Though these proportions seem high given the disadvantaged context, these students persisted beyond ninth grade, the year in which CPS students are at the highest risk of dropout (Allensworth, 2005)

11

students' COLLENROLL status is conditional on whether they graduated from high school, such that the COLLENROLL=0 for students for whom HSGRAD=0 83% of IB

students and 48% of non-IB students enrolled in college

If IB enrollment affects the probability that students will graduate from high school, then relative to non-IB students, a greater proportion of IB students will be included in the sub-sample of students that I use to estimate the impact of IB enrollment

on students' probability of college enrollment The overrepresentation of IB students in this sub-sample would then contribute to biasing my estimation of the impact of IB enrollment on students' probability of college enrollment I discuss this potential threat to

a causal interpretation of my estimates of the impact of IB enrollment on students' probability of college enrollment in the "Threats to Validity" section of this paper

Question predictor

I define my question predictor, IBTREAT, as whether students enrolled in the core

IB Theory of Knowledge course for at least a semester (l=yes, 0=no) According to the

CPS IB Diploma Program Administrator (personal correspondence, July 6, 2010),

students typically only enroll in TOK if they are attempting to fulfill the IB program

requirements and only rarely as a way to take honors courses in a piecemeal fashion The

students that enroll in TOK, on average, enroll in eleven IB courses (SD=4), confirming this characterization of TOK course enrollment According to my definition, seven

percent (1,403 students) of the full sample of 20,422 students was enrolled in the IB treatment

Covariates

I include in my analysis students' seventh-grade mathematics (MATH) and

reading (READ) test percentiles, which are based on their national percentile rank on the

ITBS test, as measures of their previous academic achievement CPS administrators use these test percentiles to determine IB eligibility 26% (5,261) of students in my full sample and 20% (279) of IB students are missing these scores I eliminate students

missing on MATH or READ, leaving an analytic subsample of 15,148 students As I show

in Table 1, among IB students, the mean ITBS mathematics-achievement percentile is 79.47 (SD=16.18) and the mean reading-achievement percentile is 73.86 (SD=17.71) In comparison, the non-IB mean mathematics percentile of 50.27 (SD=24.95) and mean reading percentile of 47.13 (SD=22.97) are each almost thirty percentiles lower

I also include as covariates in my analyses measures of students' demographic

characteristics, including their gender, family income and race I coded MALE as 1 for

the boys, who make up 48% of non-IB students and 35% of IB students I represent

family income using LUNCH, a dichotomous indicator that I coded 1 for students who

were eligible to receive free or reduced-price lunch Over all, 65% of IB students were eligible, compared to 78% of non-IB students, indicating that on average, both non-IB and IB students are from low-income families, though the probability that non-IB students are from poorer families is greater I represent student race using a vector of

dichotomous indicators that describe whether a student was Asian-American (ASIAN), African-American (AFAM), Latino (LATINO), Native American or Alaskan American (NATIVEAM), or White (WHITE), dropping the last to create a reference category

Among all IB students, 30 percent are Latino, 28 percent are African-American, 26 percent are White, 17 percent are Asian-American and less than 1 percent are Native

13 American or Alaskan, while the corresponding proportions are 42%, 37%, 15%, 6% and under 1%, respectively, among non-IB students

In following section, I use the generic variable name STUDENTDEM to represent

the vector of covariates that record students' ITBS test percentiles, gender, family income and race

Analytic Methods

My objective is to estimate the causal impact of enrollment in the IB Diploma Program on CPS eleventh-grade students' academic achievement, probability of high-school graduation and probability of college enrollment I use the same analytic strategy

to address each of my research questions, which I describe for outcome, Y, for student /' in high school j in cohort c I first assume selection into the IB Diploma Program based on students' observed characteristics by using a propensity score approach to estimate the

impact of IB enrollment on my three measures of students' academic success Then, as I discuss in the "Threats to Validity" section of this paper, following Rosenbaum (2002), I test the sensitivity of my estimates to different levels of selection bias

In fitting my propensity-score selection model, I use logistic regression analysis to

model students' probability of enrolling in the IB program as a function of a set of measured characteristics that I have determined theoretically should predict enrollment (Murnane & Willett, 2011 311) I include students' seventh-grade mathematics and reading test percentiles as the principal predictors in my propensity score selection model because I anticipate that these test results will be strong predictors of whether students

enroll in the IB Diploma Program, given the CPS IB-admission policy.91 also include an indicator of gender because CPS boys' tendency to fail classes, earn lower grade-point averages and drop out of school at higher rates than girls (Allensworth, 2007) suggests that they have a lower theoretical probability of CPS IB enrollment eligibility than girls I include measures of race and family income as predictors because historically IB has attracted White, Asian-American and economically advantaged students, while African-American, Latino and economically disadvantaged students have not had the academic preparation necessary to enter the rigorous Diploma Program (Burris, Welner, Wiley, & Murphy, 2007)

I therefore fit the following logistic regression model (1) to represent the

probability that student / in schooly in cohort c chooses to enroll in the IB program:

¥r(IBTREAT lJC = 1) = -P 0+ P,STUDENTDEM IJC+ S J+ \

I include school (S,) and cohort (Xc) fixed effects to allow for the probability that student i

will enroll in IB to differ by school and cohort The fixed effects address the concern that schools' and cohorts' unmeasured characteristics might contribute to determining whether students will enroll in IB

Based on the fitted propensity score selection model, the propensity scores (PHAT) are the predicted probabilities that students will enroll in the IB program Once I have obtained PHAT for each of the students in my analytic sample, I follow Imbens and

Wooldridge (2009) in eliminating from my analytic sample all non-IB students with

9 I use quadratic functional forms of the mathematics and reading percentiles because this more flexible form ought to be a better fit for IB students' ITBS scores Estimates of the

IB enrollment impact are statistically indistinguishable in models in which I include respective nested linear, quadratic, cubic and quartic functional forms of the ITBS percentiles

15 estimated propensity scores that fall outside of the range of estimated propensity scores for the IB students This process aligns the IB and non-IB groups explicitly on the selection model covariates, ensuring that the non-IB students in my comparison sample are as similar as possible in terms of measured characteristics to IB students The resulting comparison sample includes 11,592 student records.10

Then, following Hirano, Imbens and Ridder (2003), in my subsequent analysis of

program impact, I use PHAT io provide an inverse-probability weighting, IPW, for each

student in my comparison sample I create the weights as follows: For IB students:

IPW=\IPHAT; for non-IB students: IPW=\l(l-PHAT) The purpose of this weighting

strategy is to allocate the least weight to the IB students with the highest propensity for selection into IB and the greatest weight to the IB students with the lowest propensity for selection into IB (and vice versa for non-IB students) in order to impose identical (observed) covariate distributions for students who enrolled versus did not enroll in the

IB program

The final step is to fit my weighted least squares (WLS) estimation model (2),

which represents the relationship between my academic outcomes and membership in the

IB treatment, weighting each student i in school j in cohort c by his or her IPW as

follows11:

101 define the 3,469 students who were missing on an IB treatment status as being non-IB rather than dropping them My rationale is that according to model (1), the distribution of propensity scores for students missing on IB status (mean=0.053, SD=0.113) is to the left

of the distribution of propensity scores for students who explicitly did not choose the IB option (mean=0.065, SD=0.113) Therefore, students missing on IB status, if anything, look more like non-IB students than IB students, for whom the mean propensity score is 0.294 (SD=0.210)

11 In practice, I weight each observation by the square of the weights (Imbens &

Wooldridge, 2009)

(2) Y ijc = /3 0 + BJBTREAT ijc + B 2 STUDENTDEM ijc +6 j+Yc + e ijc

Where Y is the outcome in question (probability of taking the ACT examination, student

academic achievement, probability of high-school graduation, or probability of college

enrollment) I control for the vector of student characteristics, STUDENTDEM and

include school (6,) and cohort ( y c ) fixed effects to increase the precision of my estimates (Imbens & Wooldridge, 2009) I represent the residuals with the term Ey C and estimate standard errors to account for co-variation in students' outcomes within schools, the unit

at which the IB treatment is administered Assuming selection into IB based only on students' observed characteristics, I can interpret Bi as the impact of IB enrollment on

outcome Y

To determine if the impact of IB enrollment differs by gender, I fit model (3)

below, including the main effect of MALE in the STUDENTDEM term, the two-way interaction term IB*MALE and again weighting each student i in school j in cohort c by IPW:

(3) Y ljc = B 0 + pjBTREAT ijc + 6 2 IB*MALE ijc + 6 3 STUDENTDEM ijc + d j +Yc +s ijc

I interpret the Bi parameter estimate as the impact of IB enrollment on girls and the B2 parameter estimate as the extent to which the impact of enrollment in IB differs according

to gender

Compared to using Ordinary Least Squares (OLS) methods to address my research questions, the propensity-score approach is a more effective, robust and partially non-parametric way of controlling for observed bias attributable to measured covariates (Murnane & Willett, 2011, 310) By using the propensity-score approach, I must first model and examine explicitly the process of students' selection into the IB program The

17 extent to which I am able to capture the selection process successfully in my selection model then improves my ability to remove selection bias from my subsequent estimates

of the causal impact of IB enrollment on academic outcomes (Murnane & Willett, 2011, 288) The weighting strategy—as compared to stratifying or matching strategies—is my preferred use of the propensity scores because it allows me to impose the identical covariate distributions for IB and non-IB groups and to include in my comparison sample all non-IB students who are most comparable to IB students based on observed characteristics (Murnane & Willett, 2011) Finally, propensity scores constitute the basis

of Rosenbaum's sensitivity analysis, to which I subject my results and describe later in the "Threats to Validity" section of this paper

Results

The IB enrollment process

In Table 2, I show the results of fitting Model (I), which describes the fitted

relationship between students' probability of enrolling in the IB Diploma Program and predictors that include their demographic characteristics and previous academic achievement In the table, I present parameter estimates and standard errors for each of the demographic and ITBS test percentile covariates that, I have argued, participate in the

IB selection process

«Insert Table 2 h e r e »

In addition, to demonstrate the substantive nature of the IB selection process visually, in Figure 1, I plot the fitted probability that prototypical students will enroll in the IB program as a function of their seventh-grade ITBS reading percentiles In the figure, I present fitted lines for students who attended one CPS high school that offered

the IB program to the eleventh-grade 2006-2007 cohort, who scored at the fiftieth percentile on the ITBS mathematics test, and who were eligible to receive free or reduced price lunch (n=303) The eight fitted lines compare the fitted probabilities that Latino, African-American, White and Asian-American girls and boys enrolled in the IB program,

as a function of their seventh-grade reading test percentiles I use solid lines to represent the fitted selection probabilities for boys and dashed lines for girls I use black to represent Latino students, dark grey to represent African-American students, medium grey to represent White students and the lightest grey to represent Asian-American students

«Insert Figure 1 h e r e » Among Title 1 U.S high schools that offer the IB program, approximately two-thirds require that students demonstrate strong previous academic ability to be eligible to enroll The most commonly used selection criterion is grade point average, followed by standardized test performance (Siskin & Weinstein, 2008) Like enrollment policies in the majority of IB-offering Title 1 schools, the CPS IB enrollment policy requires that students demonstrate their academic ability to be eligible to enroll Specifically, students must score over the fiftieth percentile on both the mathematics and reading seventh-grade tests to be eligible Given this enrollment requirement, I had anticipated and the CPS IB Diploma Program Administrator concurred that it would be more probable that students who have demonstrated stronger prior academic achievement would enroll in the IB Diploma Program (personal correspondence, July 6, 2010)

The results that I present in Table 2 and Figure 1 verify my expectation about the relationship between students' previously demonstrated academic strength and the

19 probability that they will enroll in the IB program Highlights of Table 2 include that holding constant gender, race, lunch-eligibility status, school and cohort, the fitted probability that students will enroll in the IB program is greater for students with higher seventh-grade mathematics (p=0.087) and reading (p=0.001) scores Correspondingly, in Figure 1, I show that for all students who score below the fiftieth percentile on the seventh-grade ITBS reading score, there is close to a zero fitted probability that they will enroll in an IB program The fitted enrollment probability is then higher for students whose prior reading scores are higher

Gender is also an important predictor of IB enrollment As I show in Table 2, holding constant seventh grade test scores, race and lunch-eligibility status, school and cohort, the fitted odds that a boy will enroll in the IB program is half that of a girl (e"701=0.50; p<0.00l) In Figure 1, within racial groups represented by the same color

line, each of the dashed fitted lines (representing prototypical girls) indicates that there is

a greater fitted probability of enrollment in the IB program than do the solid fitted lines representing boys In fact, as the dashed lines show, the fitted probability that all White, Latino and African-American girls will enroll in the IB program is greater than the probability that all White, Latino and African-American boys will enroll I had anticipated this result, given that the theoretical probability that boys are eligible to enroll

in the IB Diploma Program in the first place is lesser than that of girls

The race-based differential in the fitted probability of enrollment in the IB program aligns with the Burris et al (2007) argument that White and Asian-American students, on average, are better prepared for rigorous courses like those offered through

IB than are African-American and Latino students In the CPS context, the fitted lines in

Figure 1 show that holding constant seventh-grade test scores, lunch eligibility status, school and cohort, the fitted probability that Asian-American students—boys and girls— enroll in the IB program is greater than the corresponding probability that students of any

other racial group enroll (p<0.00l) In addition, I show that the fitted probability of White

and Asian-American students' IB enrollment is greater than the corresponding probability for African-American and Latino students' enrollment

In Table 3,1 show the differences in measured covariates between IB and non-IB students, unweighted and then weighted, for my comparison sample of students with overlapping propensity scores (n=l 1,592) In the first and second columns, for non-IB and IB students respectively, I show the unweighted means and S.D.'s for each of the demographic characteristics and test score percentiles that I included as predictors in the propensity-score selection model In the third column, I show the unweighted estimated differences between non-IB and IB students for each of the predictors In the fourth and fifth columns, I again present the corresponding means and S.D.'s for each of the covariates, but now weighted Finally, in the sixth column, I show the estimated weighted differences in covariates between non-IB and IB students

«Insert Table 3 h e r e » Though I have restricted the sample of students in my comparison sample to include only non-IB students with propensity scores that overlap with those of IB students, statistically significant differences on covariates between the two groups remain

when the analyses do not incorporate the IPW weights As I show in the third column,

unweighted, and compared to non-IB students, there is a greater sample probability that

IB students will be female (p=0.0l3) and Asian-American (p=0.031) and a lesser sample