Linking Policies and Implementation to Learning Outcomes

This chapter focuses on analyzing student learning outcomes in Shanghai and examines how varying characteristics and education practices across schools are correlated with student learning outcomes, as measured by the 2012 Programme for International Student Assessment (PISA) results. PISA is designed to measure the cognitive skills of 15yearolds, mainly in math, science, and reading. The 2012 PISA also included for the first time a module on “problemsolving skills,” which is paid particular attention to in this chapter (box 7.1).

Trang 1

Linking Policies and Implementation

to Learning Outcomes

Introduction

This chapter focuses on analyzing student learning outcomes in Shanghai and

examines how varying characteristics and education practices across schools are

correlated with student learning outcomes, as measured by the 2012 Programme

for International Student Assessment (PISA) results PISA is designed to

mea-sure the cognitive skills of 15-year-olds, mainly in math, science, and reading

The 2012 PISA also included for the first time a module on “problem-solving

skills,” which is paid particular attention to in this chapter (box 7.1)

Shanghai’s performance on pISa 2012

A total of 5,177 students from 155 schools in Shanghai participated in PISA

2012 (tables 7.1 and 7.2) Sampling was done in strict accordance with

Organisation for Economic Co-operation and Development (OECD) protocol

and quality assurance to generate a representative sample of 15-year-olds in

school in Shanghai

Shanghai continued to be the top performer on all three major domains of

PISA (mathematics, reading, and science) in 2012 Its mean mathematics score

of 613 points, representing a 4.2 percent annualized increase from 2009, is 119

points above the OECD average, the equivalent of nearly three years of schooling

Its mean score of 570 points in reading represents an annualized improvement

of 4.6 percent since 2009 and is equivalent to more than a year and a half of

schooling above the OECD average of 496 points Its mean score in science, 580,

is more than three-quarters of a proficiency level above the OECD average of

501 points

Furthermore, Shanghai also had the largest proportion of top performers

( proficient at level 5 or 6) in mathematics (55.4 percent), reading (25.1 percent),

and science (27.2 percent) Particularly, with 30.8 percent of students attaining

level 6 in mathematics, Shanghai is the only PISA participant with more students

Trang 2

Box 7.1 Definitions of pISa Domains

reading literacy: An individual’s capacity to understand, use, reflect on, and engage with

written texts, so as to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society

Mathematical literacy: An individual’s capacity to identify and understand the role that

mathematics plays in the world, to make well-founded judgments, and to use and engage with mathematics in ways that meet the needs of that individual’s life as a constructive, con- cerned, and reflective citizen

Scientific literacy: An individual’s scientific knowledge and use of that knowledge to identify

questions, to acquire new knowledge, to explain scientific phenomena, and to draw evidence-based conclusions about science-related issues; understanding of the characteristic features of science as a form of human knowledge and enquiry; awareness of how science and technology shape our material, intellectual, and cultural environments; and willingness to engage in science-related issues, and with the ideas of science, as a reflective citizen

problem-solving skills: The problem-solving assessment of PISA 2012 was designed to focus

as much as possible on cognitive processes and generic skills rather than domain-specific knowledge Problem-solving competence is defined as an individual’s capacity to engage in cognitive processing to understand and resolve problem situations where a method of solu- tion is not immediately obvious It includes the willingness to engage with such situations to achieve one’s potential as a constructive and reflective citizen

Source: OECD 2013, 4, 17

table 7.1 Number of Schools in pISa 2012 Shanghai Sample

Source: Data from OECD 2012, PISA 2012 database (http://pisa2012.acer.edu.au/)

Note: PISA = Programme for International Student Assessment.

at this top level than at any other level Moreover, Shanghai is one of the most equal education systems among the PISA participants For example, it has the highest proportion of resilient students (19.2 percent), that is, disadvantaged students who perform among the top 25 percent of students across all participat-ing countries and economies after controlling for socioeconomic status The strength of the relationship between mathematics performance and socioeco-nomic status is also below the OECD average

Trang 3

Comparing performance between programs

Among 15-year-olds in Shanghai, students attending senior secondary general

programs achieved the highest scores on all four PISA domains (mathematics,

science, reading, and problem solving), followed by those attending junior

sec-ondary programs (table 7.3)

The gap between general and vocational senior secondary students is

particu-larly large In fact, the average scores of vocational senior secondary students are

lower than those of general junior secondary students on all four domains

(figure 7.1)

Among senior secondary general program students, those attending model or

experimental schools scored higher than those attending ordinary schools on all four

domains If model or experimental school students are compared with vocational

school students, the largest gap in performances is 178 points (on mathematics)

table 7.3 performance on Mathematics, Science, reading, and problem Solving, by program

and Ordinary versus Model

PISA scores

Subject

Junior secondary

Senior secondary general

Senior secondary vocational Ordinary Model Shanghai

Note: PISA = Programme for International Student Assessment; S.E = standard error

table 7.2 Number of Students in pISa 2012 Shanghai Sample, by type of School and

program

Junior secondary/general Junior secondary school 1,899

General senior secondary school 31 a

Senior secondary/vocational Vocational secondary school 1,083

Source: Data from OECD 2012, PISA 2012 database (http://pisa2012.acer.edu.au/).

a These students attend a general junior secondary program in a general secondary school, or they attend a general senior

secondary program in a vocational secondary school.

Trang 4

Figure 7.1 performance on Mathematics, Science, reading, and problem Solving, by program and

Ordinary versus Model

Upper secondary/vocational

Source: Data from OECD 2012, PISA 2012 database (http://pisa2012.acer.edu.au/).

Comparative data from the 2012 OECD report reveal that between-school variation explains 47 percent of the total variation in mathematics performance among students in Shanghai for PISA 2012, slightly higher than Hong Kong SAR, China (40 percent); Taiwan, China (40 percent); the Republic of Korea (39 per-cent); and Singapore (37 percent); but lower than Japan (53 percent) (figure 7.2) Additionally, it was found that as much as 58.8 percent of the between-school difference in Shanghai is explained by study programs (lower vs upper level and vocational vs general orientation), much higher than the OECD aver-age (40 percent) and other education systems in the region (for example, 7.6 percent in Hong Kong SAR, China; 13 percent in Japan; and 35 percent in Korea and Taiwan, China)

The following sections first compare the student and school characteristics between programs, then investigate, within each program (junior secondary, senior secondary general, and senior secondary vocational), how school-level characteristics are associated with student performance

Comparing Individual and Family Background Characteristics between programs

Individual and family characteristics of students attending junior secondary, eral senior secondary, and vocational senior secondary programs differ significantly from each other (table 7.4) A total of 56 percent of general senior secondary

Trang 5

gen-Figure 7.2 percentage of total Variation in pISa Mathematics performance explained by Between-School Variation and Study programs (junior or senior secondary level, vocational or general)

Note: OECD = Organisation for Economic Co-operation and Development; PISA = Programme for International Student Assessment.

Trang 6

students are girls, a higher proportion than in vocational senior secondary and junior secondary programs General senior secondary students, on average, come from wealthier families with more home educational resources and cultural pos-sessions and higher parental education levels than those attending vocational senior secondary programs Parents of general senior secondary students have on average almost two more years of education than those of vocational senior sec-ondary students in Shanghai About 93 percent of general senior secondary school students have attended preschool for at least a year, compared with 85 percent of vocational senior secondary students and general junior secondary students Among general senior secondary students, those attending model or experi-mental schools enjoy more family wealth and home educational resources Parents of model or experimental school students have almost an additional year

of education compared with those of ordinary school students Family cultural possessions and proportion of students attending preschool do not differ signifi-cantly between ordinary and model or experimental school students

Comparing School Characteristics

About 90 percent of the junior secondary schools and vocational senior ary schools are public.1 The proportion of public schools is lowest among mixed secondary schools (76 percent), whereas all general senior secondary schools are public

second-All of the private schools represented in Shanghai PISA 2012 are categorized

as government-independent because they receive less than 50 percent of their core funding from the government Among the private school student popula-tion in Shanghai, 36 percent attend private schools with no funding from the government; an equal percentage attend schools that rely completely on student fees Half of private school students attend schools that receive about 10–30 percent of funding from government; 5.8 percent attend schools that receive approximately 45 percent of their core funding from the government In contrast, funding sources seem to vary among public schools: among public school students in Shanghai, only 60 percent attend schools that do not charge

table 7.4 Comparing Individual and Family Characteristics by program

PISA

variable

Junior secondary

Senior secondary general

Senior secondary vocational Ordinary Model

All programs

Note: See variable descriptions in table 7A.1 PISA = Programme for International Student Assessment

*p < 0.05, **p < 0.01, ***p < 0.001

Trang 7

student fees as a funding source As shown in figure 7.3, panel a, a small

propor-tion of public schools in Shanghai actually receive less than half of their core

funding from the government And 3 percent of the public school student

popu-lation in Shanghai in fact attends public schools that receive more than half of

their core funding from student fees (figure 7.3, panel b)

Admissions policies differ significantly among the four types of schools:

17 percent of junior secondary schools still consider academic performance or

recommendations from feeder schools for admission Academic performance or

recommendations from feeder schools is required for admission to the vast

majority (92 percent) of general secondary schools, but only for 60 percent of

vocational secondary schools This means that the variation in student

perfor-mance across and within senior secondary programs is not only related to school

quality, but also to the admission process that sorts students according to their

academic performance before they enter senior secondary school Considering

that the PISA was administered not long after the 15-year-olds entered senior

secondary programs, the correlations presented in the following sections can be

interpreted both as “what school characteristics predict better student

perfor-mance” and as “what kinds of schools attract better-performing students?”

The main difference between general and vocational secondary schools lies in

teaching resources: the student-to-teacher ratio is as high as 17 in vocational

secondary schools, in contrast with 9 in general secondary schools Moreover, on

average 99 percent of the teachers in general senior secondary schools hold

ter-tiary qualifications, compared with 92 percent in vocational senior secondary

Figure 7.3 Distribution of Students by percentage of Funding from Government versus Student Fees

Public schools Private schools

a Funding from government

Weighted proportion of students

0.60 0.60 0.50 0.40 0.30 0.20 0.10 00.10 0.20 0.30 0.40 0.50

b Funding from student fees

101520 25

0 5

Weighted proportion of students

0.60 0.60 0.50 0.40 0.30 0.20 0.10 00.10 0.20 0.30 0.40 0.50

Trang 8

schools In addition, more creative extracurricular activities are available at senior secondary schools and mixed secondary schools than at junior secondary schools.Curiously, measures of student- and teacher-related factors affecting school climate are lowest in vocational schools but highest in general secondary schools (table 7.5) Given that both measures are based on principals’ reporting (see table 7A.1 for detailed definitions of the two measures), it is likely that, instead

table 7.5 Comparing Characteristics of Different types of Schools

PISA variable

Junior secondary school

Mixed secondary

General high

*p < 0.05, **p < 0.01, ***p < 0.001

Trang 9

of measuring the actual extent of disruption, the two variables indicate how

aware principals are of disruptive student behaviors and teaching practices Thus,

caution should be exercised in interpreting these results

Among general secondary schools, the only statistically significant difference

between ordinary and model or experimental schools lies in the student-to-teacher

ratio and class sizes (table 7.6): model or experimental schools have relatively

table 7.6 Comparing Characteristics of Ordinary versus Model or experimental

Secondary Schools

Organization, competition, and policy

*p < 0.05, ** p < 0.01, ***p < 0.001

Trang 10

higher student-to-teacher ratios (10) and larger class sizes (41) than ordinary secondary schools The greater local demand for model schools in general per-haps explains these differences This variation also seems to indicate that smaller class size and student-to-teacher ratios themselves do not automatically translate

to learning achievement Model or experimental school principals reported higher levels of student-related factors that affect school climate, suggesting that they might be more aware of student disruptive behaviors

estimating Mathematical, reading, and Scientific Literacy

How are different school characteristics associated with students’ mathematical, reading, and scientific literacy? For each program (junior secondary, senior secondary general, and senior secondary vocational), PISA scores are estimated on the three domains (mathematics, science, and read-ing) using school characteristics, controlling for individual and family back-ground characteristics Given that students were sorted into general versus

vocational programs through zhong kao at the end of ninth grade and not

long before PISA was administered, separate regression models for each program are estimated (junior secondary, senior secondary general, and senior secondary vocational) In interpreting the results, we do not intend to draw any causal inferences from the estimates; rather, we aim to characterize schools with better- versus worse-performing students We also emphasize that the relationship can be interpreted both ways: better-quality schools produce better student performance, but they also admit better-performing students to start with

Junior Secondary

After controlling for student and family background characteristics, differences

in junior secondary students’ mathematics, reading, and science scores are ated mainly with public vs private administration of the junior secondary schools: private junior secondary school students, on average, perform better than public school students on all three domains, and the differences are statistically significant for mathematics and reading scores

associ-Measures of teachers and teaching resources do not seem to explain variances

in junior secondary school student performance except that better-performing schools on the reading test are more likely to report shortages of teachers of Chinese Among indicators of school resources, creative extracurricular activities available at school are related to better performances of students across all three domains

Lower-performing junior secondary schools tend to be more autonomous in determining student assessment policies, textbooks, course content, and offerings, whereas the curricula for higher-performing junior secondary schools are deter-mined mainly by regional, local, or national educational authorities The negative association between autonomy in curriculum and performance is statistically significant for mathematics but not for reading or science (table 7.7)

Trang 11

table 7.7 estimates of Mathematical, reading, and Scientific Literacy Using School

Characteristics, Junior Secondary

PISA

variable

Coefficient Standard error Coefficient Standard error Coefficient Standard error

Note: See variable descriptions in table 7A.1 All models control for individual and family background characteristics, as well as

grade-level fixed effects PISA = Programme for International Student Assessment.

*p < 0.05, **p < 0.01, ***p < 0.001

Trang 12

Senior Secondary General

Among measures of school resources, quality of school educational resources is significantly and positively related to reading scores in general senior secondary schools (table 7.8) Echoing the previous finding that model or experimental secondary schools have larger class sizes than ordinary ones, among general secondary school students, a one unit (student) increase in class size is associ-ated with a 1.5 point higher score on mathematics and reading, after control-ling for individual and family background characteristics Among four dimensions of school leadership measures, levels of teacher participation in school leadership are positively and significantly associated with mathematical and reading literacy

After controlling for individual and other school characteristics, students from mixed secondary schools perform significantly worse across all three domains than those from nonmixed general secondary schools Furthermore, ability grouping between mathematics classes is associated with lower performance among secondary school students

Senior Secondary Vocational

For vocational school students, reading scores do not seem to be significantly correlated with school-level characteristics after controlling for individual and family characteristics (table 7.9) Mathematics performance is, on the one hand, correlated with school accountability to parents: students attending schools that face pressure from parents score on average 41 points higher on mathematics On the other hand, vocational schools with lower mathematics scores report signifi-cantly more student-related factors affecting school climate

Science performance is significantly and positively related to several sures of school resources, including quality of physical infrastructure, class size, and availability of extracurricular creative activities at vocational senior second-ary schools

mea-Individual and Family Background Characteristics

To demonstrate the correlation with individual and family background characteristics, school fixed effects models are used to estimate student perfor-mance (table 7.10).2 We find a highly significant correlation between back-ground characteristics and performance across all domains

Girls perform worse on mathematics and science and better on reading compared with boys Wealth is negatively correlated with performance In comparison, more family educational resources and cultural possessions are associated with better performance Similar results are found in other OECD countries, suggesting that on the one hand, family wealth can improve performance by providing more educational and cultural resources, but on the other hand, weakens students’ incentives to learn and reduces the cost of leisure relative to education (Spiezia 2011) Parental education is also positively related to performance Finally, students who have attended at least a year of preschool have a significant advantage across all domains over

Trang 13

Characteristics, Senior Secondary General Students

PISA

variable

*p < 0.05, **p < 0.01, ***p < 0.001

Trang 14

Characteristics, Senior Secondary Vocational

PISA

variable

Individual and family characteristics

*p < 0.05, **p < 0.01, ***p < 0.001

Trang 15

table 7.10 estimates of Mathematical, reading, and Scientific Literacy Using Individual and

household Background Characteristics, Controlling for School Fixed effects

PISA

variable

Note: See variable descriptions in table 7A.1 All models control for fixed effects and grade-level fixed effects

PISA = Programme for International Student Assessment.

*p < 0.05, **p < 0.01, ***p < 0.001

those who have not: the differences range from 16 points in science to as

many as 31 points in mathematics, even after controlling for gender and

other family background characteristics

problem-Solving Skills

Shanghai ranks sixth on overall problem-solving skills on PISA 2012 As

dis-played in figure 7.4, students in Singapore, Korea, and Japan, followed by

stu-dents in Hong Kong SAR, China, and Macao SAR, China, score higher in

problem solving than students in all other participating countries and economies

Disaggregation of data reveals that students in Hong Kong SAR, China; Japan;

Korea; Macao SAR, China; Shanghai; Singapore; and Taiwan, China, perform

strongest on problems that require understanding, formulating, or representing

new knowledge, compared with other types of problems At the same time,

stu-dents in Brazil, Ireland, Korea, and the United States perform strongest on

inter-active problems that require students to uncover some of the information

needed to solve the problem, compared with static problems for which all

infor-mation is disclosed at the outset (OECD 2014)

Estimating Problem-Solving Skills Using School Characteristics

The same set of school-level characteristics is used to estimate problem-solving

scores (Model 1), controlling for student and family background (table 7.11)

The problem-solving assessment of PISA 2012 was designed to focus as

much as possible on cognitive processes and generic skills rather than

domain-specific knowledge (OECD 2014) However, because the same cognitive processes

can also be used in mathematics, science, and reading, problem-solving scores

are positively correlated with the other three domains For students in Shanghai,

as much as 71 percent of the problem-solving score reflects skills that are also

Trang 16

measured in at least one of the three regular assessment domains; 64.8 percent of the variance in problem-solving scores is associated with more than one regular domain, and 5.8 percent of the variance is uniquely associated with mathematics (OECD 2014) Because problem-solving skills are highly correlated with perfor-mance in the mathematics, reading, and science domains, which are related to school-level characteristics, to account for omitted variable bias, mathematics, reading, and science scores are controlled for in the full models (Model 2) used to estimate problem-solving scores

For junior secondary students, after controlling for math, reading, and science scores, problem-solving skills are significantly and positively related to quality of

Figure 7.4 problem Solving, Mean Score, pISa 2012

Hong Kong SAR, China

Russian Federation

Slovak Republic

Turkey Israel HungaryCroatia

Serbia SloveniaSpain

United StatesGermany

Czech RepublicItaly

Sources: Data from OECD 2012, PISA 2012 database (http://pisa2012.acer.edu.au/); OECD 2014

Note: OECD = Organisation for Economic Co-operation and Development; PISA = Programme for International Student Assessment.

Trang 17

PISA variable

Junior secondary Senior secondary general Senior secondary vocational Model 1 Model 2 Model 1 Model 2 Model 1 Model 2 Coefficient

Standard error Coefficient

Standard error

Individual and family background

Định dạng
Số trang	34
Dung lượng	1,04 MB
File đính kèm	LearningOutcomes.rar (665 KB)