Karma al hassan education system quality indicators

The policy-making process in the education sector in developing countries can be all the more complex because it is necessary to analyze the education sector itself, in terms of the impl

Paper Presented to

School-based Reform in Arab Countries Project at AUB

(TAMAM) Sponsored by Arab Thought Foundation

Karma El Hassan, PhD

Associate Professor Director Office of Institutional research & Assessment (OIRA)

III Education quality 4

about the role of the state in shaping the life-chances of its future citizens (Hoffman, Hoffman, Gray, & Daly, 2004.) The quest for higher performance through

educational reform has been a worldwide phenomenon, especially over the last decade, and accordingly, the world witnessed a trend towards restructuring education systems Most countries experienced a change in relationships between different levels of the system, or changes in the role of the state, or changed methods and models of managing the education system (Wallin, 1995) The following section will outline the major factors that have dominated the education reform movement in the last two decades

II Education Reform in Last Two Decades

In recent years, political and educational discussions have increasingly raised questions about the quality of education This interest had its origins in a) emergence

of a widely held belief that education systems and their relative cross-national

performance were a key element in strategies designed to achieve improvement in national economic development in an increasingly globalized and competitive world, and b) public concerns that governmental expenditures were enormous and they needed to be accompanied by higher levels of scrutiny and accountability concerning the quality of education, especially as education budgets are under pressure (Ross & Genevois, 2006)

Reforms that have dominated the education scene in the 1980s, 1990s, and into the 2000s and major forums organized by international agencies also turned their attention towards issues related to the quality of education and the need for

assessment The 1990 Jomtien World Conference on Education and 2000 Dakar World Education Forum both called for a broader view of education beyond a

concentration on increased access These declarations emphasized that in addition to increased participation in education, all nations need to „improve all aspects of the quality of education and ensure excellence so that recognized and measurable leaning outcomes are achieved by all‟ Assessment information may be used to reach a judgment about the adequacy of the performance of an education system or of a part

of it

Accordingly, there has been increased government interest in monitoring and evaluating the quality of education Governments needed objective data to provide evidence through which student learning achievement may be monitored both

nationally and internationally They needed to monitor student performance over time and in a cross national comparative perspective, in order to provide information for assessing how well or how badly education systems are preparing young people for future adult roles as creative, thinking citizens who can sustain themselves and

of national and international dialogue about the importance of the „quality of

education‟ resulted in decisions by many countries to implement programs for

national assessment of educational progress, and to participate in networks that conducted large-scale cross-national educational research studies (Pelgrum, Voogt, & Plomp, 1995) Educators believed that research on the quality of education required

an international focus because variations among countries in terms of educational policies, practices, and traditions provided a natural laboratory for the study of those aspects of educational environment that were likely to have a substantial and

consistent impact upon improved student learning They also argued that national studies of the quality of education offered much more than national studies because „custom and law define what is educationally allowable within a nation, whereas the educational systems beyond one‟s national borders suggest what is educationally possible (Foshay et al, 1962, as cited in Ross & Genevois, 2006) The increasing awareness of the usefulness of these monitoring systems has resulted in a need for indicators that can help in tracing shortcomings and improving educational outcomes The need for information on quality of education has led to many different research initiatives; such as, national assessment studies, international comparative studies, national indicator development projects, etc (Pelgrum, et al, 1995)

cross-International comparisons of education, despite their problems and costs, have one particular advantage They have become important instruments for education policy makers in responding to an increasing demand for greater accountability in the public sector of education Many countries, for example, have set national and local standards for assessing outcomes Though many of these standards are not

internationally comparable, yet they establish important benchmarks Above all, they foster a culture of self-evaluation around outcomes (Bengston, 2004)

The following sections will attempt to provide a conceptual framework for quality education, and will present different models emanating from this conceptual

framework Then, will attempt to provide an explanation of indicators, their uses and types, and finally will describe some of indicator sets currently in use for measuring education quality at system level

III Education Quality

1 Definition

The debate and the views of what constitute education quality are as old as

education itself and they are constantly evolving It is a debate influenced by values, norms and subjective judgments (Bengston, 2004) It is not meaningful to try to arrive

at a tight, single definition of educational quality, given the evolving scope of

education, and the great variety of education systems and value-structures There is

quality as the adequacy or appropriateness of objects or processes for the purposes for which they were intended (Kellaghan & Greaney, 2001) Conventional definitions have focused on inputs and have included literacy, numeracy and life skills, and those have been linked directly to such critical factors as teachers, content, methodologies, curriculum, examination systems, policy, management, and administration

The expanded vision of what is education as articulated by the Jomtien

Conference, the exponential growth of new knowledge, and the emerging educational needs of the new millennium have rendered the traditional meaning of „quality of education‟ obsolete, and a need arose to re-think the concept more comprehensively Most of education systems are national initiatives and entities that were not built for a rapidly developing global knowledge economy, where the production, mediation and use of knowledge increasingly operate in a borderless world (Bengston, 2004) Moreover, global economic competition has brought to the fore the critical

importance of quality of human resources, and the demand for new competencies in today‟s information society The educational system, schools, and individual students were all under increasing pressure to perform (Kellaghan & Greaney, 2001)

Accordingly, and with the advent of knowledge society, educational systems needed to focus on what is learned and how it is learned, and they need be

transformed into systems of life long learning with new definitions of education and learning quality They need to develop concepts, practices and quality standards for a range of different types of knowledge from know-what to know-how, from explicit to tacit knowledge; a broader range of competencies and skills for the 21st century (Bengston, 2004)

Given the diversity of understanding and interpretation of quality evident in the different traditions discussed above, defining quality and developing approaches to monitoring and improving it requires the following:

 broad agreement about the aims and objectives of education and this embodies moral, political, and epistemological issues;

 a framework for the analysis of quality that enables its various dimensions to

be specified;

 an approach to measurement that enables the important variables to be

identified and assessed;

 a framework for improvement that comprehensively covers the interrelated components of the education system and allows opportunities for change and reform to be identified

(UNESCO, 2005)

The following section will outline conceptual model for quality of education and then will describe various understandings of this model by international

organizations

2 Conceptual Framework

A Basic Framework Perspectives on education quality can be clarified on basis

of a conceptual framework that describes education Most frequently used one

is the one that depicts education as a productive system, in which inputs are transferred into outcomes (Kellaghan & Greaney, 2001; Scheerens, 2004;

UNESCO, 2005) This basic model, Figure 1, can be elaborated using following steps:

a Context dimension that provides inputs, constraints, and is a generator of required outputs to be produced

b Outcomes as direct outputs, longer term outcomes and ultimate societal impact;

c Conditions and processes with a hierarchical nature (system, schools, and classroom levels)

Figure 1 Basic Systems Model of Functioning of Education

Within this basic framework, at least six ways in defining quality can be identified by emphasizing certain parts, aspects or relationships from basic framework (Scheerens, 2004)

a Productivity view Success of system is depending on attainment of aspired outcomes/outputs; accordingly output/outcome/impact indicators are predominant or only type of quality indicator that needs to be monitored

b Instrumental effectiveness view Emphasizes context, input and process indicators as their effectiveness is essential for successful outputs, therefore instrumental potential are of vital importance

c Adaptation perspective How to do the right things leading to a critical analysis of educational goals

d Equity perspective When inputs, processes and outcomes are analyzed in their equal or „fair‟ distribution among participants in education with different characteristics

Context

Output

System School Class Levels

e Efficiency perspective Considers highest possible outcomes at lowest possible costs

f The disjointed view Each element of the model is considered „on its own‟ and judges whether it is manifested in an acceptable way or at an acceptable level For ex level of teacher training, class size, etc

Based on this basic education model, several indicator sets at system level were developed like ones used in OECD INES project and the „Sixteen Quality Indicators‟ of the European Commission A slightly different conceptualization of quality education was presented by international organizations like UNESCO and UNICEF that will be described in next section

B UNESCO Framework In its report Learning: The Treasure Within, Report to UNESCO of the International Commission on Education for the Twenty-first Century, the commission saw education throughout life as based upon four pillars:

 Learning to know acknowledges that learners build their own

knowledge daily, combining indigenous and „external‟ elements

 Learning to do focuses on the practical application of what is learned

 Learning to live together addresses the critical skills for a life free

from discrimination, where all have equal opportunity to develop themselves, their families and their communities

 Learning to be emphasizes the skills needed for individuals to develop

their full potential

This conceptualization of education provided an integrated and comprehensive view of learning and, therefore, of what constitutes education quality (Delors et al., 1996) In addition, UNESCO promotes access to good-quality education as a human right and supports a rights-based approach to all educational activities (Pigozzi, 2004) Within this approach, learning is perceived

to be affected at two levels At the level of the learner, education needs to seek

out and acknowledge learners‟ prior knowledge, to recognize formal and informal modes, to practice non-discrimination and to provide a safe and supportive

learning environment At the level of the learning system, a support structure is

needed to implement policies, enact legislation, distribute resources and measure learning outcomes, so as to have the best possible impact on learning for all (UNESCO, 2005)

To meet the above vision of education, the main elements of the education systems and the central dimensions influencing core processes of teaching and learning were characterized as follows:

 learner characteristics dimension;

 contextual dimension;

 enabling inputs dimension;

 teaching and learning dimension;

 outcomes dimension

The framework is comprehensive, in that the quality of education is seen as encompassing access, teaching and learning processes and outcomes in ways that are influenced both by context and by the range and quality of inputs available It emphasizes education‟s relevance to the world, and addresses social and other dimensions of learning According to this perspective, quality of education must recognize the past, be relevant to the present, and have a view to the future It should reflect the dynamic nature of culture and languages, and the value of the individual in relation to the larger context (Pigozzi, 2006) The role of the education system is to create and support learning experience Figure 2 lists the various elements of the model

UNICEF‟s view of quality is quite similar and it strongly emphasizes what might be called desirable dimensions of quality, as identified in the Dakar

Framework Its paper Defining Quality in Education recognizes five dimensions

of quality: learners, environments, content, processes and outcomes, founded on

„the rights of the whole child, and all children, to survival, protection, development and participation‟ (UNICEF, 2000)

Figure 2 UNESCO Framework for Understanding Education Quality

 Economic and labour market conditions in the community

 Socio-cultural and religious factors

 National governance and management strategies

 Philosophical standpoint of teacher and learner

 Peer effects

 Parental support

 Time available for schooling and homework

 National standards

 Public expectations

 Labour market demands

 Teaching and learning materials

 Physical infrastructure and facilities

 Human resources, teachers, principals, inspectors, supervisors, administrators

Having identified the aims and objectives of education and specified the dimensions of its quality, attention should be focused on identifying variables that need to be assessed and measured in order to monitor efficiency of the education system The following sections will describe indicator tools used for this purpose and will present an outline of various indicator sets in use at system level

IV Indicators and Indicator Sets at System Level

1 Indicators

Education indicators are statistics that are useful for planning, management and policy making (OECD, 1991) They may be defined as tools that offer a good description of the education system and provide a sense of its state, in addition to providing the grounds for an analysis of education policy at the national level A system of indicators must function like a control panel, facilitating the identification

of problems and measuring their magnitude Accordingly, the role of indicators could

be defined as:

 describing the present situation,

 quantifying the objectives which have been set,

 providing continuous updates on progress towards certain objectives or,

 providing insights into which factors might have contributed to achieving the results (European Commission, 2002)

For an indicator system to function properly and fulfill its role, it requires a good information system and an education policy and plan Most countries have education databases that are updated regularly There is a problem, however, linked to the presentation and dissemination of available data With few exceptions, data are only published in cumbersome statistical yearbooks that contain excessive raw data and a dearth of analysis Decision-makers need an easier document to read and interpret, one that is more analytical and more relevant (ETF, 2003)

Accordingly, there was a need for indicator documents that report on the functioning of the education system in all its aspects with a small number of relevant indicators, that are simple, easy to read, accessible, defined at a high level of

aggregation putting all data on education into a single, all encompassing set To meet this need, several types of publications containing a range of indicators have recently appeared: UNESCO initiated the first comparative international work, the World Education Report, the OECD has substantially developed this area over the last ten years, Education at a Glance: OECD Indicators, and several publications have also come out on single countries during the same period These have mostly been

produced by Ministries of Education The first of these were L’état de l’école in France and Indicateurs de l’éducation in Quebec (ETF, 2003)

The following sections will present on some of the indicators sets at the system level

2 Indicator sets at system level

A OECD Indicators Education at a Glance is the OECD‟s annual round-up of

data and analysis on education Partly based on economic models, it provides a rich, comparable and up-to-date array of indicators on education systems in the OECD‟s

30 member countries and in a number of partner economies Each edition introduces new countries, indicators and new methodologies, the aim being to over excel both quantitatively and methodologically

These indicators enable educational policy makers and practitioners alike to see their education systems in the light of other countries‟ performances and, together with OECD‟s country policy reviews, are designed to support and review the efforts that governments are making towards policy reform They aim to build a profile of education on which the „prominent specialists agree‟, and they function as an encyclopedia (description of actual situation) and a dictionary (highlighting trends and developments) (Sedel, 2004)

The OECD indicator set is based on basic education model of process-output at the national education system level However, there is increasing recognition that many important features of the development, functioning and impact

context-input-of education systems can only be assessed through an understanding context-input-of learning outcomes and their relationships to inputs and processes at the level of individuals and institutions To account for this, the indicator framework distinguishes between a macro level, two meso-levels and a micro-level of education systems These relate to:

• The education system as a whole;

• The educational institutions and providers of educational services;

• The instructional setting and the learning environment within the institutions;

• The individual participants in education and learning.

The OECD Indicator set provide information on the human and financial resources invested in education, on how education and learning systems operate and evolve, and on the returns to educational investments They are presented within an organizing framework as described in the following matrix (Figure 3) Our main concern in this paper is the education indicators at the system level, i.e level four The various components of the model defined at level of national education system and their respective indicators are presented below:

Context indicators refer to characteristics of the society at large and structural

characteristics of national education systems Examples are:

 Demographics; relative size of school-age population;

 Basic financial and economic context; e.g GDP per capita

 Educational goals and standards by level of education; equitable distribution of university graduates, high completion rate

 Structure of schools in country, as characterized by International Standard Classification of Education (ISCED-97)

Input indicators refer to financial and human resources invested in education

Examples are:

 Expenditure per student,

 Expenditure on Research &Development in education,

 Percentage of total labor force employed in education,

 Pupil teacher ratios per education level,

 Characteristics of the stock of „human resources‟ in terms of age, gender, experience, qualifications and salaries of teachers

Figure 3 OECD Organizing Framework

1 Education and learning outputs and outcomes

2 Policy levers and contexts shaping educational outcomes

3 Antecedents or constraints that contextualize policy

I Individual participants in education and learning

1.I The quality and

distribution of individual educational outcomes

2.I Individual

attitudes, engagement and behaviour

3.I Background

characteristics of the individual learners

II Instructional settings

1.II The quality of

instructional delivery

2.II Pedagogy and

learning practices and classroom climate

3.II Student

learning conditions and teacher working conditions III Providers of

educational services

1.III The output of

educational institutions and institutional performance

2.III School

environment and organization

3.III Characteristics

of the service providers and their communities

IV The education system as a whole

1.IV The overall

performance of the

education system

2.IV System-wide

institutional settings, resource allocations

and policies

3.IV The national

educational, social, economic and demographic contexts Education at a Glance: OECD Indicators 2007

Process indicators are characteristics of learning environment and

organization of schools that are defined at system level Examples are:

 Pattern of centralization/decentralization specified as proportion of decisions taken in a particular domain at a particular administrative level

 Priorities in intended curriculum per education level, expressed as teaching time per subject

 Priorities in education reform agenda, expressed for example as proportion

of total education budget to specific reform programs

 Investments and structural arrangements for system level monitoring and evaluation at a given point in time

Output or outcome indicators refer to statistics on access and participation,

attainment and aggregated data on educational achievement Examples are:

 Participation rates in various education levels

 Progression through the education system, drop out rates,

 Average achievement in basic curricula areas measured at end of primary

or secondary,

 Cross curricular competences

Impact or long-term outcome indicators Changes in other sectors of society

that can be seen as effect of education like

 Impact of education on youth unemployment ,

 Income related to education and training level,

 Delinquency rate per level of educational attainment

The full list of OECD indicators at system level are presented in Appendix The OECD Indicators project main function is to offer data to be interpreted and used for different purposes at different levels of the state and education system Results should not be considered to have meanings of their own but instead to obtain their meaning in the context of the totality of the indicators Should not interpret isolated data, but use indicator data in context and not as a number of isolated facts that have little meaning in themselves (Walin, 1995)

B European Commission (EU)

1 „Sixteen Quality Indicators‟ The EU Sixteen Quality Indicators Report (2000) was developed with the objective of identifying a limited number of key indicators „to assist national evaluation of systems in the area of school standards‟ It adopts the conceptual framework of OECD INES indicator set and includes four indicator categories: Context, input, process, and outcomes/impact Under EU model, indicators included under each category are

 Input: resources and structures

 Process: monitoring of education

 Outcomes: attainment, success, and transition

 Context and Impact indicators were not defined

The complete set includes the following indicators by category:

Indicators on Attainment (outcomes)

Indicators on Success and Transition (outcomes)

8 Drop-out rates

9 Completion of upper secondary education

10 Participation in tertiary education

Indicators on Monitoring of Education (process)

11 evaluation and steering of school education

12 Parents‟ participation

Indicators on resources and structures (input)

13 education and training of teachers

14 participation in pre-primary education

15 number of students per computer

16 educational expenditure per student

2 Fifteen Quality Indicators for Lifelong Learning, EU 2002 After the presentation

of the previous report, the expert group who prepared it were asked to continue the work and to extend the initiative to cover all the strands of education and training encompassed by lifelong learning

“Lifelong learning is seen as encompassing all purposeful learning activity, whether formal or informal, undertaken on an ongoing basis with the aim of improving knowledge, skills and competence” Early in 2001, the working group

agreed upon a limited number of relevant indicator areas The most appropriate

indicators for each of these areas have subsequently been selected

The fifteen quality indicators are contained in the following four areas (A-D):

Area A: Skills, Competencies and Attitudes

1 Literacy

2 Numeracy

3 New Skills in the Learning Society

4 Learning-to-Learn Skills

5 Active Citizenship Cultural and Social Skills

Area B: Access and Participation

6 Access to Lifelong Learning

7 Participation in Lifelong Learning

Area C: Resources for Lifelong Learning

8 Investment in Lifelong Learning

9 Educators and Training

10 ICT in Learning

Area D: Strategies and System Development

11 Strategies of Lifelong Learning

12 Coherence of Supply

13 Counseling and Guidance

14 Accreditation and Certification

15 Quality Assurance Their indicators are listed in Appendix To attain quality lifelong learning, five challenges need to be overcome: The skills, competencies and attitudes challenge, the resource challenge, the challenge of social inclusion, the challenge

of change, and the challenge of data and comparability

C UNESCO‟s indicator sets To meet UNESCO‟s integrated vision of education

quality; several indicator systems were developed that comprise UNESCO‟s conceptions of central dimensions of teaching and learning Among these systems, two will be described the World Education Indicators (WEI) and Education for All (EFA)

1 World Education Indicators (WEI) The main aim of the World Education Indicators (WEI) programme is to establish a comparative perspective on key policy issues to better monitor education systems The WEI programme, funded by World Bank and coordinated by the OECD and UNESCO, aims to address the new information needs as countries shift to more advanced stages

of educational development It has sought to: develop indicator methodologies based upon a common set of policy concerns where cross-national

comparisons add value; review methods and data collection instruments and set the direction for future developmental work and analysis

The publication provides comparable education indicators for 63 countries covering 72% of the world‟s population and consists of five thematic sections which present and interpret leading education indicators for WEI countries They are

Section 1 The outputs of education systems Section 2 Sources and flows of education expenditure Section 3 Levels and uses of education expenditure Section 4 Access to education, participation and progression Section 5 Teachers and the learning environment

Details of the various sections are provided in Appendix

2 Education for All (EFA) Indicators The Education for All movement is a global commitment to provide quality basic education for all children, youth and adults by 2015 It includes six key goal areas and reports following indicator tables:

1 Background statistics

2 Literacy Rates Literate Environment

3 Early childhood care and education (ECCE): care Early childhood care and education (ECCE): education

4 Access to primary education

5 Participation in primary education

6 Internal efficiency: repetition in primary education

7 Internal efficiency: primary education dropouts and completion

8 Participation in secondary and post-secondary non-tertiary education

9 Participation in tertiary education Tertiary education: distribution of students by field of study and female share in each field

10 Teaching staff in pre-primary and primary education Teaching staff in secondary and tertiary education

11 Commitment to education: public spending

12 Trends in basic or proxy indicators to measure EFA goals 1, 2, 3, 4 and 5

As evident from above indicator sets, they are a combination of context, inputs and outcomes dimensions, and quality of education is seen as

encompassing access, teaching and learning processes and outcomes in ways that are influenced by context and by the range and quality of inputs available

D World Bank

World Bank noted that measuring the quality of education can be approached

from two perspectives First, it can be construed as fundamental quality: how

many students have attained the basic skills (however defined) to successfully complete their courses of instruction and productively participate in the national labor market, policy, society, etc.? Second, it can be taken to reflect an education

system‟s production of excellence: how many students from a particular country

have entered into “world-class” research universities; or how many national universities produce “world-class” research or technicians/professionals? (World Bank, 2008, p277) Accordingly, although measurement of quality is elusive, it can be approximated by using different indicators, and World Bank has identified several under the following headings: inputs, access, equity, efficiency, and quality and outcomes

More specifically, indicators used are the following:

A Physical, Human and Financial Capital Inputs

 Pupil-Teacher Ratio in Primary Education

 Pupil-Teacher Ratio in Secondary Education

 Student-Teacher Ratio in Tertiary Education

 Percentage of Trained Teachers in Primary Education

 Percentage of Trained Teachers in Secondary Education

 Public Expenditure in Education as Percent of GDP

 Public Expenditure in Education as Percent of Government Spending

B Access

 Gross Enrollment Rate in Primary Education

 Gross Enrollment Rate in Secondary Education

 Gross Enrollment Rate in Tertiary Education

 Net Enrollment Rate in Primary Education

 Gross Intake Rate to Grade 1

C Equity

 Gender Parity Index of Primary Gross Enrollment Rate

 Gender Parity Index of Secondary Gross Enrollment Rate

 Gender Parity Index of Tertiary Gross Enrollment Rate

 Gender Parity Index of Gross Intake Rate

 Gender Parity Index of Repetition Rate in Primary Education

D Efficiency

 Survival Rate to Grade 5

 Primary Completion Rate,

 Repetition Rate in Primary Education,

 Repetition Rate in Secondary Education,

 Dropout Rate in Primary Education,

 Dropout Rate in Secondary, Lower Secondary, and Upper Secondary Education

 Private Enrollment Share in Primary Education

 Private Enrollment Share in Secondary Education

 Private Enrollment Share in Tertiary Education

E Quality and Outcomes

 TIMSS Score in Math of 8th Grade, 1995, 1999, and 2003

 TIMSS Score in Science of 8th Grade, 1995, 1999, and 2003

 Adult Literacy Rate (aged 15 and older)

 Average Years of Schooling of Adults

V Conclusion and Future Directions Although the review of above international indicator systems revealed that they use different classifications, yet an eclectic interpretation of education quality is predominating, and various perspectives on quality previously discussed are sort of represented in indicator sets The productivity and efficiency views on educational quality are evident in indicator sets with process indicators being related to outputs, to provide an impression on which factors work in a particular country and across countries

The resources/inputs-activities/process-outcomes/impact seems best to facilitate analysis and to provide an explanatory model of education The three components are linked by close, multidirectional relationships and are affected by the characteristics

of the context or sociodemographic environment that interacts with each of the components

In addition, a shift in major categories of indicators over time can be noted from a relatively heavy emphasis on financial indicators and attained levels of education to a completion of these indicators with indicators on educational environments and processes as well as output indicators based on international comparative assessment studies (Scheerens & Hendriks, 2004) Institutional context is emerging as an important variable in determining the quality of education systems According to

Hoffman et al., 2004, certain „incentive creating‟ institutional factors explain 75% of cross-country variation in mathematics achievement, and therefore calls for

improving institutional policies as they are more effective in improving quality than revising resource policies

Even though the current indicator sets have become quite comprehensive, and there are clear signs of their use in policy-debates, there are still some important aspects in which their impact on educational policy and practice could be strengthened (Scheerens & Hendriks, 2004).There is a need to further develop the indicator sets in the following directions:

1 Their ways of reporting need to become more theme-based, more synthetic, and more geared to policy issues

2 More qualitative indicators and descriptive micro level information on goals and applied methods of education indicators need to be developed for use by policy makers, as these are not offered by current indicator systems

3 Process indicators need to be emphasized and to be related to outcomes data

4 Institutional context of education systems needs to be included in indicator sets,

as it has been effective in improving quality At its center are a nexus of cutting relationships pertaining to the relative sizes of the public and private sectors, the financial bases, governance structures, the „locus of control, and the influence of parents and community (Hoffman et al., 2004)

inter-5 Micro-level information collected at lower aggregation levels: school, teacher/classroom and the student levels, need to be included This will make use

of educational and organizational literature on school effectiveness and will yield

a more extensive set of indicators, defined at multiple levels of education systems (Scheerens, 2004) Education at a Glance has contained since 1996 information

on process indicators of school functioning that have a clear association with knowledge base on teaching effectiveness, these indicators are based on school and student level surveys and are thus based on micro level information This is also the case with respect to PISA; however the EU „key data‟ lacks this perspective

6 International indicator work should be made more available to schools and teachers and feedback should be provided to lower levels in the system

7 The importance of national cultural contexts in the meaning and interpretation of international data should be emphasized There is a concern for more justice to the intricacies of the national context, and several critics have voiced caution that international studies may contribute to the standardization of cognitive skills informed by a set of culturally exclusive principles and knowledge As with all aspects of development, a balance should be struck between ensuring the relevance of education to the socio-cultural realities of learners, to their aspirations, and to the wellbeing of the nation (UNESCO EFA, 2005)

8 Some commonly used indicators sets need to be reconsidered to take into account evolving understanding of the various dimensions of quality and the recent developments in the rethinking of human capital It is not only defined by educational attainment but by a wider set of human capital that include the ability

and motivation to learn, effective job search, and personal characteristics (Bengston, 2004)

Indicators were developed in response to the changes brought about by globalization and the new rules of the game to which governments are expected to adhere to and education systems to adjust There has been a growth in concern with what students learn because of their educational experiences While, until recently, the focus in assessing quality in education was on inputs, the question posed today by policy-makers is „are students acquiring appropriate knowledge, skills, behaviors and attitudes‟? The assumption is that it is possible to measure aspects of education, in particular essential aspects, with few indicators that provide information on system‟s quality Such an idea supposes the existence of a point of reference, a standard or a norm on which one can base a judgment of value (Scheerens, 1995) The evaluation

of quality education be it a national or/and supranational level supposes a common vision of problems and solutions amongst the participants Research supports this supposition by showing that schooling could be more similar than different across cultures and countries (Brophy, IAE, 2000, as cited in Hoffman et al 2004) In addition, several indications show that some principles of schooling are applicable universally, while others are much more sensitive to local and cultural variation (Hoffman et al 2004) No matter whether these problems and solutions are cross-country or within country, rational decision-making necessitates the availability of objective information The development of an indicators system, which is updated regularly, is essential for that purpose Indicators are simple to implement, not very costly, and only political decision is needed to maximize the benefits of their use

References

Bengston, J (2004) The quality of education at the beginning of the 21st Century

Paper prepared for EFA Global Monitoring Report 2005

Delors, J (1996) Learning: The Treasure Within Paris: UNESCO

European Commission (May 2000) Sixteen quality indicators European report on quality of school education Brussels: European Commission

European Commission (June 2002) Quality indicators of lifelong learning: Fifteen quality indicators Brussels: European Commission

Hoffman, Hoffman, Gray, & Daly (eds.), 2004 Institutional context of education

systems in Europe Netherlands: Kluwer Academic Publishers

Kellaghan, T., & Greaney, V (2001) Using assessment to improve the quality of education Paris: IIEP, UNESCO

OECD (2007) Education at a Glance Paris: OECD

Pelgrum, W., Voogt, J., & Plomp, T (1995) Curriculum indicators in international

comparative research In Measuring of school quality Paris: OECD

Pigozzi, M.J (2006) What is the quality of education? UNESCO Perspective Paris:

IIEP, UNESCO

Ross, K & Genevois, I.J (eds.) (2006) Cross-national studies of the quality of education: Planning their design and managing their impact Paris: IIEP,

UNESCO

Sauvageot, C & Bella, N. (April 2003) Educational Indicators and Policies:

A Practical Guide.Torino Italy: European Training Foundation

Scheerens, J (2004) Perspectives on education quality, education indicators and

benchmarking European Educational Research Journal, 3(1), 115-138

Scheerens, J (1995) Internationally comparable indicators of educational programs

and processes: Identification, measurement and interpretation In Measuring of school quality Paris: OECD

Scheerens, J & Hendricks, M (2004) Benchmarking the quality of education

European Educational Research Journal, 3(1), 101-114

Sedel, J (2004) Three sets of indicators on education: Education at a Glance (OECD), Key Data on education (European Union), The State of Education

(French Ministry of Education) Elements of comparison and analysis European Educational Research Journal, 3(1), 139-176

UNESCO (2005) Understanding education quality EFA Global Monitoring Report

Paris: UNESCO

UNESCO Institute for Statistics (UIS) (2006) World Education Indicators

Montreal: UNESCO

UNICEF (2000) Defining Quality in Education New York: UNICEF

Wallin, E (1995) Interpretation and use of education indicators In Measuring the quality of schools Paris: OECD

World Bank (2008) The road not traveled education reform in the Middle East and North Africa Washington DC: World Bank.

Appendix: Examples of International Indicator Sets

A1.4 Fields of education (2004) A1.5 Ratio of 25-to-34-year-olds with ISCED 5A and 30-to-39-year-olds with ISCED 6 levels of education to 55 -to-64-year-olds with ISCED 5 A and 6 levels of education, by fields of

education

A2 How many students finish secondary education?

A2.1 Upper secondary graduation rates (2005) A2.2 Trends in graduation rates at upper secondary level (1995-2005) A2.3 Post-secondary non-tertiary graduation rates (2005)

A3 How many students finish tertiary education?

A3.1 Graduation rates in tertiary education (2005) A3.2 Trends in tertiary graduation rates (1995-2005) A3.3 Percentage of tertiary graduates, by field of education (2005) A3.4 Science graduates, by gender (2005)

A3.5 Relationship between motivation in mathematics at 15 years old (PISA 2003) and type A graduation rates, by gender

tertiary-A3.6 Survival rates in tertiary education (2004)

A 4 what are students‟ expectations for education?

A4.1a Percentage of students expecting to complete different levels of education A4.2a Percentage of students expecting to complete ISCED levels 5A or 6 by mathematics performance level (2003)

A4.3a Percentage of students expecting to complete ISCED levels 5A or 6 by gender A4.4 Odds ratios that students expect to complete ISCED levels 5A or 6 by socio- economic status

A4.5 Odds ratios that students expect to complete ISCED levels 5A or 6 by immigrant status

A5 What are students‟ attitudes towards mathematics?

A5.1 Means on students‟ attitudes towards mathematics, approaches to learning, and related indices (2003)

school-A5.2a Relationship between students‟ attitudes towards mathematics and mathematics performance (2003)

A5.2b Relationship between students‟ approaches to learning and mathematics performance A5.2c Relationship between school-related indices and mathematics performance

A6 What is the impact of immigrant background on student performance?

A6.1a Differences in mathematics performance, by immigrant status (2003) A6.2a Percentage of native students at each level of proficiency on the mathematics scale A6.2b Percentage of second-generation students at each level of proficiency on the mathematics scale (2003)

A6.2c Percentage of first-generation students at each level of proficiency on the mathematics scale

A6.3 Index of instrumental motivation in mathematics and student performance on the mathematics scale (2003)

A7 Does the socio-economic status of their parents affect students‟ participation in higher education

A8 How does participation in education affect participation in the labor market?

A8.1a Employment rates and educational attainment, by gender (2005) A8.2a Unemployment rates and educational attainment, by gender (2005)

A8.3a Trends in employment rates, by educational attainment (1991-2005) A8.4a Trends in unemployment rates by educational attainment (1991-2005)

A9 what are the economic benefits of education

A9.1a Relative earnings of the population with income from employment (2005 or latest available year)

A9.1b Differences in earnings between females and males (2005 or latest available year) A9.2a Trends in relative earnings: adult population (1997-2005)

A9.3 Trends in differences in earnings between females and males (1997-2005) A9.4a Distribution of the 25-to-64-year-old population by level of earnings and educational attainment (2005 or latest available year)

A9.5 Private internal rates of return for an individual obtaining.n upper secondary or secondary non-tertiary education ISCED ¾

A9.6 Private internal rates of return for an individual obtaining a university-level degree, ISCED 5/6 (2003)

A9.7 Public internal rates of return for an individual obtaining an upper secondary or post- secondary non-tertiary education ISCED 3/4 (2003)

A9.8 Public internal rates of return for an individual obtaining a university-level degree, ISCED

B1.3a Cumulative expenditure on educational institutions per student for all services over the theoretical duration of primary and secondary studies (2004)

B1.3b Cumulative expenditure on educational institutions per student for all services over the average duration of tertiary studies (2004)

B1.4 Annual expenditure on educational institutions per student for all services relative to GDP per capita (2004)

B1.5 Change in expenditure on educational institutions for all services per student relative to different factors, by level of education (1995, 2004)

B2 What proportion of national wealth is spent on education?

B2.1 Expenditure on educational institutions as a percentage of GDP by levels of education (1995, 2000, 2004)

B2.2 Expenditure on educational institutions as a percentage of GDP by level of education

B2.3 Change in expenditure on educational institutions (1995, 2000, 2001, 2002, 2003, 2004) B2.4 Expenditure on educational institutions as a percentage of GDP by source of fund and level of education (2004)

B3 how much public and private investment is there in education?

B3.1 Relative proportions of public and private expenditure on educational institutions for all levels of education (1995, 2004)

B3.2a Relative proportions of public and private expenditure on educational institutions, as a percentage, by level of education (1995, 2004)

B3.2b Relative proportions of public and private expenditure on educational institutions, as a percentage, for tertiary education (1995, 2004)

B3.3 Trends in relative proportions of public expenditure on educational institutions and index of change between 1995 and 2004 (1995=100 constant prices), for tertiary education (1995, 2000,

2001, 2002, 2003, 2004)

B4 What is the total public spending on education?

B4.1 Total public expenditure on education (1995, 2004) B4.2 Distribution of total public expenditure on education (2004)

B5 How much do tertiary students pay and what public subsidies do they receive?

B5.1a Estimated annual average tuition fees charged by tertiary-type A educational institutions for national students (academic year 2004-2005)

B5.1b Distribution of financial aid to students in tertiary-type A education.(academic year 2005)

2004-B5.1c Financial support to students through public loans in tertiary-type A education (academic year 2004-2005)

B5.2 Public subsidies for households and other private entities as a percentage of total public expenditure on education and GDP for tertiary education (2004)

B6 On what resources and services is education funding spent?

B6.1 Expenditure on institutions by service category as a percentage of GDP (2004) B6.2 Expenditure on educational institutions by resource category and level of education (2004)

B7 How efficiently are resources used in education?

B7.1 Estimates of technical efficiency for primary and lower secondary public sector education

C Access to Education, Participation and Progression C1 How prevalent are vocational programmes?

C1.1 Upper secondary enrolment patterns (2005) C1.2 Annual expenditure on educational institutions per student for all services, by type of programme (2004)

C1.3 Performance of 15-year-old students on the PISA mathematics scale by programme orientation (2003)

C2 Who participates in education?

C2.1 Enrolment rates, by age (2005) C2.2 Trends in enrolment rates (1995-2005 ) C2.3 Transition characteristics from age 15 to 20, by level of education (2005) C2.4 Entry rates to tertiary education and age distribution of new entrants (2005) C2.5 Trends in entry rates at the tertiary level (1995-2005)

C2.6 Students in tertiary education by type of institution or mode of study (2005)

C3 Who studies abroad and where?

C3.1 Student mobility and foreign students in tertiary education (2000, 2005 C3.2 Distribution of international and foreign students in tertiary education by country of origin (2005)

C3.3 Citizens studying abroad in tertiary education, by country of destination (2005) C3.4 Distribution of international and foreign students in tertiary education by level and type of tertiary education (2005

C3.5 Distribution of international and foreign students in tertiary education by field of education (2005)

C3.6 Trends in the number of foreign students enrolled outside their country of origin (2000 to 2005)

C3.7 Percentage of tertiary qualifications awarded to international and foreign students, by type of tertiary education (2005)

C4 How successful are students in moving from education to work?

C4.1a Expected years in education and not in education for 15-to-29-year-olds (2005) C4.2a Percentage of the youth population in education and not in education (2005) C4.3 Percentage of the cohort population not in education and unemployed (2005) C4.4a Trends in the percentage of the youth population in education and not in education (1995- 2005)

C5 Do adults participate in training and education at work?

C5.1a Participation rate and expected number of hours in non-formal job-related education and training, by level of educational attainment (2003)

C5.1b Expected number of hours in non-formal job-related education and training by age group and labor force status (2003)

C5.1c Expected number of hours in non-formal job-related education and training, by level of educational attainment (2003)

D The Learning Environment and Organization of SchoolsD1 How much time do students spend in the classroom?

D1.1 Compulsory and intended instruction time in public institutions (2005) D1.2a Instruction time per subject as a percentage of total compulsory instruction time for 9-to- 11-year-olds (2005)

D1.2b Instruction time per subject as a percentage of total compulsory instruction time for 14-year-olds (2005)

12-to-D2 What is the student-teacher ratio and how big are classes?

D2.1 Average class size, by type of institution and level of education (2005) D2.2 Ratio of students to teaching staff in educational institutions (2005) D2.3 Ratio of students to teaching staff, by type of institution (2005)

D3 How much are teachers paid

D3.1 Teachers‟ salaries (2005) D3.2 Change in teachers‟ salaries (1996 and 2005) D3.3a Adjustments to base salary for teachers in public institutions (2005) D3.4 Contractual arrangements of teachers (2005)

D4 How much time do teachers spend teaching

D4.1 Organization of teachers‟ working time (2005)

D5 How do education systems monitor school performance?

D5.1 Evaluation of public schools at lower secondary education (2005) D5.2 Use of information from school evaluation and accountability of public schools (lower secondary education, 2005)

II European Commission Quality Indicators of Life Long Learning Area A: Skills, Competencies and Attitudes

3 New Skills for the Learning Society

Percentage of students per country below the score of 400 points on the PISA scientific literacy scale

4 Learning-to-learn Skills

Percentage of students per country in the lower 25% of overall performance on the PISA

5 Active Citizenship, Cultural and Social Skills

Qualitative, Civic knowledge and interpretative skills (IEA), Civil knowledge, civic engagement and civic attitudes across countries (IEA)

Area B: Access and Participation

6 Access to Lifelong Learning

7 Participation in Lifelong Learning

Participation in education and training of those aged 25 to 64

Area C: Resources for lifelong learning

8 Investment in Lifelong Learning

Total public expenditure on education as a percentage of GDP

9 Educators and Learning

Percentage of teachers having received education and training during the previous four weeks

10 ICT in Learning

Percentage of households who have Internet access at home

Area D: Strategies and System Development

11 Strategies of Lifelong Learning

Member States‟ positions on developing lifelong learning strategies

12 Coherence of Supply

13 Counseling and Guidance

14 Accreditation and Certification

15 Quality Assurance.

Note: Some areas do not yet have measurable indicators like 6, 12-15

III UNESCO World Education Indicators (WEI), 2006

Section 1 The outputs of education systems

a Educational attainment of the adult population

b Educational attainment by age group

c Gender differences in educational attainment

d Relative size of school-age population

e Upper secondary graduation ratios

f Graduation ratios in tertiary education

g Female graduates in tertiary education Section 2 Sources and flows of education expenditure

a Total education expenditure as a share of GDP

b Distribution of public and private expenditure on education

c Public expenditure on education as a percentage of total public spending

d Public funding mechanisms Section 3 Levels and uses of education expenditure

a Educational expenditure per student

b Educational expenditure per student relative to GDP per capita

c Differences in expenditure per student by education level

d Use of funds by level of education Section 4 Access to education, participation and progression

a Pre-primary education expectancy

b Overall education expectancy

c Tertiary education expectancy

d How universal is education provision?

e Primary and secondary grade repetition

f Secondary and tertiary entry ratios

g Patterns of upper secondary enrolment

h Female participation in education Section 5 Teachers and the learning environment

a Enrolment in public and private schools

b Enrolment in public and private tertiary institutions

c Pupil-teacher ratios

d Average class size

e Statutory instructional time for students

f Teaching hours in public schools

g Age distribution of teachers

h Teacher salary scales

Discussion Paper No.170

Education Indicators to Examine the Policy-Making Process in the Education Sector

NAGOYA UNIVERSITY NAGOYA 464-8601, JAPAN

〒464-8601 名古屋市千種区不老町

名古屋大学大学院国際開発研究科

Education Indicators to Examine the Policy-Making Process in the Education

Sector of Developing Countries

1 Introduction

Evaluation of past policies constitutes an indispensable part of public policy formulation That this is also true in the case of educational policies seems quite obvious In reality, however, evaluation-based policy-making has not been sufficiently practiced in many countries This is

For these countries, the utilization of policy evaluation results in the process of educational policy-making poses a great challenge

The policy-making process in the education sector in developing countries can be all the more complex because it is necessary to analyze the education sector itself, in terms of the implementation

of educational policies, levels of student achievements and so on, as well as educational development aid provided by developed countries (donor countries) and international agencies In other words, evaluation must concern both the practical efficacy of educational policies in developing countries and the effectiveness of educational development aid to these countries

In view of the need to ameliorate this situation, this paper discusses education indicators that have been developed in connection with educational policy-making in developing countries, as well

as how they have been, and can be or should be, utilized For these purposes, the EFA Fast-Track Initiative (FTI) is taken up as a case to examine how such indicators can be applied in connection with educational development assistance to developing countries

2 Educational policies and their evaluation in developing countries

Educational reforms are undertaken essentially to improve the system, administration, content and methodology of the present situation of education in terms of access, equity, quality, relevance, efficiency and cost/finance (Buchert, 1998; Williams and Cummings, 2005) For many developing countries confronted with serious problems in all of these aspects, it is not an easy task to identify priority issues to be tackled, translate them into policies and then into actions that can be practiced in the actual educational settings

In translating the objectives of educational reforms into national educational policies, many developing countries generally set policy goals from three different standpoints (or interests) each

standpoint that emphasizes human rights can be traced back to such international agreements as the Universal Declaration of Human Rights (1946) and the Convention on the Rights of the Child (1989), which proclaim the ideal of guaranteeing equal access to education to all, regardless of their gender, age, race and ethnicity Educational reforms from this standpoint often give priority to increasing school enrollment and diminishing gender disparity The economic growth-oriented standpoint is founded on the idea that training quality human resources is vital for national or social economic development From this standpoint, the effectiveness of education as investment is of primary importance, and evaluation mainly concerns the internal and external efficiencies of the education sector From this standpoint, education (particularly school education) is assessed in terms of productivity, based on input-output analysis Educational reforms from the standpoint of social integration aim at nurturing a national identity and citizenship among the nation’s people through education In developing countries that are often multicultural, multiethnic and/or multilingual, social integration is expected to be achieved as a result of the diffusion of education From this standpoint, access to and equity in the opportunity for education and the relevance of educational content are inevitable foci of attention

These three standpoints notwithstanding, many developing countries experiencing great difficulty with economic progress have an undeniable tendency to emphasize the training of human

Samoff, 1990)3) Furthermore, even with an optimal allocation of resources realized in public policy-making and implementation including educational policies, the tradeoff between efficiency and equity tends to generate inconsistencies and conflict in the definition of scope of priority investment in the education sector, a major cause of confrontation between stakeholders with respect

to educational policies (Stiglitz 1998) As well, while it is essentially important to pay attention to human rights and social integration in countries and regions troubled with ethnic or religious disputes, it should not be forgotten that poverty and other economic problems often underlie such disputes Therefore, taking into consideration the way various factors are intricately intertwined in reality, objectives of educational reforms and actual policies in developing countries cannot be adequately analyzed without a multifaceted standpoint and reasoning framework (Riddell, 1999a) Likewise, educational policy-making cannot be pursued from only one of the three standpoints cited above Rather, they should be adopted in combination in a ratio that is optimal to the political, economic, social and cultural contexts of each country concerned The concept of Education for All (EFA) adopted in the World Conference on Education for All, held in Jomatien, Thailand in 1990, encompasses those differing interests, while confirming the importance of diffusion of basic education in developing countries as a major challenge for the entire international

education as areas of investment in the pursuit of educational reforms in developing countries At the same time, the importance of micro-level (school or community-based) educational reforms is being recognized as concepts such as school effectiveness and school improvement are gradually taking root and attracting growing interest in developing countries Meanwhile, administrative and financial authority in education is increasingly actively delegated from national to local government in many developing countries as they undergo decentralization under the influence of ideological trends of neo-liberalism (Hirosato and Kitamura, forthcoming)

Under such circumstances, then, how should educational policies be formulated, so that they can comprehensively cover various interests from macro- to micro-levels as they are expected? It is generally accepted that the educational policy-making process is comprised of three main tasks: analysis of importance and objectives, data analysis and estimation, and prediction In each of them, the clarification of importance of policies and objectives they represent and the analysis of past and present data and situations are indispensable (Davis, 1990; Ross and Mählch, 1990) From such analysis, policy options must be derived from the standpoints of feasibility, affordability and desirability, and then compared and analyzed before final policies are formulated (Haddad and

realistic educational policies in the future Many criticize, however, that these results are not always appropriately utilized as input into educational policy formulation (Riddell, 1999b)

Evaluation of the education sector involves various stakeholders As mentioned above, in developing countries, it concerns the practical efficacy of educational policies in developing countries and the effectiveness of the educational development aid that the international donor community provides to them Such evaluation is conducted by various stakeholders involved in educational policy-making at several different levels, including students and parents as direct beneficiaries of education at the most micro level; teachers and school supervisors (school directors)

at the school level; educational administrators at the local governmental (district/municipal), the prefectural/provincial level, and the national level often represented by the Ministry of Education at the most macro level; and officers in charge of educational programs in international agencies and

Since these stakeholders represent different positions and interests with respect to educational reforms, they inevitably have differing criteria for evaluating the education sector, although they share the common objective of improving the educational situation in the country Realistically speaking, it is extremely difficult to realize evaluation in a manner that can satisfy all of these stakeholders’ criteria Instead, an approach that can cover their greatest common denominator should

be sought out To do this, it is indispensable to utilize education indicators based on a range of statistical data considered to be fundamentally objective In view of this, the subsequent sections focus on education indicators that have been developed to analyze the educational situation in developing countries, and how they have been utilized

3 Development of education indicators by international agencies

In evaluating the education sector of a given country, optimally objective evaluation criteria can be established by placing the country’s educational situation in relative terms through comparison with that of other countries Marc-Antoine Julien de Paris of France, the founding father of Comparative Education, was the first to propose that surveys be undertaken based on a comparative analysis table, according to which information on education in several countries (in terms of system, objectives, methodology, content, etc.) is systematically collected, so as to effectuate comparative compilation and analysis of their educational situations Julien also proposed that a special educational committee

be established so that respective countries would be able to conduct educational reforms based on such research results With this proposal, he was indeed predicting the present-day situation: collection, compilation and analysis of information on education around the world led by

of educational situations as practiced today This section provides an overview of a range of indicators that have been developed mainly by international agencies thus far for international comparison of education

The United Nations Educational, Scientific, and Cultural Organization (UNESCO) and the Organization for Economic Cooperation and Development (OECD) are the most notable international bodies that collect and analyze international statistics concerning education They often work in collaboration through their respective specialized institutions or divisions UNESCO, in cooperation with the ministries of education of countries around the world, gathers statistical data concerning literacy levels, school enrollments at different levels, numbers of graduates and teachers,

reliable and up-to-date statistical data required for policy analysis by national governments and international organizations, UNESCO established the UNESCO Institute for Statistics (UIS) in

data-based (and evidence-based) policy formulation in many public sectors including the education sector (UNESCO, 2002) OECD also collects statistical data reflecting the educational situations of its member states (as well as data on non-member countries, depending on subjects) Data collected concern the distribution of educational levels, educational expenditure, opportunities for education, numbers of students, standard school hours, post-graduation situations such as enrollment in advanced studies and employment, Program for International Student Assessment (PISA) results,

The UIS and OECD have jointly developed the World Educational Indicator (WEI) to be used

in public policy formulation, with the participation of experts from 19 middle-income countries

countries Since its development program commenced in 1997, large numbers of surveys have been conducted in the participating countries concerning the allocation of financial resources to organizations charged with primary education and at different levels of educational policy making

One survey conducted in 2002 (UNESCO-UIS/OECD, 2003) elucidated the effectiveness of education as economic investment, confirming that the diffusion of education contributed to economic growth not only at the national level but also the individual level It demonstrated the general tendency that the level of education was proportional to the rate of employment and job security and that higher qualifications resulted in higher income More specifically, in Indonesia, the

distributed resources in human resources can expand knowledge, an important economic asset, leading to the country’s economic growth A 2007 survey (UNESCO-UIS, 2007) showed that the average number of years of school enrollment of children in the countries participating in the WEI development was about 14 years, almost 4 years shorter than that of the OECD countries, indicating the persistent insufficiency of opportunities to enroll in upper-secondary education and higher education in those middle-income countries

Through annual international comparison of education indicators, these survey results allow clarification of policy challenges of respective countries studied, with indicators as measures of improvement in the educational situation Since primary education is basically widespread in the countries participating in the WEI development, the survey results usually point to the need for greater access to secondary and higher education and qualitative improvement in education at all levels as important challenges

Following the international agreement on EFA goals, EFA 2000 Assessment was conducted

in the late 1990s in order to review the status of basic education in countries around the world On

monitoring and evaluating the status of diffusion of basic education in respective countries, as well

as for international comparison of actions taken by different countries to promote EFA Of the 18 EFA Indicators, 13 overlap those used for above-mentioned educational statistical data annually collected by UNESCO This indicates that a range of education indicators including the WEI and those developed by UNESCO and OECD are in active use for the monitoring and evaluation of EFA progress

In the EFA Global Monitoring Report published by UNESCO since 2002, the EFA

Development Index (EDI) is adopted for measuring the degree of EFA achievement in respective countries The EFA-EDI is obtained by measuring a given country’s degree of achievement in the four areas of the EFA goals (universal primary education, adult literacy, quality of education, and gender equity) in percentages, converting them into indices between 0 and 1 and calculating their average The closer the country’s average index is to 1, the higher its degree of EFA achievement is

indicating their order of superiority in terms of achieving the EFA goals This ranking is inspired by

that of the Human Development Index (HDI), developed for the Human Development Report

published annually by the United Nations Development Programme (UNDP) In the most recent

EFA Global Monitoring Report 2008 (UNESCO 2007), 129 countries including developed countries

were ranked and classified into high-EDI countries (51 countries with EDI at 0.950 or higher),

Norway, which was ranked first (0.950), with Chad, which was ranked 129th (0.409), because of the enormous gaps in the economic and social situations between the two countries Meanwhile, the fact that countries classified into the low-EDI group (17 in sub-Saharan Africa, 4 in South Asia, 3 in Arabic States, and 1 in Southeast Asia) are mostly situated in sub-Saharan Africa followed by South Asian countries, where large percentages of the population are economically and socially disadvantaged, clearly reflects the current status of diffusion of basic education in the international community At the same time, it should be noted that both EFA Indicators and EFA-EDI are based

on national-level data and not micro-level data which can reflect regional differences within a country Therefore, it is necessary to develop indicators for international comparison that reflect micro-level data as well To this end, items in school surveys already conducted in countries around the world should be redesigned so as to be linked with the EFA Indicators and EFA-EDI

Those education indicators mainly developed by international agencies can be utilized in educational reforms in developing countries from the three standpoints mentioned above (each emphasizing human rights, economic growth, and social integration) For educational reforms from the human rights-oriented standpoint, which emphasizes equity and justice through the diffusion of education, indicators relating to enrollment and gender disparity are particularly important The economic growth-oriented standpoint attaches importance to indicators relating to the completion and content of education since the quantity and quality of labor force for future labor market are major concerns in educational reforms From the standpoint of social integration, indicators relating

to the guarantee of access to opportunities for education to all the people constitute the foundation of educational reforms, while equity and the relevance of curriculum should also be closely examined Table 1 shows these indicators classified by different areas of objectives of educational reforms Those listed in this table are generally-utilized indicators, and do not represent all the indicators that should be utilized in the actual evaluation of educational reforms of a given country, which require multifaceted examination from various angles

Countries have been collecting and analyzing educational statistical data by using these education indicators developed by UNESCO and OECD and supplementing them with their own country-based indicators In developing countries in particular, these education indicators are indispensable for policy evaluation, necessary for conducting educational reforms and applying for aid from the international donor community (developed countries and international agencies) assisting educational reforms In the subsequent section, the EFA Fast-Track Initiative, which is being actively introduced into low-income countries, is taken up as a case of international assistance

Table 1 Education indicators classified by areas of educational reforms Areas of Education

Reforms

Education Indicators

drop-out/repetition rates, test scores of students, school inputs/facilities*

pupil/teacher ratio, pupil/classroom ratio, quality of teaching/learning process*, adequacy of teaching content* Relevance of

Curriculum

employment/unemployment rate, graduate tracer studies (usually for higher or technical education)*, admission of university graduates to graduate studies abroad*, labor market feedback on job opportunities of graduates by field*

completion and graduate rates, pupil/teacher ratio, pupil/classroom ratio

expenditure as % of GDP or government budget, public versus private costs

Note: These indicators may not be quantified and would require more qualitative analysis

Source: Table prepared by the author with reference to UNESCO Nairobi Cluster (2006)

4 Utilization of education indicators in assistance for developing countries: a case of the EFA Fast-Track Initiative

Even after the World Education Forum held in Dakar, Senegal in 2000, the diffusion of basic education in developing countries did not progress as well as expected In this situation, developing countries and civil society organizations (NGOs, teachers’ unions, foundations, religious organizations, etc) started raising their voices, asking donor countries and international agencies to

countries, the EFA Fast-Track Initiative (EFA-FTI), whose secretariat was also placed within the World Bank This international initiative involves providing intensive financial and technical assistance to the education sector of selected low-income countries, so as to enable them to get on the “fast track” toward the goal of universal primary education, considered particularly important among the EFA goals (World Bank 2002a)

In the background of the movement leading to the formation of the EFA-FTI was a series

of international conferences held since 2000, in which major developed countries and international

at the International Conference on Financing for Development held in Monterrey, Mexico in 2002, the United States, the European Union and other major donor countries pledged an increase in their official development assistances on the condition that developing countries commit themselves to appropriate utilization of aid This did not mean, needless to say, that all developing countries would

be automatically entitled to abundant funds Effective and efficient utilization of limited resources still remained an essential requirement Accordingly, it was decided within the framework of the EFA-FTI that priority would be given to developing countries that had already demonstrated an active political commitment to the diffusion of basic education and such countries would be selected

The EFA-FTI, thus adopted for the goal of effective and efficient utilization of development aid funds, is characterized by its Indicative Framework with numerical targets concerning the diffusion of primary education as benchmarks These benchmarks are indicators calculated on the basis of achievements made by high-performance developing countries in the area of basic education, including the realization of universal primary education (World Bank 2002a) The benchmarks, set

up especially for low-income countries that require extensive improvement in the education sector, serve as norms that direct the national governmental educational services and financial reforms in the education sector of developing countries Therefore, the numerical targets presented in the table are not absolute standards and may be applied flexibly in line with the context of respective

As recent years have seen a growing interest in qualitative improvement in educational development and reforms in developing countries, the EFA-FTI clearly reflects interest in students’ learning process and aims at qualitative improvement in student academic achievement and school quality on the foundation of partnerships with various actors including families and community (World Bank 2002b) Within the Indicative Framework, instead of the conventionally employed

of education

Nevertheless, there is much room for improvement with the current Indicative Framework Those indicators (primary completion rate, repetition rate, etc) currently adopted are not sufficient to measure the quality of education It is obvious that a more comprehensive study examining the level

of student academic achievement and other matters is indispensable The current Indicative Framework, however, does not require surveys enabling such examination The Indicative Framework can also be improved by including, among others, the enrollment in lower-secondary education since qualitative improvement in primary education is usually expected to increase enrollments in secondary education

The Indicative Framework, a set of indicators to be utilized for improvement in the education sector by the government of developing countries, also serves as indicators for educational development assistance by developed countries and international agencies The importance of monitoring and evaluation of effectiveness and efficiency of educational development assistance has been frequently pointed out since the 1960s, without much progress in the formulation of internationally-agreed evaluation criteria The education indicators developed by UNESCO and OECD, widely applied to gauge the educational situation in developing countries, have been utilized only in a limited manner to assess educational development assistance The Indicative Framework,

on the other hand, can be utilized as indicators for this purpose since it evaluates the education sector with reference to numerical indicators concerning the financial situation, effectiveness of policies, etc., thereby providing a “common frame of reference for all countries” (World Bank 2002b, p 11)

It can also be used by donors in the decision-making process of educational development assistance since the Indicative Framework, comprised of internationally comparable indicators, can be utilized

to determine the degree of political leadership of developing governments in the process of adopting appropriate primary education policies, which can determine the implementation of aid (Prouty 2002)

Now, let us examine how education indicators are utilized in the actual process of implementing educational development assistance through the EFA-FTI, an international initiative designed for the active use of the Indicative Framework and various other education indicators Such examination should ideally include the verification of national-level changes realized in the actual situation of primary education in FTI recipient countries However, it is still too early to verify the effectiveness of the initiative itself since only a few years have passed since its full-scale introduction Therefore, this paper examines how education indicators are actually utilized in the process of selecting FTI recipient countries