Principles for Large-Scale Classroom-Based Teacher Assessment of English Learners’ Language An Initial Framework From School-Based Assessment in Hong Kong TRƯỜNG ĐẠI HỌC MỞ BÀI GIẢNG GIÁO TRÌNH

Principles for Large-Scale Classroom-Based Teacher Assessment of English Learners’ Language: An Initial Framework From School-Based Assessment in Hong Kong LIZ HAMP-LYONS Universi

Greene, J A., & Azvedo, R (2007) A theoretical review of Winne and Hadwin’s model

of self-regulated learning: New perspectives and directions Review of Educational

Research, 77 , 354–372

Leung, C (2007) Dynamic assessment: Assessment for or as teaching? Language

Assessment Quarterly, 4 , 257–278

Ramaprasad, A (1983) On the defi nition of feedback Behavioral Science, 28, 4–13

Smith, F., Hardman, F., Wall, K., & Mroz, M (2004) Interactive whole class teaching

in the National Literacy and Numeracy strategies British Educational Research

Journal, 30 , 395–411

van Lier, L (1996) Interaction in the language curriculum: Awareness, autonomy and

authenticity Harlow, England: Pearson Education

Principles for Large-Scale Classroom-Based Teacher

Assessment of English Learners’ Language: An

Initial Framework From School-Based Assessment

in Hong Kong

LIZ HAMP-LYONS

University of Nottingham

Nottingham, England

 Davison and Leung (this issue) describe the fi eld of teacher-based

English language assessment as having “much variability, a lack of

system-atic principles and procedures and a dearth of information as to the

impact of teacher-based assessments on learning and teaching” (p 389)

In this Forum contribution, I briefl y explore an example of teacher-based

assessment of oral English that has been implemented across Hong Kong

(see Davison 2007; Davison & Hamp-Lyons, in press) and the ways which

elements of practice have contributed to or inhibited the establishment

of the systematic key principles that are needed to establish any form of

large-scale alternative assessment

As this issue shows, assessment for learning is different from assess

-ment of learning Black and Wiliam (1998) have summarized fi ve

ele-ments that seem to be essential in the effectiveness of any attempt to

improve learning by applying formative assessment or assessment for

learning approaches: (a) the setting of clear goals; (b) the design of

priate learning and assessment tasks; (c) the communication of

assess-ment criteria to teachers and learners; (d) the provision of high quality

feedback (both oral and written); and (e) the conscious provision of

opportunities for self- and peer-assessment Assessment for learning gives

a real role to the teacher as assessor of her/his own students’ learning

and as judge of whether each student has met the expected standards of the subject at the level taught However, when assessment for learning meets formal examinations, paradigm confl ict can emerge (Hamp-Lyons, 2007) In Hong Kong, the Hong Kong Examinations and Assessment Authority has implemented a policy of inserting a form of teacher-based assessment across most senior secondary subjects, locally referred to

as school-based assessment (SBA), into the existing formal territory-wide

examination system Although SBA looks very different in different sub-jects (see, e.g., Cheung, 2001, on teacher assessment in advanced-level physics, and Yung, 2001, on advanced-level biology), the ambitious aims

of English SBA, of assessing authentic interactive use of the language in ways not easily assessed in formal examinations, have raised the possibil-ity of a paradigm shift (Davison, 2007; Davison & Hamp-Lyons, in press) 2

English SBA aims to bring together the highly formative aims of assess-ment for learning with the need to be meaningful at the level of the indi-vidual school and classroom, and to be accountable territory-wide Because it is carried out in every secondary fourth- and fi fth-year English classroom in Hong Kong and the scores awarded by teachers comprise 15% of the English language public examination result, SBA still has to meet the traditional expectations of rigour for summative reporting, at the same time promoting formative assessment 3

Although research into and validation of SBA is ongoing, it has become clear that if this attempt to reconcile the formative goals of the innovation with system-wide accountability demands is to be successful, only a prin-cipled approach makes sense An expedient approach would be subject to the vagaries of shifting vested interests and the demands of different stake-holder groups Furthermore, interest in the innovation from other coun-tries, especially in Asia, has already made it important to propose a statement of principles for SBA that might guide new adopters of this or a similar approach What follows is not—and cannot at this stage be—a fully developed theoretical and argued treatment of school-based language assessment principles, because of the wide range of test and assessment conditions into which a form of SBA might be introduced Rather, it is

an attempt to extrapolate some early-stage principles for further

2 The basic plan for an interactive, readingbased, assessment of students’ oral English profi -ciency by classroom teachers grew from a major curriculum and assessment reform (IBM,

2003 , and the Consultancy Team of Chris Davison and Liz Hamp-Lyons, with a team of research staff at The University of Hong Kong, have since early 2005 worked on the research, design, development, standards setting, and implementation of the new assess-ment, funded by the Hong Kong Examinations and Assessment Authority, and also on several research and teacher capacity-building projects funded by the Quality Education Fund and the Hong Kong Research Grants Council

3 SBA results play a signifi cant part in deciding whether or not a young person continues schooling and aims for university or leaves education at this stage

argumentation and validation from the development decisions made

dur-ing the four-plus years of work on introducdur-ing into the Hong Kong

Certifi cate of Education (English) exam a classroom-based, teacher-led

speaking assessment It is an attempt to formulate a more specifi

cally-tar-geted set of principles for SBA than has previously been available in the

educational measurement literature The principles proposed in this brief

article also acknowledge the specifi c nature of language assessment in

that, with the assessment of oral interaction skills based on a

reading–view-ing programme of input, we are simultaneously assessreading–view-ing medium and

message

The work of Marion and Pellegrino (2006) has been of particular

value Marion and Pellegrino argue that, for alternative assessment, a

new model of assessment is required that accounts for the relationship

between the characteristics of the students tested and how they develop

“domain competence ” (p 52), the nature of observation of student

per-formance, and the ways in which observation data are to be interpreted

Marion and Pellegrino also stress that “decision consistency” (p 52) is a

key quality to be achieved by alternative assessments within a

standards-referenced reporting framework (as is the case in Hong Kong), a more

humanistic view of what mainstream educational measurement would

refer to as reliability However, the rapidly growing international

assess-ment-for-learning movement holds as a principle that a classroom-based,

teacher-led assessment that is intended to directly affect the quality of

teaching and learning must ensure that the teachers themselves are

enabled to conduct valid assessments In the case of school-based,

teacher-conducted assessment, teachers will require extensive

profes-sional development For Hong Kong English oral SBA, a comprehensive

set of DVDs was developed, comprising a 24-hour programme of

profes-sional development (PD) which can be delivered by trained teacher

edu-cators (SBA Consultancy Team, 2006) These PD materials have

emphasized the close connections between teaching, learning, and

assessment, and teachers have been shown models of lesson sequences,

sample texts and sources, model tasks, relevant readings from the

profes-sional literature as well as short input texts on key principles written by

the consultants, and an array of student performances at a wide range of

levels These PD courses have been delivered to more than 4,000

teach-ers to date

The key principles of a test or assessment instrument are commonly

referred to as reliability, validity, practicality, and (sometimes)

conse-quences However, other formulations are emerging, including McMillan’s

(2000), which proposes principles that embody “the big ideas that, when

effectively applied, will effectively guide good assessment practices,

regardless of the grade level, subject matter, developer, or user of the

results” (p 1) McMillan argues that

1 Teacher assessment is a process of professional judgement requiring understanding of basic measurement principles and a critical evalua-tive ability to effecevalua-tively interpret evidence from student perfor-mances and behaviour and make value judgments about the meaning

of results

2 Assessment decisions are the result of a series of tensions or trade-offs (e.g., between formative and summative, or between criterion refer-enced and norm referrefer-enced)

3 Assessment infl uences student motivation and learning 4

4 Good assessment must be aware of both its intended and unintended consequences

5 Good assessment is fair

6 Good assessment uses multiple methods and sources of evidence

7 Assessment must be effi cient and feasible as teachers and schools have limited time and resources 5

These principles are valuable, but in developing and implementing SBA, the Hong Kong research team felt the need to elaborate on particu-lar areas that are less commonly discussed and are not well understood in Hong Kong and many other Asian contexts These elaborations are emerging not only from the educational measurement literature but from empirical interview data collected in a 4-year longitudinal study of the SBA implementation:

1 A classroom-based assessment should foster student learning and development by

(a) ensuring that all students have equitable access to the resources, tools, and information they need to succeed;

(b) providing tasks that are responsive to the needs and abilities of stu-dents within the school and the class and that stimulate the authen-tic use of English to read and view texts, and to talk about texts; (c) providing equitable opportunities for students to learn the relevant materials, skills, and knowledge to succeed in the curriculum; (d) building the capabilities and motivation of teachers to improve students’ teaching and learning through appropriate, timely, and accessible professional development

4 McMillan cites Wiggins (1998 ), who uses the term educative assessment to describe

tech-niques and issues that educators should consider when they design and use assessments in order to ensure that the assessments themselves lead students to have meaningful engage-ment in the learning process

5 The greatest resistance to SBA has come from teachers who perceive it to be a major added burden in terms of time and workload, which suggests that further teacher professional development is needed along with even more support from education authorities

2 Validity for a classroom-based assessment must take special account

of the construct being assessed by

(a) defi ning the domain to be assessed (in the case of SBA,

speak-ing profi ciency) and ensurspeak-ing an appropriate and fully

repre-sentative form of observation of student performance ( construct

representativeness );

(b) accounting for the relationship between the characteristics of

the students tested (starting point, needs, opportunities to

learn, etc.) and the pathways by which they develop

compe-tence in the ability to be assessed ( construct relevance );

(c) systematically documenting how the observation data are to be

interpreted ( accountability ) In the specifi c case of school-based

assessment, it is particularly important for teachers and

admin-istrators to understand their roles in gathering and interpreting

validity evidence

3 A classroom-based assessment should foster an atmosphere of

fair-ness in teaching, learning, and assessment by

(a) including equal opportunity to learn as a key aspect of fairness in

classroom-based assessment;

(b) ensuring that scores reported for individual students are based

on the best possible sample of language performance that can

be obtained within the classroom context;

(c) ensuring that teachers within each school meet regularly to

dis-cuss student performances within the year but between classes;

(d) ensuring that teachers from different schools can meet

occa-sionally to view and discuss students’ performances, reminding

them of the wide variation in student abilities across the whole

school system;

(e) providing professional development opportunities to teachers

to improve their ability to see the links between observing

per-formances; recording strengths, weakness, and needs from

those observations; and building support and/or further

chal-lenges for learners as appropriate;

(f) recognizing that fairness is as much a sociocultural as a technical

construct 6

4 A classroom-based assessment should foster educational

improve-ment and accountability through a system-wide process by

(a) providing evidence of student and school performance using

multiple indicators of student learning over suffi cient time to

demonstrate improvement;

6 Different cultures vary in their interpretation of fairness; it is not just a technical issue This aspect

of the SBA implementation is discussed in an article by the author currently under review

(b) ensuring that teachers and involved administrative staff are pro-vided with training and opportunities to improve in areas where they fi nd themselves to be weak;

(c) carrying out ongoing quality assurance and validation

The fi nal issue, or principle, that needs to be considered in shepherd-ing any assessment innovation toward success is the powerful role played

in education by politics within an education system SBA, and any new approach to assessment, cannot be expected to be successful in the class-room unless conditions at higher levels—the school, the education sys-tem, the state, and the society as a whole—are sympathetic to reform Thus, the need for educational and social activism is one that cannot be ignored

THE AUTHOR

Liz Hamp–Lyons is Special Professor in English Language Education at the University

of Nottingham, England Her research interests include the development and valida-tion of English language performance assessments; rater training; English language teacher professional development, especially in school-based assessment; writing ped-agogy, theory, and materials for teaching writing; language programme evaluation; advanced academic literacy; and English for academic purposes

REFERENCES

Black, P., & Wiliam, D (1998) Assessment and classroom learning Assessment in Education, 5 , 7–75

Cheung, D (2001) School-based assessment in public examinations: Identifying the

concerns of teachers Education Journal, 29 , 109–123

Davison, C (2007) Views from the chalkface: School-based assessment in Hong

Kong Language Assessment Quarterly, 4 , 1–32

Davison, C & Hamp-Lyons, L (in press) The Hong Kong Certifi cate of Education: School-based assessment reform in Hong Kong English language education In

L.-Y Cheng & A Curtis (Eds.), English language assessment and the Chinese learner

New York: Routledge

Hamp-Lyons, L (2007) The impact of testing practices on teaching: Ideologies and

alternatives In J Cummins & C Davison (Eds.), The international handbook of English language teaching , (Vol 1, pp 487–504) Norwell, MA: Springer

IBM (2003) Strategic review of the public examination system: Final consultancy report

Hong Kong SAR, China: Examinations Authority

Marion, S F., & Pellegrino, J W (2006) A validity framework for evaluating the

tech-nical quality of alternative assessments Educational Measurement: Issues and Practice,

20 (4), 47–57

McMillan, J H (2000) Fundamental assessment principles for teachers and school

administrators Practical Assessment, Research & Evaluation , 7(8) Retrieved March

14, 2007, from http://PAREonline.net/getvn

SBA Consultancy Team, Faculty of Education, University of Hong Kong (2006)

Professional development courses for teachers in preparation for the school-based assessment

in HKCE English language examination [visit http://web.hku.hk/~sbapro/research.

html for further information]

Wiggins, G (1998) Educative assessment: Designing assessments to inform and improve

stu-dent performance San Francisco: Jossey-Bass

Yung, B H W (2001) Examiner, policeman or students’ companion: Teachers’

per-ceptions of their role in an assessment reform Educational Review, 53 , 251–260

Issues in the Development of a Descriptor

Framework for Classroom-Based Teacher

Assessment of English as an Additional Language

CATRIONA SCOTT

Newcastle University

Newcastle upon Tyne, England

 This article identifi es some key conceptual issues in teacher-based

assess-ment arising from the developassess-ment of a framework for formative

class-room-based teacher assessment of learners with English as an additional

language 7 (EAL) in publicly funded schools in England 8 Teacher-based

assessment, particularly for formative purposes, has become increasingly

prominent in educational policy documents both in the United Kingdom

and internationally Building on Black and Wiliam’s (1998a, 1998b) early

work on formative assessment, there is increasing evidence that such

assessment, also referred to as assessment for learning (AfL) (see,

e.g., Black et al., 2003), can be a powerful tool in promoting learning

In England there is no separate framework for the assessment of EAL

learners, who are assessed according to National Curriculum Levels,

7 English as an additional language ( EAL ) and English as a second language ( ESL ) both refer to

“learners who are using English as the medium of instruction in school contexts but who

are not English fi rst language (L1) speakers” (Rea-Dickins 2000, p 115), as distinct from

English as a foreign language (EFL) contexts in which English is not the lingua franca Where

these terms occur in this article, they are used synonymously

8 This contribution draws on the EAL Assessment Frameworks Research Project jointly

funded by the National Association for Language Development in the Curriculum

(NALDIC) and the Paul Hamlyn Foundation (2002–2003), which surveyed six assessment

EAL/ESL frameworks from Australia, the United States, Canada, and the United Kingdom,

and also my recent experience in developing a draft formative assessment framework for

trialing in elementary schools