Teacher Assessment Knowledge and Practice A Narrative Inquiry of a Chinese College EFL Teacher''s Experience TRƯỜNG ĐẠI HỌC MỞ BÀI GIẢNG GIÁO TRÌNH

Teacher Assessment Knowledge and Practice: A Narrative Inquiry of a Chinese College EFL Teacher’s notions of sacred stories and secret stories in teachers’ professional knowl-edge lan

Teacher Assessment Knowledge and

Practice: A Narrative Inquiry of a

Chinese College EFL Teacher’s

notions of sacred stories and secret stories in teachers’ professional

knowl-edge landscapes (Connelly & Clandinin, 1995), it examines Betty’s account of her experience of assessment reform This account com-prises three stories involving her colleagues and students Following Connelly and Clandinin’s (2006) three strands of narrative inquiry, this

article discusses three structural conditions of teacher knowledge,

tempo-rality , sociality , and place , and their effects on teachers’ knowledge

con-struction of assessment We argue that teachers’ prior assessment experience will affect their current practices and future plans for assess-ment (temporality); power relationships in teachers’ workplace will greatly in uence their assessment decision-making (sociality); and the specifi c contexts in which assessment takes place will affect teachers’ sense of security and therefore the effectiveness of the assessment (place) The fi ndings highlight the recognition of teachers’ agency in assessment practice, the importance of negotiation with teachers of the reform policies, and the urgent need for professional development

Since opening its door to the outside world in the late 1970s, China has made learning English a national priority (Lam, 2005) Since 2001, the starting age to learn English as a compulsory subject in school has been lowered to nine (Liu, 2007) As examinations enjoy a wide societal acceptance as a fair means of measurement for selection and promotion (Cheng & Qi, 2006), passing them has become one of the essential requirements for admission to a higher level of schooling (Cheng, 2008)

At the tertiary level, in uential tests such as College English Test, Bands 4

and 6 (CET 4/6) are conventionally considered, if not offi cially required,

as a prerequisite for graduation (Jin, 2004) These large-scale tests have

been shown to produce a signifi cant backwash effect on teaching, known

as the “teaching-to-the-test” phenomenon (Tang & Biggs, 1996, p 163),

which is widely regarded as undesirable in educational terms

To avoid such negative effects, Chinese TESOL educators have begun

seeking an alternative means of evaluation Formative assessment, or

assess-ment for learning (Black & Wiliam, 1998), has been advanced because

of its proven benefi ts in relation to student learning (Genesee & Upshur, 2001;

Marsh, 2007; Sutton, 1992; Torrance & Pryor, 1998) The Chinese Ministry

of Education (MoE) has promulgated the use of formative assessment in

the newly published curriculum framework of EFL teaching at the tertiary

level, College English Curriculum Requirements (2004, 2007), in the hope

that students can improve their learning based on teachers’ feedback in

formative assessment and teachers can obtain useful information from

students for better instructional planning

The increased policy attention to formative assessment, however, does

not ensure its effective implementation First, the existence of public

assessment scales does not by itself ensure that teachers will use the

knowledge assumed by the published assessment framework (Leung,

2004), implying an urgent need to understand teachers’ actual assessment

criteria in practice When compared with the bulk of studies on student

gain in relation to formative assessment, relatively little attention has

been paid to teachers’ criteria and practice except in a handful of recent

studies (Ash & Levitt, 2003; Graham, 2005; Leung, 2004; Leung & Mohan,

2004) Moreover, although it is generally believed that teachers need

long-term professional development opportunities before they are

competent to embed formative assessment effectively in their teaching

(Black, Harrison, Lee, Marshall, & William, 2003; Shepard, 2000;

Torrance & Pryor, 2001), formative assessment is not a major component

of teacher education (Leung, 2004) In China, college EFL teachers

receive little formative assessment training in their preservice bachelor’s

or master’s degree programs 2 and no assessment training after graduation

In addition, the reciprocal relationship between teaching and assessment

1 The CET is a large-scale standardized test administered nationwide by the National College

English Testing Committee on the behalf of the Higher Education Department of the

Ministry of Education in China It is intended to measure the language profi ciency of non–

English-major college students

2 According to a survey conducted by Zhou (2005) among 1,200 EFL teachers in

49 different colleges, 56% of teachers hold a bachelor of arts degree, and 28% hold a

mas-ter of arts degree The bachelor’s degree programs that prepare English teachers are mostly

provided by English departments in teachers’ colleges, where most of the teacher training

focuses on practical teaching (Wu, 2002) According to Wang (2003), most of the master’s

degree programs of applied linguistics that prepare college EFL teachers in China

gener-ally provide one testing course, which has little to do with formative assessment

is not fully visible to teachers who tend to see assessment as an additional burden to teaching rather than an integral part of it (Heritage, 2007) In sum, the fact that “formative assessment is not well understood by teachers and is weak in practice” (Black & Wiliam, 1998, p 20) makes it a daunting task to probe teachers’ knowledge, practice, and professional development

in formative assessment

TEACHERS’ KNOWLEDGE, PRACTICE, AND

PROFESSIONAL DEVELOPMENT IN ASSESSMENT

A review of literature on language teachers’ classroom-based assessment 3 suggests that relevant studies focus on the following three issues: teachers’ knowledge of assessment, assessment practices, and professional devel-opment in assessment Earlier research concerned with teachers’ knowledge of assessment has found that teachers are often torn between two con icting beliefs: that assessment should be for the facilitation of learning or that assessment should be for reporting achievement (Brindley, 1989; Stiggins, 1999) More recent research shows that teachers do not use externally prescribed standards only as a basis of their judgment; they also turn to their own implicit knowledge and beliefs, especially when con icts arise between their personal assessment and standardized criteria (Arkoudis & O’Loughlin, 2004; Davison, 2004) By observing the con ict between teachers’ own professional judgments of learner performance and the external standards they were required to meet, Davison (2004) posits that individual teachers may interpret rating rubrics differently on the basis of their different assessment beliefs With this understanding

of the importance of teachers’ personalized judgment, researchers have further explored teachers’ assessment cognition as well as its sources For example, both Yin (2005) and Rea-Dickins (2007) examine teachers’ decision-making in assessment practice and trace the sources of such cognitive activities, which include teacher training, previous teaching experience, and professional contexts These studies reveal that the complexity of teacher knowledge needs to be further investigated in teachers’ assessment practice

The second issue is centered on teachers’ assessment practice Compared with the extensive use of teacher assessment in classroom prac-tice, relatively few studies empirically examine the assessment practices

of TESOL educators and practitioners in schools (Cheng, Roger, & Hu, 2004; Clarke & Gipps, 2000) Among the limited number of studies that investigate teachers’ assessment practice, some have investigated teachers’

3 Classroom-based assessment is here used in a broad sense that includes all teacher

assess-ment, irrespective of summative or formative purpose (Rea-Dickins, 2008)

assessment practice in classroom settings and larger sociocultural contexts

(Leung, 2004; McNamara, 2001; Rea-Dickins, 2008) McNamara points

out that assessment is a social practice; by extension then, any research on

teacher assessment needs to be conducted in classroom settings so as to

understand the lived experiences of teachers and learners Arkoudis and

O’Loughlin (2004) explore how teachers work with the state-mandated

assessment policy within the local educational contexts, and they caution

that teachers need to be fully aware of the social realities and power

relations within the educational bureaucracy in which they work The

interactive nature of formative assessment and the multiplicity of contexts

in which it is situated have made its practice a complex issue

The third issue is teacher development in assessment The complexity

of teachers’ knowledge and the poverty of their practice in formative

assessment both demonstrate the signifi cance of teacher development

in assessment There is growing evidence revealing a real need for

building teacher capacity (Bachman, 2000) and developing new skills

and abilities in teacher assessment (Edelenbos & Kubanek-German,

2004) It has been suggested that teacher capacity building includes

not only teachers’ understanding of the language (Grierson, 1995), but

also their understanding and practice of effective classroom assessment

(Gardner & Rea-Dickins, 2001; Oliver, Haig, & Rochecouste, 2005) Both

of the aforementioned multifaceted requirements of teacher assessment

competence and recent ELT teacher education research (Freeman, 2002)

indicate that teacher development in assessment cannot be achieved by

teacher training alone Drawing on the evidence from two case studies,

Ash and Levitt (2003) argue that formative assessment provides teachers

with opportunities for professional growth as it involves both individual

and mutual appropriation of learner products Likewise, Graham’s

(2005) study discusses how preservice teachers benefi t from

classroom-based assessment training and assessment-driven planning, and it also

reports that professional dialogue about planning and assessment in

campus classes and mentored fi eld experiences in uences teacher

growth in assessment In sum, teacher change in assessment practice, as

Leung (2004) notes, is “often a very complex, different, highly personal

and long-term process” (p 34)

In reviewing these studies, we have reached the following conclusions

about teachers’ knowledge, practice, and development in assessment

First, the tension between teachers’ implicit knowledge of assessment

and the prescribed standards calls for new methodology to address the

uniqueness and individuality of teachers’ assessment practice Second,

the salient issue of teachers’ actual practices in assessment and the limited

understanding of such practices require further empirical research The

interactive nature of teachers’ practice also highlights the importance of

situating teachers’ assessment practice in larger sociocultural contexts

Third, the complexities of teacher capacity building and the relatively slow and highly personal process of teacher change call for sustained efforts to promote teacher development in assessment

Based on these conclusions from the literature, this article explores the knowledge and practice of teacher assessment through a narrative inquiry into the experience of a Chinese college EFL teacher, Betty, 4 who was involved in an assessment reform program By investigating Betty’s stories

of assessment and situating them in multiple contexts, the article attempts

to make sense of Betty’s knowledge and practice of assessment and the effects of structural conditions on teacher knowledge construction The implications for teacher development in assessment are also highlighted

THEORETICAL FRAMEWORK: THE LANDSCAPES OF TEACHERS’ PROFESSIONAL KNOWLEDGE

One of the most powerful theories of teacher knowledge is that proposed

by Connelly and Clandinin (1988, 1995) In contrast to the technical rationalistic approach to teacher knowledge, they contend that teacher knowledge is not something objective and independent of the teacher that may be learned and transmitted but is rather the sum total of the teacher’s

experiences Following Elbaz’s (1983) notion of practical knowledge, which

is oriented toward contexts, Connelly and Clandinin (1988) emphasize

the personal aspect of teacher knowledge and refer to it as personal practical

knowledge because it is derived from a person’s narratives and aimed at

meeting the demands of a particular situation Adopting an experiential philosophical approach, Connelly and Clandinin further argue that teacher knowledge is “in the teacher’s past experience, in the teacher’s present mind and body, and in the future plans and actions” (p 25) Connelly and Clandinin’s sustained inquiries into the personal practical knowledge of teachers led them to consider how teachers’ personal practical knowledge shapes and is shaped by the environment

in which they work To that end, they have developed the concept of

teachers’ professional knowledge landscapes (Connelly & Clandinin, 1995),

which they describe as follows:

A landscape metaphor allows us to talk about space, place, and time Furthermore, it has a sense of expansiveness and the possibility of being

fi lled with diverse people, things and events in different relationships Because we see the professional knowledge landscape as composed of relationships among people, places, and things, we see it as both an intel-lectual and moral landscape (p 5)

4 To protect the real identity of the EFL teacher, Betty, all personal names, and names of institutions are fi ctitious

The professional knowledge landscape provides a way to contextualize

teacher knowledge In addressing the question of how teacher knowledge

is shaped by the professional knowledge context in which teachers work,

Clandinin and Connelly (1996) borrow Crites’s (1971) terms sacred story

and secret story to illustrate the epistemological dilemmas that teachers

may face in their knowledge construction both in and out of the

classroom

According to Crites (1971), sacred stories emerge from the

out-of-classroom place where teachers are expected to do as the imposed

prescriptions instruct They are often other people’s visions of what

is right for teachers and students; they may arise from researchers,

policy makers, and administrators, in the form of theory, policy

statements, and improvement schemes In contrast, when teachers

move into their classrooms, they are “free from scrutiny” so that they

“live stories of practice” (Clandinin & Connelly, 1996, p 25) The real

story that is actually believed and acted on is a secret story Because

sacred stories and secret stories are told by the authorities and teachers,

respectively, they are analytically distinctive in conceptualizing teachers’

stories This article uses these two concepts as the main vehicle for

analysis

METHODOLOGY: NARRATIVE INQUIRY

OF TEACHER KNOWLEDGE

The investigation of Betty’s assessment knowledge and practice was

conducted through a narrative inquiry Bruner (1990) has noted that

human life is fundamentally narrative and stories are a common way of

organizing knowledge The story is the basic means by which we

understand the world and the closest way of making sense of our

experiences Narrative inquiry, or the study of experience as story, is

therefore “fi rst and foremost a way of thinking about experience”

(Connelly & Clandinin, 2006, p 375)

In educational research, narrative has emerged as both a method in,

and an object of, inquiry in teacher education, especially in descriptions

and analyses of teacher knowledge (Clandinin & Connelly, 2000; Elbaz,

1983) Researchers of teacher education argue that teacher knowledge is

largely structured through stories and that the story is epistemologically

the most authentic way to understand teaching from the teacher’s point

of view (Clandinin & Connelly, 2000; Lyons & LaBoskey, 2002) Through

telling and retelling, living and reliving teachers’ own stories, teachers’

lived experiences, including their knowledge and practices in educational

reforms, can be interpreted and understood narratively (Clandinin &

Connelly, 1996; Craig, 2007; Tsui, 2007)

Connelly and Clandinin (2006) identify three commonplaces of

narrative inquiry: temporality , sociality , and place Together, these constitute

a three-dimensional space of narrative inquiry Temporality refers to the temporal transition in which events and people exist It requires the narrative inquirer to consider an event, person, or object in relation to its past, present, and future Sociality encourages narrative inquirers to describe both the personal and social conditions of the person under study, whereas place requires them to attend to the specifi city of location where events take place All three strands come into play in settings both

in and out of the classroom

In the current study, Betty’s assessment knowledge and practice are analyzed through a narrative inquiry based on the stories that she told There are three sets of data The fi rst set includes two formal semistructured interviews, each of which lasted for about 3 hours, and in which Betty shared with us her experiences of conducting assessment as well as what she found signifi cant in her teaching and assessment practice The second set of data is derived from personal communications with Betty Before and after the interviews, we discussed her learning and teaching experiences in several personal conversations These discussions were used as input to formulate the interview questions and to develop a broad description of the contextual and personal contours of the EFL teacher The third set of data is Betty’s personal records, including her teaching evaluation sheets, 5 teaching and assessment plans, and the notes she made in relation to her assessment practice and her students’ responses

To allow Betty to better express herself, all interviews and conversations were conducted in Chinese The interviews were fi rst transcribed verbatim and then translated into English 6 To make the teacher narratives more visible, the transcribed interviews were then rewritten in English as three condensed and coherent stories of Betty’s assessment practice (Søreide, 2006) To ensure that we had made no major changes in the content of the refi ned teacher narratives, we fi rst compared them with the original transcripts and then sent them to Betty for her to verify the authenticity

of the stories and to give consent for our use

Because the narrative approach has an analytical interest in individuals and provides personal meanings in interpretation (Clandinin et al., 2006), we chose to use the case of a single teacher to illuminate the complexity of teachers’ assessment knowledge and practice The study does not aim to draw a universally applicable conclusion but rather to

5 In Betty’s university, the teaching evaluation sheet is the electronic record of the students’ evaluation of the teacher in each semester It consists of the scores on each of the ten spe- cifi c scales and students’ feedback of and comments on the teacher’s teaching It is acces- sible to the teacher when the semester is over

6 To ensure its conformity to the Chinese version, the translation was given to two Chinese EFL teachers, colleagues of the authors, to read

highlight the salient issues in implementing teacher assessment and

encourage teachers to re ect on their assessment practices

BETTY’S ASSESSMENT KNOWLEDGE AND PRACTICE

The ELT Teaching and Assessment Reform Landscape

As mentioned earlier, researchers have realized the negative backwash

effects brought about by the predominance of summative assessment and,

as a result, they have called for the use of formative assessment to adjust

student learning (Genesee & Upshur, 2001; Wang & Fu, 2006) The School

of English Education (SEE), the school in which Betty teaches, initiated a

communicative language teaching reform with the assessment reform as

a critical component in 2001 The assessment reform was implemented in

all of the English courses at the university and was intended to transform

the purpose of assessment from summative to formative For example,

in the case of the Comprehensive English course, the assessment

system has gone through signifi cant changes since 2001, with the

focus shifting from tests to regular informal assessment (see Table 1 )

As Table 1 indicates, 2004 was a turning point for the assessment

system For the fi rst time, the assessment of daily performance was

given more weight than exam results in the fi nal evaluation Since then,

the assessment of students’ daily performance has consisted of four

parts—classroom participation, assignments, learning projects, and

presentation—all of which aim to encourage students to learn English

for use rather than for tests Teachers are required to implement this

assessment plan as it has been prescribed in each course syllabus It is

within this language assessment landscape that Betty’s lived experience

of assessment was situated

Betty’s Stories: A Diligent, Independent Student

and a Critical, Well-Respected Teacher

Betty was born in a small and poor village in southern China in the

early 1980s While studying in the best middle school in town, Betty found

TABLE 1 The Evolution of the Assessment System in the Comprehensive English Course

Academic year Mid-term score Final exam score Daily performance

that she was good at scientifi c subjects; thus, she dreamed of being admitted to a key university as a science major However, as a result of the national entrance exams, against her wishes, she was enrolled as an English major at Shangda, a key university in northern China Betty soon found that among a group of highly profi cient English learners, she was at a disadvantage with the limited English knowledge and heavily accented spoken English she had learned in her hometown Feeling pressured to catch up with her classmates, she worked very hard and made every effort to do well in all kinds of examinations, which were the only means of assessment in her university By the time she graduated, Betty was admitted to the master of arts program without taking the entrance exams because of her outstanding academic record Her learning experiences as a graduate student were marked by much academic activity, many publications, and the development of her critical thinking skills

Upon her graduation with master of arts degree from Shangda, Betty secured a teaching position in Tianda, a university in Guangdong Province with a good reputation for quality language learning and teaching Since her recruitment, she has been assigned to teach different courses to non-English majors, including Comprehensive English, Writing, and Listening and Speaking She is popular among students and, as a result, her classroom evaluation score is one of the highest of all in the faculty Her popularity, however, is not achieved by being kind and lenient but by being strict and critical She seldom praises her students and is sparing with her positive feedback At fi rst, many of her students felt frustrated and uncomfortable, but they soon found that they had learned a great deal and that their potential was developed with Betty’s constructive criticism As a result, Betty’s “critical” way of teaching has gradually been accepted, and she has become well respected by her students

Recently, Betty’s students gave her a photo album as a present for Teachers’ Day, in which they expressed their gratitude and good wishes One of the students, however, wrote a list titled “What Betty Has Taught Us,” with one of the quotations re ecting Betty’s unwavering belief in assessment: “I don’t believe that the test-oriented education would produce ‘high-score–low-ability’ numbness as what the media advocates

My experience told me that it did not bring about such numbness but

‘high-score-high-ability’ intelligence.” Betty’s three assessment stories outlined as follows correspond to three kinds of experience that she has had in assessment The fi rst story tells how Betty used her self-developed

“reward-or-punishment” strategy to assess student presentation The second story tells how she was pressed by her colleague to change the grades she assigned to the students The third story is about how she empowered her students to become self-assessors by giving them the right to do self-assigned homework

“Reward or Punishment”: Knowledge Construction With Self

Classroom-based student presentation is one of the assessment activities

in the Comprehensive English course that Betty teaches According to

the course syllabus, teachers are supposed to award a general score worth

20% of the fi nal grades, in addition to providing feedback to help improve

student performance Betty mentioned repeatedly in the interviews that

“I don’t trust formative assessment,” which intrigued and induced us to

fi nd out how a follower of nonformative assessment deals with a more

formative kind of assessment Her reward-or-punishment assessment

strategy provides the answer to our question

I have an initial evaluation of students’ language competence based on

their grades of quizzes, and rank them into different levels such as A, B, C,

or D I remember a student whose name is Yao is about Level B according

to her results of quizzes But the presentation she made was carefully

designed and professionally presented, which far exceeded my

expecta-tion of her So I gave her positive feedback as one of the few excepexpecta-tions in

class, and awarded her presentation the grades of 19 out of 20, with 2

points as a reward for her excellent performance On the contrary, I would

decrease the student’s grades as a punishment if he or she failed to reach

my expectation

As the story shows, Betty would punish or reward her students by

decreasing or increasing their grades on presentations, depending on

whether the particular performance failed to reach or exceeded her

expectation of the student based on their quiz results The quizzes are

four language achievement tests, each of which is developed by her course

group and conducted after each unit They are aimed at measuring

students’ achievement in the particular unit and consist of three sections

of multiple choice questions including vocabulary, grammar, and reading

comprehension From Betty’s account, we knew that she had used this

reward-or-punishment strategy for two consecutive semesters and found

it effective to use the quiz grades as a frame of reference for her formative

assessment practice She added that the quizzes had given her a quick

estimation of a student’s language competence, which would assist her

assessment of the student’s ongoing learning performances over the

long term

“Are Your Grades Too High?”: Knowledge

Construction With Colleagues

Betty’s reward-or-punishment strategy seems to re ect her independent

decision making and her own knowledge construction in assessment