EFL teachers’ views of english language assessment in higher education in the united arab emirates and kuwait

EFL Teachers’ Views of English Language Assessment in Higher Education in the United Arab Emirates and Kuwait SALAH TROUDI University of Exeter Exeter, England CHRISTINE COOMBE

EFL Teachers’ Views of English Language

Assessment in Higher Education in the

United Arab Emirates and Kuwait

SALAH TROUDI

University of Exeter

Exeter, England

CHRISTINE COOMBE

Dubai Men’s College

Dubai, United Arab Emirates

MASHAEL AL-HAMLIY

University of Kuwait

Kuwait City, Kuwait

 Issues of assessment design and implementation in Kuwait and the

United Arab Emirates (UAE) have attracted some attention over recent

years, but teachers’ philosophies about assessment remain

underex-plored This article reports the fi ndings of a qualitative study into the

assessment roles and philosophies of a group of teachers of English as a

foreign language (EFL) in the UAE and Kuwait Based on an open-ended

questionnaire, the study showed that teachers’ views on the nature of

assessment were informed by their knowledge of the fi eld of language

learning and teaching and by the contextual milieu and sociopolitical

factors that govern their employment conditions The study also showed

that teachers did not play a major role in assessment because of top-down

managerial approaches to education and a concern for validity and

qual-ity assurance in large programmes

Assessment continues to play a major role in learning and teaching

and is extensively and intensively addressed in research studies and

theo-retical articles both in mainstream education and TESOL/TEFL

litera-ture However, the majority of published work in testing is still informed

by psychometric views of assessment The view of assessment espoused

in this article does recognise elements of validity and reliability of the

psychometric model, but it goes beyond them to view assessment as an

act of social and cultural practice and the product of a complex

interac-tion of political, educainterac-tional, economic, and historical factors and

agen-das (Pennycook, 1994, 2001; Shohamy, 2001) We defi ne assessment as

“the process of collecting information about a student to aid in

deci-sion making about the progress and language development of the

stu-dent” (Cheng, Rogers, & Hu, 2004, p 363).With this practical defi nition,

however, we also believe that assessment is an exercise of power that is caught up in an array of issues about testers’ and test-takers’ voices, roles, and beliefs This critical view of assessment, with an agenda of under-standing and social change, suggests that teachers, students, and other stakeholders “construct the assessment knowledge by trying to make sense of the knowledge in a dialogical and co-operative way” (Shohamy,

p 136)

Although little research is being conducted into teachers’ beliefs and attitudes toward assessment practices and how these are shaped by cul-tural and institutional contexts (Davison, 2004), some recent research studies have started exploring the role of the English as a foreign lan-guage and English as a second lanlan-guage (EFL/ESL) teacher and his/her views of what constitutes an enriching assessment practice Cheng et al (2004), for example, report multifaceted and complex roles played by university instructors in Canada, Hong Kong, and China This large-scale comparative study found that teachers’ practices varied in three major areas: assessment purposes, methods, and procedures These differences, within and across settings, were due to varying cultural, institutional, and contextual factors; the nature of the courses; teachers’ knowledge of assessment; teaching experience; students’ needs; and the role of “exter-nal testing on teaching and learning” (p 378)

In another comparative study by Davison (2004), 24 secondary school teachers in Hong Kong and Australia were asked about their views and interpretations of the construct being assessed, with specifi c reference to written argument and whether they felt that their judgements were trusted and legitimated in their communities The qualitative data revealed that teachers’ practices and orientations can be classifi ed along

a cline “from assessor as technician, to interpreter of the law, to princi-pled yet pragmatic professional, to arbiter of ‘community’ values, to asses-sor as God” (p 324) This variety in interpretation of roles was also linked

to the effect of assessment approaches such as norm-, criterion-, or con-struct-referenced on teachers’ views and how they interact with teachers’ knowledge

Thus far, several studies have looked into teachers’ assessment prac-tices, with a few focusing on teachers’ beliefs and knowledge affecting their decision-making processes in classroom-based assessment (Chang, 2005; Davison, 2004) No research of this kind, however, has been con-ducted in the Gulf region So far, all studies have concentrated on either the psychometric and quantitative element of assessment with a focus on students’ performance (e.g., Addamigh, 2006; Gamaroff, 2006) or on a number of factors involved in the testing of the four language skills (e.g., Al-Busaidi, 2007; Al-Hamly & Coombe, 2005; Lanteigne, 2008) Farah’s (2007) study on the effects that a high-stakes international test has on stu-dents’ access to a fi eld of study of their choice is the only research with a

critical agenda aimed at questioning certain assessment practices in the

Gulf region Looking into issues of consequential and ethical validity,

Farah uncovered some of the detrimental effects that some major tests

have on students’ lives The study also found that teachers did not have a

voice and had no choice but to use externally imposed criteria to assess

their students Continuing with this particular interest in teachers’ voice,

the current study aims to redress a major gap in the assessment literature

in the Gulf

Informed by a sociocultural perspective and located within a broad

interpretive and participatory framework, this study also draws on work

done in the area of critical language testing to raise such issues as voice

and inclusion in assessment decision making about test format, content,

and structure This framework allows us to investigate teachers’ roles and

views of assessment in the UAE and Kuwait The unique situation of the

expatriate English language teacher (ELT) in these two countries is likely

to play a signifi cant role in the assessment scene

THE STUDY

The study sought to address the following main questions: What are

teachers’ roles in student assessment in the UAE and Kuwait, and what

are teachers’ assessment philosophies? With an interpretive framework

of research that seeks to “develop an in-depth and interpreted

under-standing of the social world” (Ritchie & Lewis, 2003, p 22) by focusing

on the individual as the main source of this interpretation, we developed

an open-ended questionnaire to assist us in collecting rich qualitative

data (Holliday, 2001) For logistical reasons it was not possible to conduct

interviews with participants, who worked in different parts of the UAE

and Kuwait We therefore developed a set of open-ended questions which

we sent via e-mail to a number of tertiary institutions where we thought

teachers interested in the issue of assessment were likely to participate in

the study This purposive sampling technique is appropriate for the aims

of this exploratory qualitative study (Dörnyei, 2007)

The questions developed for the open-ended qualitative questionnaire

(see appendix) were contextual, explanatory, and generative in nature

They were meant to help us identify what exists in terms of assessment

practices in the social world of the participants, explain why phenomena

occurred, examine reasons and forces behind their occurrence, and

gen-erate and develop new conceptions or understanding (Richards, 2003;

Ritchie & Lewis, 2003) The nature of the questions allowed the

partici-pants to express their views, and in some cases they replied in

consider-able detail

The Participants

A total of 21 tertiary ELTs replied to our invitation to take part in this study We followed established ethical research procedures, and all participants gave us their consent to use the data for publication and oral dissemination purposes Pseudonyms were used to protect the identities of the participants, who represented a total of nine colleges and universities All of them are expatriate teachers from a wide range

of countries such as Sudan, the United Kingdom, the United States, and Egypt, working for fi xed and renewable contracts The majority hold master’s degrees in TEFL/TESL or applied linguistics, and three hold doctorates; their teaching experience ranges from 3 to more than

20 years The participants work in a variety of English for academic purposes (EAP) programmes which prepare students for their transi-tion to university departments where subjects are taught through the medium of English, a common educational policy in the Gulf (Troudi, 2007) Three participants work in English for specifi c purposes (ESP) programmes

Data Collection and Analysis

Given our personal views on assessment, our familiarity with many of the institutions where the participants worked, and our critical agenda,

we sought to minimize the impact of our views on the analysis of the data

We therefore attempted to use a “strategic and technical detachment” approach to both data collection and analysis (Holliday, 2001,

p 178) To avoid imposing our views on the data, we analyzed it using exploratory content analysis The emerging themes were categorised and codifi ed and then compared with the whole set of data using a constant comparison method that included reading and rereading within and across the responses of the participants (Lalik & Potts, 2001) The analy-sis revealed a number of recurrent themes, minor categories, and even some individual occurrences However, it is beyond the scope of this arti-cle to include all the fi ndings, such as the variety of practices in the two countries

FINDINGS AND DISCUSSION

The participants invoked a number of varied personal views toward assessment, but there were also some points of similarity, which are dis-cussed in the following sections

Teachers’ Roles: Exclusion Versus Inclusion

One recurrent theme was that teachers were not involved in

assess-ment decision making about matters of approach and design Some

teachers did not participate in the process of designing assessment tools

which dealt with areas of content, format, number of test items, and

marking rubrics Some teachers also reported that when institutional

changes in assessment occurred, teachers were often excluded and their

views were not solicited A number of EAP programmes had a structured

and hierarchical approach to assessment planning and implementation

Testing committees or supervisors were in charge of developing policies

and assessment tools because teachers were perceived not to have

exper-tise in this area In fact, in some cases teachers were not informed about

the content of midterm or fi nal tests and had to wait until the testing day

to discover this with their students This element of secrecy caused many

teachers to feel distrusted and disrespected One of our teacher

infor-mants, Ahmed, considered that “teachers should be given more power in

assessing their own students in a way that meets students’ needs and

future/expected job demands,” and his opinion was echoed by others in

different institutions Because these programmes have large numbers of

EFL teachers, effi ciency, practicality, and reliability require that

assess-ment be centrally managed, which allows only a few teachers to be directly

involved in decisions on assessment Another explanation is that teacher

evaluation is linked, although not offi cially, to students’ performance, so

there is a concern that teachers might be inclined to assist their own

stu-dents to achieve high grades if they were involved in writing tests This

distrust was mutual: Some of the excluded teachers felt that the problem

was that, as one of the informant teachers put it, “there was very little to

no expertise and/or training It is often done by people who think that

they know what they are doing but in reality have no idea of basic testing

theory.” Employment conditions and policies are at the root of such an

atmosphere of distrust, for expatriate teachers are all contractually

employed, and teacher evaluation and, in some cases, students’ grades

may affect contract renewal

Teachers’ Philosophies

Alternative Assessment and Fairness

Teachers were aware that one way of ensuring fairness for students

was to use multiple methods in assessing their work Although they

rec-ognized the role of traditional or established measures like

standard-ized tests, they believed that learners would benefi t from a variety of

instruments, as stated by Paul, who teaches in one of the institutions in the UAE:

I believe that there should be a distinct difference between testing and assessment While I believe in various forms of traditional testing methods and the value of them in ESL/EFL, I also believe in the importance of alternative assessment techniques Students, in my opinion, should not always be merely tested on a regular basis or at the end of a particular unit

of study Instead, they should be regularly assessed on their ability to use the language in real and meaningful situations Alternative assessments are ideal in these instances Students should also be given more autonomy

in their particular learning situations and alternative assessments such as peer assessment, portfolios and rubrics/rating scales are excellent for accomplishing these goals

Some participants invoked the issue of fairness and viewed authenticity and multiple-measures assessment as an approach to ensure that tests were fair and that students were not unfairly disadvantaged The stu-dents’ interests were at the centre of their concern In fact, one of the teachers noted that “every effort should be made within an assessment system to make the assessment of students fair.” In this context, perfor-mance-based assessment was suggested as a viable approach Mohamed, who teaches EAP to fi rst-year students, explained that “teachers needed

to be trained to implement performance-based assessment in a way that capitalises on its strength, i.e., validity, and minimises its weakness, i.e., reliability.”

Initially, some views seemed to be presented in a culturally detached manner without references to the local and cultural contexts where the participants worked On further exploration, we could see that teachers’ assessment orientations were infl uenced by internal and external criteria

as well as by their knowledge of their students and the educational and cultural contexts Elaborating on his own philosophy, Mohamed stated that “this personal philosophy is adopted as long as the context is teach-ing relatively smaller classes to meet specifi c job and/or community expectations.” We would add that teachers’ views are also infl uenced by their own educational principles and pedagogical experience This posi-tioning confi rms observations made by Davison (2004), who argues that within a student-based assessment approach, teachers consider “not only common assessment criteria and community constructs, but also the learner and the context” (p 326)

No Assessment Philosophy

One signifi cant fi nding in this study is that some participants ques-tioned the idea of a philosophy of assessment (or any abstract principles)

They were more concerned about their daily assessment practices and

their impact on students In fact, Salwa in Kuwait explicitly admitted to

not having a particular philosophy: “I don’t have a philosophy about

assessment I mean what does that mean? Testing and assessment is a part

of studying that I don’t question much—beyond issues of validity and

reli-ability in individual tests I am involved with.” However, she raises a major

concern about the nature of assessment and what it means to assess a

stu-dent’s language ability In addition, although teachers recognised the

intellectual debate about the authenticity and effi ciency of assessment,

they do not see a viable alternative They focus therefore on how to develop

assessment tools that best serve the interests of the students Assessment

was seen as a learning opportunity and not only as a demonstration of

pro-fi ciency These concerns are some of what Shohamy (2001) views as the

main aims of the critical language-testing agenda Teachers did not

explic-itly invoke students’ rights and testers’ obligations, but they repeatedly

stated their attention to and concern about the quality of their students’

educational experience

Technical and Pedagogical Concerns

The focus on the pedagogical and the practical side of assessment was

echoed by the majority of teachers In the words of one participant,

test-ing is a “complex matter [for] there is not a one-size-fi ts-all solution”;

stu-dents may prefer different assessment methods While aware of the

importance of reliability and content validity, some teachers stressed that

“tests should be a motivating factor for students and, to achieve this,

[a test] must be designed to be a learning as well as assessment

experi-ence” (see Leung & Mohan, 2004) It was also clear from a number of

responses that teachers were speaking their minds regardless of

institu-tional practices and mechanisms in place Their formal knowledge,

expe-rience, and educational principles informed their views and their interest

in the educational experience of their students These views often clashed

with some of the assessment practices put in place by testing committees

and programme directors, as expressed by Pamela:

Tests should be designed so that the format (question types, test structure,

test timing, etc.) doesn’t get in the way of the students showing their “true”

abilities/profi ciency/achievement In light of that, students should be

exposed, several times at least, to whatever format is chosen Moreover,

the tests must refl ect the test specifi cations which, in turn, refl ect the

cur-riculum designed in light of students’ needs

In this study, contrary to Rea-Dickins’ (2004) observation that

teach-ers often prioritize formal procedures and “underplay the

observation-driven approaches to assessment which are strongly in evidence in this

everyday classroom practice” (p 249), teachers were clearly aware of the limitations of the current formal procedures and were willing to chal-lenge them

CONCLUSION

The analysis revealed a gap between teachers’ philosophies and their practices Overall, teachers were not involved in assessment-related decision-making processes They had little voice in this important element of the curriculum and were marginalised within a top-down managerial approach

to assessment Most teachers wanted a more effective role and expressed frustration at being ignored This echoes fi ndings in the literature that teachers fi nd themselves at the confl uence of different assessment cul-tures and faced with signifi cant dilemmas in their assessment practices: sometimes torn between their role as facilitator and monitor of language development and that of assessor and judge of language performance as achievement (Rea-Dickins, 2004, p 253)

With years of experience in the Gulf, many of these teachers have devel-oped substantial knowledge of the culture of their students and their ways

of learning It is this knowledge and experience that institutions need to capitalise on if the situation is to change to a more conducive environ-ment for teaching and learning Local expertise needs to be recognised and continuously supported One way of doing so is by showing more pro-fessional respect to teachers’ knowledge and views and by providing them with professional development opportunities Despite being mostly voice-less in decisions on assessment matters, the participants in this study voiced major concerns about the nature of assessment While recognizing the importance of standardized and reliable measurement, they stressed the role of classroom-based teacher assessment that can be used for learn-ing purposes and called for its incorporation into the curriculum

THE AUTHORS

Salah Troudi teaches applied linguistics and language education at the Graduate School of Education at the University of Exeter, England His research interests are in the areas of teacher education, critical applied linguistics, and language policies He coordinates the doctor of education in TESOL program in Dubai

Christine Coombe has a doctorate in foreign/second language education from the Ohio State University in the United States She is currently a faculty member at Dubai Men’s College in the United Arab Emirates Her research interests are assessment, teacher evaluation, teacher effectiveness and leadership skills in English language teaching

Mashael Al-Hamly is an associate professor of applied linguistics at Kuwait University,

Kuwait She has a doctorate in computer-assisted language learning (CALL) from

the University of East Anglia in England She has published in regional as well

inter-national journals in the areas of CALL, testing, and translation studies

REFERENCES

Addamigh, K (2006) Construct validity of foreign language tests In C Coombe,

P Davidson, & D Lloyd (Eds.), Proceedings of the 7th & 8th Current Trends in English

Language Testing Conference (pp 55–72) Dubai: TESOL Arabia Publications

Al-Busaidi, S (2007) Assessing the active and passive vocabulary knowledge of EFL

students in the Sultanate of Oman In A Jendli, S Troudi, & C Coombe (Eds.),

The power of language: Perspectives from Arabia (pp 114–127) Dubai: TESOL Arabia

Publications

Al-Hamly, M., & Coombe, C (2005) To change or not to change: Investigating the

value of MCQ answer changing for Gulf Arab students Language Testing, 22 ,

509–531

Chang, C (2005) Oral language assessment: Teachers’ practices and beliefs in Taiwan

colle-giate EFL classrooms with special reference to Nightingale University Unpublished

doc-toral dissertation, University of Exeter, England

Cheng, L., Rogers, T., & Hu, H (2004) ESL/EFL instructors’ classroom assessment

practices: Purposes, methods, and procedures Language Testing, 21 , 360–389

Davison, C (2004) The contradictory culture of teacher-based assessment: ESL

teacher assessment practice in Australia and Hong Kong secondary schools

Language Testing, 21 , 305–334

Dörnyei, Z (2007) Research methods in applied linguistics Oxford: Oxford University

Press

Farah, B (2007) Students’ voices on their experiences with a high stakes language

test In S Midraj, A Jendli, & A Sellami (Eds.), Research in ELT contexts (pp 252–

270) Dubai: TESOL Arabia Publications

Gamaroff, R (2006) What do test scores mean? In C Coombe, P Davidson, & D

Lloyd (Eds.), Proceedings of the 7th & 8th Current Trends in English Language Testing

Conference (pp 87–89) Dubai: TESOL Arabia Publications

Holliday, A (2001) Doing and writing qualitative research London: Sage

Lalik, R., & Potts, A (2001) Social reconstructivism as a framework for literacy

teacher education In C M Roller (Ed.), Learning to teach reading: Setting the research

agenda (pp 119–135) Newark, DE: International Reading Association

Lanteigne, B (2008) Using advertisements in test item writing In A Jendli, C

Coombe, & S Troudi (Eds.), Best practices in English language teaching (pp 345–

354) Dubai: TESOL Arabia Publications

Leung, C., & Mohan, B (2004) Teacher formative assessment and talk in classroom

contexts: Assessment as discourse and assessment of discourse Language Testing,

23 , 335–359

Pennycook, A (1994) The cultural politics of English as an international language

London: Longman

Pennycook, A (2001) Critical applied linguistics: A critical introduction London: LEA

Rea-Dickins, P (2004) Understanding teachers as agents of assessment Language

Testing, 21 , 249–258

Richards, K (2003) Qualitative inquiry in TESOL London: Palgrave

Ritchie, J., & Lewis, J (2003) Qualitative research practice: A guide for social science

stu-dents and researchers London: Sage

Shohamy, E (2001) The power of tests: A critical perspective on the uses of language tests

Harlow, England: Pearson Education

Troudi, S (2007) The effects of English as a medium of instruction In A Jendli,

S Troudi, & C Coombe (Eds.), The power of language: Perspectives from Arabia

(pp 3–19) Dubai: TESOL Arabia Publications

APPENDIX

Open-Ended Questionnaire

Please try to answer these questions in as much detail as you can Use as many additional pages

as necessary

1 How long have you been teaching English in the Gulf?

2 What is your highest educational qualifi cation?

3 What is your personal philosophy about ESL/EFL assessment*?

4 Please describe how students are actually assessed in your current professional context (please provide concrete examples)

5 What grading system is in use in your institution (how is grade divided in terms of percent-age? e.g midterm 30%, fi nal 50%, projects 10%)?

6 What do you think of the current assessment practices in your institution?

7 Do you have any suggestions about how students should be assessed?

8 What are the teachers’ roles in assessment in your context?

9 Ideally how do you see your role in assessing students’ language?

10 Are there any assessment challenges that teachers face in your context?

11 What coping strategies do you utilise when faced with some of the challenges you mentioned?

* Assessment could include testing, measurement, and all other ways of evaluating students’ language