A critical review of current el testing practices at thai nguyen lower secondary school

LIST OF ABBREVIATIONSCLA : Communicative Language Ability CLT : Communicative Language Teaching DOET : Department of Education and Training ESL : English as a Second Language LSS : Lower

VIETNAM NATIONAL UNIVERSITY - HOCHIMINH CITY

UNIVERSITY OF SOCIAL SCIENCES AND HUMANITIES

A CRITICAL REVIEW

OF CURRENT EL TESTING PRACTICES

AT THAI NGUYEN LOWER SECONDARY SCHOOL

SUBMITTED IN PARTIAL FULFILMENT

OF THE REQUIREMENTS FOR THE DEGREE OF

MASTER OF TESOL

Supervisor:Dr V TH PH NG ANH

NHA TRANG September 2005

CERTIFICATE OF ORIGINALITY

I certify my authorship of the thesis submitted today entitled:

A CRITICAL REVIEW

OF CURRENT EL TESTING PRACTICES

AT THAI NGUYEN LOWER SECONDARY SCHOOL

in terms of the statement of Requirements for Theses in Master’s Programmes

issued by the Higher Degree Committee

Nha Trang, September 10th, 2005

RETENTION AND USE OF THE THESIS

I hereby state that I, TR N NG KHÁNH LINH, being the candidate for thedegree of Master of TESOL, accept the requirements of the University relating tothe retention and use of Master’s Theses deposited in the Library

In terms of these conditions, I agree that the original of my thesis deposited in theLibrary should be accessible for purposes of study and research, in accordance withthe normal conditions established by the Library for the care, loan or reproduction

of theses

Nha Trang, September 10th, 2005

I would like to express my deepest gratitude to my thesis supervisor, Dr V

Th Ph ng Anh, who provided insightful discussions, valuable comments,criticisms, and support in the preparation and completion of this thesis

My special thanks go to the organizers of this master course, Mr Lê H u

Ph c, Head of the Department of Post Graduate Studies of Ho Chi Minh CityUniversity of Social Sciences and Humanities; and Mr Nguy n Hu nh t, Head

of the Department of English Linguistics and Literature of the university; and theirstaff members

This thesis would never have been possible without the permission andfinancial support from Mr Lê Thanh Bình, Rector of Nha Trang Teacher TrainingCollege; and Mr Hà V n H nh, Head of the Foreign Language Department of thecollege I would like to express my thanks to these people

I also wish to thank all the staff members at the English Section, Nha TrangTeacher Training College for their help and moral support, in particular Ms.Nguy n Th H ng Liên, and Ms oàn Linh Chi I am also grateful to Mr Nguy nCao Phúc, the Foreign Language Specialist of Khanh Hoa DOET; the Englishlanguage staff; and the Grade-6 students of Thai Nguyen Lower Secondary School,who heartily participated in the research Thanks also to all my friends, especiallyThanh Nhã, Lan Anh, ng Nguyên, Hoàng Mai, H nh Huy n, who havecontinually encouraged me to complete my study

My warmest thanks go to my mother-in-law and my husband, who took allthe family burden during my long absence from home with great understanding, andwhose love has backed me to attain to my aim

Last, but not least, I especially owe my little son for not giving him adequatemotherly care during his infancy to finish my study in Ho Chi Minh City Hissacrifice makes a great contribution to the completion of my post graduate course ingeneral and of this thesis in particular

LIST OF ABBREVIATIONS

CLA : Communicative Language Ability

CLT : Communicative Language Teaching

DOET : Department of Education and Training

ESL : English as a Second Language

LSS : Lower Secondary School

MOET : Ministry of Education and Training

MSOET : Municipal Service of Education and Training

TLU : Target Language Use

TMF : Test Method Facet

English language teaching at LSSs in Khanh Hoa has undergone a reformwith the promotion of the CLT since 1995 and the use of new text books in largescale since 2002 During the renovation much attention has been paid to newteaching techniques but little care has been taken over the testing aspect of the newteaching methodology This thesis takes a critical look at current English languagetesting practices at one of the best schools in Khanh Hoa: Thai Nguyen LSS Thepurpose of the thesis is to find if the neglect of the testing aspect has made testingunable to cater for the course objectives; and, if it is the case, how the backwardtesting has affected the school’s teaching practices

To do this, a study was carried out to investigate the school’s testingpractices at Grade Six - the grade on which the new textbook has been used forthree years, an adequate time for any evaluation All aspects of the testing realitywere examined: (a) the teachers’ perception of the role and principles ofcommunicative language testing; (b) their testing practices (both formal andinformal); and (c) their students’ communicative language ability The methodsemployed in the investigation includes: (1) reference of relevant documents for thecourse’s TLU domain and TLU tasks; (2) a survey on how well the staff understandwhat it means to test language ability communicatively, with questionnaires givenout to 8 staff members having experience in the new teaching program; (3) classobservation for the teachers’ informal testing practice and the washback effect oftheir testing practices on their teaching, with observation sheets tailored to highlightthe evaluation criteria; (4) a systematic analysis of the test papers used during thefirst semester of academic year 2004-2005; and (5) the assessment of thecommunicative language ability of the school’s students - the products of thecurrent teaching-testing practices - through their performance on a communicativeachievement test designed by the researcher

The results of the study shows that communicative testing has NOT been ineffect at Thai Nguyen LSS, the pioneering school in the province’s teaching reform.Even worse, the backward testing practices have caused negative washback to theschool’s teaching: the teachers are either conducting grammar-based classes or

giving extra-curricular evening classes to prepare the students for the tests Theproblems are defined mainly as the consequence of the disorientation of the DOETand MSOET through their test specifications and term-test design Besides, it is thestaff’s superficial understanding of Communicative Language Testing that results intheir misleading informal testing practice The thesis, therefore, makes suggestions

to all the stake-holders of the provincial education for a reform in language testingpractices at LSSs in general and at Thai Nguyen LSS in particular Therecommendations of the study are believed to contribute to a comprehensive reform

in the provincial teaching and help to solve the critical problem of uncontrollableextra-curricular classes in the province

Table of contents

INTRODUCTION TO THE STUDY 1

1 The problem 1

2 Aims of the research and overview of the thesis 4

CHAPTER 1 BACKGROUND TO THE RESEARCH 6

1.1 Description of Grade - 6 English program at Thai Nguyen LSS 6

1.1.1 Teaching approach, course objectives, and time allocation 6

1.1.2 Course book and additional teaching materials 9

1.1.3 Teaching and learning environment 10

1.1.4 Test administration 10

1.2 The English Language Teaching Staff and the Examination Authority 11

1.2.1 The English Language Teaching Staff 11

1.2.2 The examination authority 12

1.3 The students 12

1.4 Summary 12

CHAPTER 2 REVIEW OF RELEVANT LITERATURE 13

2.1 The importance of testing in language teaching and learning 13

2.2 The Communicative Approach to Language Testing 15

2.3 Features of communicative language tests 17

2.4 Communicative Language Ability 20

2.5 Bachman’s framework of Test Method Facets (TMF) 23

2.6 Summary 26

CHAPTER 3 METHODOLOGY 27

3.1 Research questions 27

3.2 Research design 28

3.2.1 For course orientation and course objectives 28

3.2.2 For the school teachers’ perceptions and their teaching-testing practices 29

3.2.3 For the evaluation of test qualities 30

3.2.4 For the students’ Communicative Language Ability 31

3.3 The study 31

3.3.1 Subjects 32

3.3.2 Instruments 33

3.3.2.1 Guiding documents from the course designers and the education authorities 33

3.3.2.2 Survey questionnaires 33

3.3.2.3 Class observation sheets 34

3.3.2.4 The progressive and summative tests in practice at the school 35

3.3.2.5 The communicative test 36

3.3.3 Data collection procedure 38

3.3.3.1 Instruments administration 38

3.3.3.2 Scoring of test results 39

3.4 Summary 39

CHAPTER 4 DATA ANALYSIS AND FINDINGS 40

4.1 Analysis of guiding documents on the new teaching program 40

4.2 Analysis of the teachers’ feedbacks on the questionnaires 42

4.3 Analysis of class observation results 47

4.4 Analysis of the school’s test papers 52

4.5 Analysis of the sample’s performance on the communicative test 61

CHAPTER 5 SYNTHESIS OF FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 66

5.1 Summary of findings from each research method 66

5.1.1 Findings from documents on new orientation of English language

teaching 66

5.1.2 Findings from the teachers’ responses of the questionnaire items 67

5.1.3 Findings from class observation 67

5.1.4 Findings from the analysis of the school’s test papers 70

5.1.5 Findings from the students’ performance on the communicative test 71

5.2 Towards a conclusion 72

5.2.1 The school’s effort to renovate its teaching of English 72

5.2.2 The backwardness of the school’s testing practices 73

5.2.3 The harmful washback the backward testing leaves on the school’s teaching 74

5.2.4 The student’s lack of Communicative Language Ability 75

5.2.5 The deficiency of the new course books 75

5.3 Delimitations and Limitations of the study 76

5.4 Suggestions and Recommendations 77

INTRODUCTION TO THE STUDY

Testing: an important part of every teaching and learning experience.

The study reported in this thesis examines the renovation in English teaching

at Thai Nguyen Lower Secondary School (Thai Nguyen LSS) in Nha Trang, toinvestigate how testing has changed towards the new teaching approach andmethods The aim of this introductory chapter is to identify the problem the thesisattempts to solve, to state the aim of the study, and to present an overview of thethesis

1 The problem

Since 1995 the English language-teaching curriculum in Lower SecondarySchools (LSS) has undergone a process of change and reform This process hasinvolved the promotion of a new teaching methodology and the recent development

of a new curriculum with new course books: the Ti ng Anh 6 since 2002, the Ti ngAnh 7 since 2003 and the Ti ng Anh 8 since 2004 The overall approach in thecourse books is eclectic As claimed by the authors, though, there is a move towards

a more communicative approach, which is exemplified by, for example, the use ofpair work and group work, the emphasis on language skills, and the view ofgrammar and vocabulary as means of communication and information exchange,

not as the objectives of the teaching-learning process (Ti ng Anh 6 - Teacher s

book, page 8 & 11) All these strides have been made to motivate the students to use

the target language competently

It is hoped that this reform in teaching has been well-backed by the way thestudents’ progress and achievement are tested, i.e the students’ competence inusing English must be the supreme goal of any tests, either formal or informal Nolonger can there be tests that only attempt to measure the students’ linguisticknowledge For one reason, it is unfair to the students to teach them one thing andthen measure their ability at another More importantly, it is essential for the success

of the new teaching methodology that test content and testing techniques be fully

congruous with the aims and objectives of the new language-teaching curriculum.Studies have gained facts showing that testing always has some wash back effect (or

‘backwash’ as defined by Arthur Hughes, 1997) on teaching and learning Whenwell-matched tests are given, learning can be enhanced by the learners’ awareness

of the course objectives and the areas of emphasis in the course Such tests can alsohelp teachers to see how effective the process of teaching and learning is

This congruence is especially crucial when we consider the setting of Englishlanguage learning in Vietnamese LSSs, where informal and formal tests in class putgreat pressure on both the student and the teacher, for test results provide thestakeholders with the only information about the students’ competence and theteachers’ professional skills In such a situation, success in tests is the supreme aim

of every student and teacher Therefore, if the tests continuously fail to measurewhat the students have been asked to learn, all the teaching and learning effort thatprecedes the tests become meaningless As a result, the course objectives as quoted

in the course books will soon be abandoned to give way to test content That is whyWeir (C.J.Weir, 1993: 5) insists that language testing in the classroom cannot bedivorced from the course syllabus and the course objectives Testing, which caneven play the role of a navigator for the teachers and the learners, is really animportant part of every teaching and learning experience

Important as it is defined, testing in LSSs in Khanh Hoa is still now anunder-privileged matter, even in the view of the authorities To our knowledge,although the Department of Education and Training (henceforth the DOET) haveheld a training course in testing for LSS teachers during the teaching renovation,they have failed to find ways to check if the teachers can put the theory intopractice No specialists have been asked to look into this aspect of the teachingmatter Also, since the promotion of the new course books, the annual teachingcontests in the province have honored numerous demonstration classes for goodteaching with emphasis on teaching techniques such as how to present newlanguage items, how to manage pair work, group work, or how to exploit visual aids

to boost teaching and learning, etc Yet, never have one found any contests focusing

on testing techniques or test designing Even worse, the effectiveness of the lessons

in the teaching contests has always been evaluated from the students’ performance

on mini tests designed by the judges themselves, not by the teachers Takenseriously, this can be understood to imply that it is not the LSS teachers’ duty todevelop tests that can reflect the degree of effectiveness in their lessons; it is theduty of experts outside the class In a more friendly interpretation, the teachers canlearn a message: so long as other teaching techniques are not mastered, it is not timefor us to think too much of testing techniques Either of these views, though, maylead to neglect in test development

The natural questions for every professional in this situation are: Has therebeen such neglect? Or to put it differently, how is the testing in the provincecharacterized? How communicative, valid, reliable, and appropriate are the testsadministered to the students? Especially, is the assessment system related directly tothe aims and objectives of the course? Or to what extent have the tests administeredreflected students’ achievement in the course objectives?

A reliable answer to these questions is clearly of great importance andgeneral interest to every stakeholder in the situation: the DOET, the teaching staffs

of LSSs, and the students’ parents Besides, such a study of the problem can provideimplications for two other forces One is the syllabus designers The study can grantthem a chance to look back into their course books from another perspective to seehow much the course content is consistent with the course objectives This ispossible because test writers are asked by course designers to ‘base achievement

tests directly on the course content without skipping the course objectives and course level at the time of testing.’ (Ti ng Anh 6 - Teacher s book, page 13).

Therefore, if the test content is inconsistent with the course objectives, or if thecourse objectives are unrealistic, the tests will reveal a failure to achieve theobjectives The last but not the least beneficiary is the teacher trainers at Nha TrangTeacher Training College They can use the study results as valuable empirical facts

to tailor their pre-service training courses, especially the testing course, for greaterpracticality With such an adaptation, not only can they fill the gap between thetheory learnt at college and the real tasks to be practiced at LSSs, but they can also

make great contribution to the province’s renovation of English language teaching

by producing new generations of teachers with better qualification in testing

2 Aims of the research and overview of the thesis

As an initial attempt to investigate the reality of English language testing inthe province, the study in this thesis was carried out to examine the testing ofEnglish 6 at Thai Nguyen LSS to see how much it is congruent with the courseobjectives

There are two reasons accounting for the narrowing in the research scope.Firstly, an insight into the testing at every grade of all the LSSs in theprovince would be impossible for the scale of a thesis Hence, there is a need tochoose a good sample among the population; and Thai Nguyen LSS appears to be aperfect one For one thing, it is one of the best LSSs in Khanh Hoa, which hasalways been pioneering in the reform of the provincial education, especially in therecent promotion of the new course books A look into the school can provide afairly clear understanding of the extent to which the province have reached in therenovation More importantly, Thai Nguyen LSS is a permanent counterpart of theNha Trang Teacher Training College, where the researcher works as a teacher ofEnglish language teaching methodology Every year, her second and third-yearstudents are sent to the school for a six-week period of teaching practice On onehand, this close and long-rooted relationship between the two schools wouldfacilitate the collection of data, which can thus allow high reliability for theresearch On the other hand, the choice of the school as the subject would enable thestudy to have an immediate meaning: making benefit for the teachers of bothschools, and thus, would be able to boost their cooperation to a higher level ofefficiency Personally, with this research the researcher hoped to learn how to fillthe gap between theory and practice in the teaching and training of teachers in theprovince

Secondly, only the testing of Grade 6 at Thai Nguyen LSS was chosen forinvestigation This was because the course book of this grade has been put into usefor three years, an adequate time for a mature (1) in the understanding and

implementation of the syllabus writers’ ideas, and (2) in the practice of the newteaching approach, the Communicative Language Teaching (CLT).

The aim of the research is to seek empirical evidence to support theresearcher’s presuppositions on the nature of English language testing at Grade 6 ofThai Nguyen LSS, i.e (1) traditional testing practice is still dominating due to theauthorities’ neglect of the testing aspect in the course of teaching renovation; (2)there is a mismatch between tests focus and the course objectives for traditionallanguage testing emphasizes the importance of linguistic knowledge, notcommunication skills; (3) traditional language testing may have deviatedcommunicative language teaching; and (4) Grade-6 students at this school must be

in lack of communication skills

Data were collected from four sources: (1) the guiding documents from theMOET and DOET on the course orientation and objectives, especially on testing;(2) the teaching staff including their demographic data (i.e., gender, age, traininginstitutions, teaching experience), their opinions of the new teaching syllabus, theirperceptions of language teaching and testing, and their implementation of theperceptions (i.e., their real lessons); (3) the content characteristics of the progressand final achievement tests administered during the first semester of the academicyear 2004-2005; and (4) sample students’ performance on a communicative testdesigned by the researcher

Both qualitative and quantitative instruments were used to collect data, such

as questionnaires, documentation, observation, diagnostic tests, and statistics

The subjects of the study were 11 teachers of English at Thai Nguyen school,especially the three who were in charge of Grade 6 at the time; and 60 studentsrandomly taken from the twelve Grade 6 classes;

The thesis consists of 5 chapters, in addition to an introductory chapter (thepresent chapter) identifying the problem and providing an overview of the thesis

Chapter I, II, and III respectively provide the background of the study, review the

relevant literature, and present the methodology employed in the study Chapter IV

reports the results of the study and present discussions of the findings from thesources of data It is hoped that to this part of the thesis, all the shortcomings as well

as the positive aspects of the present testing system will be pointed out Chapter V

ends the thesis with a summary of the findings, pinpoints the limitations of thestudy, draws conclusions about the implications of the findings, and makessuggestions for future research

CHAPTER 1 BACKGROUND TO THE RESEARCH

This chapter describes the context in which the research is placed, andprovides background information crucial for a general understanding of the problemstudied The chapter is organized into three sections The first section is adescription of the existing English program for Grade 6 at Thai Nguyen LSS, thefocus of this research Section 2 presents the essential information about the Englishteaching staff and the examination board Finally, section 3 provides a brief report

on the characteristics of the 6-Graders at Thai Nguyen School

1.1 Description of Grade - 6 English program at Thai Nguyen LSS

This section presents a description of the current English program for Grade

6 at Thai Nguyen LSS The following information about the program will bediscussed:

(1) course objectives, teaching approach and time allocation;

(2) course book and additional teaching materials;

(3) teaching and learning environment;

(4) testing administration

1.1.1 Teaching approach, course objectives, and time allocation

The Grade-6 classes examined in the research were taking the new Englishcourse, which had been in practice nationwide for 3 years The course results fromthe nation’s recent innovation in English language teaching for CommunicativeLanguage Teaching (CLT), which is characterized by theme-based textbooks andlearner-centredness

Grade-6 course, as described in its Teacher’s book, is the first out of four (4)levels of a teaching program that aims to satisfy the following primary objectives:

Hình thành và phát tri n h c sinh nh ng ki n th c, k n ng c

n v ti ng Anh và nh ng ph m ch t trí tu c n thi t ti p t c

c ho c i vào cu c s ng lao ng.

[ To form and develop students fundamental knowledge and skills

in English as well as the intellectual qualities essential for their

future study or work ]

(Ti ng Anh 6 - Teacher s book: 5; English Translation mine)

From this overall aim, four specific goals are drawn, which serve as therequirements for students’ proficiency upon finishing the LSS English curriculum:

• Obtaining fundamental, minimal, and somewhat systematic

knowledge of up-to-date teenage-level English in use;

• Having basic skills in Listening - Speaking - Reading - Writing to

use English as a simple means of communication;

• Having general understanding of English-speaking countries

cultures;

• Forming language-learning and mentality-developing skills,

which can support students use of their mother tongue and form a more thorough language competence in them.

(Ti ng Anh 6 - Teacher s book: 5-6; English Translation mine)With such objectives, Grade-6 course has been planned as follow:

- Time allocation: 3 class hours/week×35 weeks = 105 class hours

Present simple tense (to be) Pronouns (this, that, these, those)

Yes / No questions Present simple tense (to have)

Imperatives Nouns (singular / plural) Descriptive adjectives

Themes / Topics Competencies Language

Wh- questions Modal verbs (can, must…) Prepositions (in, at)

4 Health

Parts of the body

Foods and drinks

Determiners (some, any)

6 The world around us.

Weather and seasons

Countries and languages

Animals and plants

Describing weather, seasons, animals and plants.

Talking about countries and languages

Comparatives Names of countries and nationalities / languages Adjectives

Figure 1: Teaching focus for Grade-6 syllabus

- Teaching orientation:

Classroom activities must be designed for students’ active participation, throughwhich communicative goals are reached The teacher’s role is to present, model newlearning items and facilitate pair work, group work, role-plays

- Testing orientation:

Students’ communicative skills (receptive and productive) are the major criteria for

assessment; Test content must match the course objectives and course level;

Communicative competence is measured through communicative activities(Listening - Speaking - Reading - Writing, with the first three taking priority).Language elements are only measured through skills performance; various modes ofassessment should be applied - through progress tests, summative tests, andstudents’ self-assessment

(Source: English Course Specifications, 2000:7, issued together with Decision No 2434/QD /BGD&DT - THCS dated July 8 th , 1999)

1.1.2 Course book and additional teaching materials

The newly designed course book used for Grade 6 is the Ti ng Anh 6, a

theme-based one The six themes (You and me, Education, Community, Health,

Recreation, The world around us) are the basis for the choice of the topics,

communicative activities and the grading of language items It is believed anddemanded by the writers that activities for communicative skills, languagefunctions, grammar and vocabulary be all harmoniously combined and exploited to

their full extent (English Course Specifications, 2000: 4).

The writers’ viewpoints are also characterized through the following:

- A skills-based approach to language teaching is highlighted: new language itemsare introduced through natural communicative contexts, with controlledaccuracy-based activities leading to fluency-based communicative practice Inthis way, there is a link between grammatical forms and communicativefunctions Grammar is thus presented for communication purposes, not forlinguistic knowledge;

- Authenticity is obtained through a selection of real-life materials andcommunication contents so that students can use the language outside the class;

- There is an emphasis on the practice of communicative skills, especiallyListening, Speaking, and Reading;

- Each unit is structured and organized to cover the practice of both language itemsand skills Three different exercise types are provided in each unit: (1) Exercises

for presenting language items (Listen and Repeat, Listen and Read, or Read), (2) Exercises for consolidation (Remember, Grammar Practice, or Grammar), and (3) Skills-based exercises (Write, Match and Write, Complete the sentences,

Practice with a partner, Read Then answer the questions, Listen and Answer…).

- Color pictures throughout the book are deliberately designed to facilitate not onlythe introduction of language and communicative contexts but also the practice ofthe four skills

In parallel with the student book is a workbook that provides furtherexercises on grammar, vocabulary and language functions through reading andwriting activities After every three units in the workbook, there is a test for

students’ self-assessment The new course book and workbook are the mainlearning materials Most classroom activities are generated from the books.

Besides, the teachers have access to a Teacher’s book which providesguidelines and models for their lesson plans Other reference books can be used atthe teachers’ will Finally, a set of book-based laminated A3-size pictures and anaudiocassette with a cassette player are supplied for use as classroom visual aidsand listening practice

1.1.3 Teaching and learning environment

This academic year 2004-2005, Grade 6 at Thai Nguyen LSS consists of 12classes with roughly 45 students each Three teachers are in charge of the 12classes, giving each class three class hours per week

1.1.4 Test administration

Students’ achievement is assessed through a number of formative andsummative tests they have throughout each of the two semesters of the school year.The tests are scored to yield grade-based feedback The grades are then multiplied

by proper coefficients previously designated to each test, next summed up and lastlydivided by the coefficient sum to give the final average grade of two digits Fivegrades and over constitute a pass The lower grades mean a fail and the studentshave to take a summer course and re-sit for an extra exam

The following table shows how the testing practises

Test types Coefficients Quantity Requirements

Oral test &

Designed by the DOET specialist.

Table 1: English Language Testing at Thai Nguyen LSS

1.2 The English Language Teaching Staff and the Examination Authority

1.2.1 The English Language Teaching Staff

The English teaching staff at Thai Nguyen LSS consists of 11 members, agedfrom 27 to 50, one of whom is male A majority (8/11) have teaching experience ofover 10 years Two have 5-10 years’ experience, and the youngest has beenteaching for three years All of them had formal tertiary training in languageteaching in different educational institutions inside Vietnam Six out of eleven(6/11) graduated from senior colleges like Dalat University or Hue College ofPedagogy; others (5/11) from Nha Trang Teacher Training College Interestinglyenough, except for one who obtained her Associate degree in 1989, others havetheir degrees granted just recently, in 1999, 2000 or 2003 This fact reveals twothings: Firstly, most of the teachers have formally learnt about the CommunicativeLanguage Teaching because the approach has been in practice at colleges for morethan ten years now; Secondly, those with teaching experience of over 5 years butnewly-conferred degrees must have taken in-service training The number for thisgroup is nine Overlooking our prejudice on the quality of in-service trainingprograms, from this we can see the school’s and the teachers’ effort to update theirknowledge However, we must be aware of the fact that for the previous and longerpart of their lives, they had been taught and then they themselves had taught to theirstudents with traditional teaching methods Such a learning and teachingbackground can be a great disadvantage to them if they are dependent too much onthe former

The following table summarizes general qualifications of the Englishlanguage teaching staff at Thai Nguyen LSS

Qualifications / Training experience No of staff

Having Associate degree of English 5/11 (with 4 from recent in-service training)

Having attended workshops in teaching

Having taken short-termed training courses

Having participated in seminars on new

Table 2: General qualifications of the English teaching staff at Thai Nguyen LSS

Besides the information about the teachers’ highest degrees, the table alsogives information of other training forms the teachers have taken The high ratio ofteachers taking part in short-termed CLT training courses and seminars canadequately account for the teachers’ excellent responses to the questionnaire ontheir teaching beliefs, which will be addressed later in Chapter III.

1.2.2 The examination authority

Term-ending tests at LSSs are designed by the English language specialists

of the DOET or MSOET In case the MSOET takes the responsibility, the DOET’sspecialist has the duty to revise the tests The present specialist of the DOET is amember of the Nation’s English Language Teaching Council, a training specialist ofSEMEO, and a co-coordinator of the Learntech Excels Programme He wasformally trained in testing and has been to the UK, Australia, and the Philippinesseveral times to attend training courses or workshops on English Language Testing.With such a background, he can be supposed to be expert in CommunicativeEnglish Language Testing

1.3 The students

6-Graders at Thai Nguyen School are mostly eleven-year-old Nha Trangnatives who come from different Primary schools in Nha Trang Officially, they hadnever taken any English course at school before entering Thai Nguyen LSS Many,however, have spent some time attending Children’s English courses at Languagecenters Therefore, English is quite familiar to some, while rather new to others

The 545 six-graders of this school year are divided into 12 classes, withapproximately 45 students in each class They learn English as a compulsory subjectthey have to pass to move to the upper class Tests and test results are, therefore,their strongest motivation

1.4 Summary

This chapter has provided relevant information regarding the situation ofEnglish teaching environment at Thai Nguyen LSS in Nha Trang as a whole and forGrade 6 in particular After reading this chapter, the readers may have an overview

of the course objectives, the teaching and testing focus as designated by the coursedesigners The chapter also provides a clear image of the course practitioners,namely the teaching staff and the English language testing specialists Thedescription of the students’ characteristics adds final details to the picture ofteaching and testing reality at Thai Nguyen LSS

CHAPTER 2 REVIEW OF RELEVANT LITERATURE

While chapter 1 describes the context in which the research is placed, andprovides background information crucial for a general understanding of the problemstudied, this chapter discusses major issues in communicative language testing Thediscussion is meant to provide a theoretical basis for the design of the study and theanalyses of the data The chapter addresses the following issues: (1) the importance

of testing in language teaching and learning; (2) the Communicative LanguageTesting; (3) test qualities; (4) the concept of communicative language ability; and(5) Bachman’s framework of Test Method Facets (TMF)

The first three sub-sections in this chapter provide a sketch of relevantliteratures in the field of language testing in general and in communicative languagetesting in particular (issues 1, 2 &3) The review is made for better understanding of(a) the study instruments including the questionnaire, the class observation sheet,and the communicative achievement test as presented in the following chapters ofthe thesis, and (b) the analyses of the data collected by using these instruments Onthe other hand, the issues presented in the last two sub-sections of the chapter(issues 4 & 5) are essential for the analysis of the test papers under evaluation in thestudy and the design of the achievement test to assess the students’ languageperformance (see section 3.2 of the next chapter for the design of the study) Thediscussion of issue 4 is essential for characterizing the ‘what’ of language testing,while the framework presented in issue 5 is useful for the description of the ‘how’

of language testing A clear understanding of all the components of these twoaspects of a test can help us identify all factors that can affect test performance, andtherefore, is of great importance for analyzing current tests or designing,developing, and using new tests

2.1 The importance of testing in language teaching and learning

The testing discussed in this research confines itself to classroom testing,which is distinguished from that of external examination by Heaton in that it is notfor the purpose of selection , but mainly for “enabling the teachers to incerease theirown effectiveness by making adjustments in their teaching to enable certain groups

of students or individuals in the class to benefit more” (Heaton 1989:6) Theclassroom tests are, therefore, achievement tests, which are defined by TimMcNamara as to be “associated with the process of instruction” and “shouldsupport the teaching to which they relate”(McNamara 2000:6-7)

Previously, classroom testing was often expected as only to follow teaching.Davies (1968:5 as cited in Hughes 1968:2) said that ‘the good test is an obedientservant since it follows and apes the teaching.’ However, later research has provedthat ‘the proper relationship between testing and teaching is that of partnership’(Hughes 1989:2) This is because testing can have effect on teaching and learning:the washback effect Washback (the term commonly used, for example in Aldersonand Wall 1993, McNamara 2000) or backwash (as named in Hughes 1989) issignificant in teaching and learning If a test is regarded as important, as it is inVietnamese educational context, then preparation for it can come to dominate allteaching and learning activities With such a role, washback can be harmful orbeneficial For example, if the test content and testing techniques fail to reflect thecourse objectives, then there is likely to be harmful washback as the later course ofteaching and learning will be distorted to suit the test focus Whereas, there may beoccasions when teaching is poor or inappropriate but testing is based directly on thelearners’ needs, then testing can exert beneficial washback: classes will beconducted differently to match the focus of the tests, and as thus both teaching andtesting can cater for the students

With an understanding of the beneficial or harmful washback testing has onteaching and learning, teachers have to demand testing ‘to be supportive of goodteaching and, where necessary, exert a corrective influence on bad teaching’(Hughes 1989) If teachers can do this with good testing practices, they will find anumber of benefits both to them and to their students

According to Madsen, well-made tests of English can help students in manyways First, such tests can help create positive attitudes toward instruction by givingstudents a sense of accomplishment and a feeling that the teacher’s evaluation ofthem matches what has been taught A second way that good tests can benefitstudents is by helping them master the language by requiring them to study hard,

emphasizing course objectives, giving them a series of definite goals and showingthem where they need to improve (Madsen 1986:3; Doff 1988: 257) On theteachers’ part, these tests tell them what the students can and cannot do, howsuccessful the teaching has been, to what extent they have attained the courseobjectives, and what areas need to be taught in the future (Heaton 1989: 6-7).

As thus, in classroom situation it can be said that testing is always essentialfor better learning and teaching Tests enable us to measure students’ progress in amore individualized way because individual weaknesses can be obscured whenstudents are working together in the classroom context

2.2 The Communicative Approach to Language Testing

Madsen (1983) states that language testing has evolved through three majorstages, which reflect people’s attitudes towards the goals of language teaching andlanguage learning These stages are summarized as follows:

1 The Intuitive Stage focuses on subjective testing and is dependent onpersonal impressions of the teachers

2 The Scientific Stage stresses objective evaluation focusing on languageusage

3 The Communicative Stage emphasizes evaluation of language use ratherthan usage

Testing language has traditionally taken the form of testing knowledge aboutlanguage, usually the testing of knowledge of vocabulary and grammar However,there is much more to being able to use language than knowledge about it DellHymes proposed the concept of communicative competence He argued that aspeaker could be able to produce grammatical sentences that are completelyinappropriate In communicative competence, he included not only the ability toform correct sentences but to use them at appropriate times Since Hymes proposedthe idea in the early 1970s, it has been expanded considerably, and various types ofcompetencies have been proposed However, the basic idea of communicativecompetence remains the ability to use language appropriately, both receptively and

productively, in real situations It is Hymes’s theory of communicative competencethat brings forth new concepts of how to test communicative competence.

The common concepts needed in communicative testing include reliability,

validity, practicality, and authenticity They fall under the heading of desirable test

characteristics Marshall and Hales (1972) point out that any test that is to be usedeffectively as a measuring instrument should be reliable, valid, authentic, andpractical They warn that a drawback in any of these test attributes can render a testfutile

Reliability has to do with test consistency Two tests should give evidence

that they are likely to produce the same results when taken at different times by thesame or similar students That is, students who obtain high scores on one set ofitems also obtain high scores on other sets of equivalent items, and those who have

a low score on one set of items also have a low score on other sets of items,(Scannel and Tracy, 1975)

Validity in testing refers to whether the test measures what it claims to

measure, and whether it measures what was taught For example, a test which isdesigned to determine the extent to which a particular group of students havemastered specific algebraic concepts will not be valid when administered to adifferent group of students with the intent to determine their performance inElizabethan literature Similarly, a test of English as a Second Language (ESL) isnot valid for students learning translation theory, (Heaton, 1995) Questionspertaining to the validity of a test include what the test measures, whether itmeasures what it wants to measure, and whether it measures what was taught.Henning (1987) claims that a good language test should consider how relevant islanguage behavior being tested to the meeting of communicative needs and whether

or not the users of the test will accept its content and format

Practicality or usability is the third important attribute of a good test It

involves the economical use of time and expenses in test construction, testadministration, and test scoring A test may be highly reliable and valid and yet not

be practical for use in a school-testing program

Another equally important feature of a good test is authenticity In

communicative testing, authenticity is a key element in the designing of materialsand test items It means assessing language behavior by observing it in real, or atleast realistic, language-use situations that should be as authentic as possible,(Gronlund, 1985)

To sum up, much has recently been written about communicative languagetesting Discussions have focused on the desirability of assessing the ability thattakes part in the acts of communication All interests assume that it iscommunicative competence that teachers want to test Tests should therefore assessthe learner’s communicative behavior and not be based on linguistic items alone Intaking communicative tests, student’s performance should be measured not only interms of formal correctness, but also primarily in terms of interaction, for theconcern is not how much the students know, but how well they can perform

2.3 Features of communicative language tests

The communicative approach is based on the premise that language is firstand foremost a tool for communication From this perspective, tests designed toassess student proficiency can be tailored to include items which possibly measurethe students’ communicative ability in all levels of language The typicalcharacteristics of a communicative language test have been discussed in numerousstudies, eg Madsen 1983; Brown (1987); Heaton 1989; Kitao 1996; Rajan 1998;Thanh Ha 1999; etc Take, for example, Brown’s elaboration on the characteristics

of a communicative language test:

A communicative test has to meet some rather stringent criteria It

has to test for grammatical, discourse, sociolinguistic, and

illocutionary competence as well as strategic competence It has to

be pragmatic in that it requires the learner to use language

naturally for genuine communication and to relate to thoughts and

feelings, in short, to put authentic language to use within a context.

It should be direct (as opposed to indirect tests which may lose

validity as they lose content validity) And it should test the learner

in a variety of language functions (Brown 1987: 230)

An important observation in this quotation is that in testing communicativeperformance, test items should measure how well students are able to engage inmeaningful, purposeful, and authentic communicative tasks Students must have agood performance linguistically and communicatively That is, they must have agood command of the components involved in communication The best exams inthis communicative era, Madsen (1983) comments, are those that combine thevarious sub skills necessary for the exchange of oral and written ideas He assertsthat communicative tests need to measure more than isolated language skills, tocomprehensively indicate how well a person can function in another language.

To put it another way, communicative language tests are intended to be ameasure of how the testees are able to use language in real life situations In testingproductive skills, emphasis is placed on appropriateness rather than on ability toform grammatically correct sentences In testing receptive skills, emphasis is placed

on understanding the communicative intent of the speaker or writer rather than onpicking out specific details Tests of communicative spoken ability should reflectnormal spoken discourse and give the candidate chances to initiate There shouldalso be an element of unpredictability As Morrow (1981:16) points out, “Theprocessing of unpredictable data in real time is a vital aspect of using language.”

In addition, receptive and productive skills should be tested in combination(integration) in communicative testing, so that the testee must both comprehend andrespond to real tasks In real life, the different skills are not often used entirely inisolation Students in a class may listen to a lecture, but they later need to useinformation from the lecture in a paper In taking part in a group discussion, theyneed to use both listening and speaking skills Even reading a book for pleasure may

be followed by recommending it to a friend and telling the friend why you liked it

The “communicativeness” of a test might be seen as being on a continuum.Few tests are completely communicative; many tests have some element ofcommunicativeness For example, a test in which testees listen to an utterance on atape and then choose from among three choices the most appropriate response ismore communicative than one in which the testees answer a question about themeaning of the utterance However, it is less communicative than one in which the

testees are face- to-face with the interlocutor (rather than listening to a tape) and arerequired to produce an appropriate response.

Communicative tests are often very context-specific A test for testees whoare going to British universities as students would be very different from one fortestees who are going to their company's branch office in the United States If at allpossible, a communicative language test should be based on a description of thelanguage that the testees need to use Though communicative testing is not limited

to English for Specific Purposes situations, the test should reflect thecommunicative situation in which the testees are likely to find themselves In caseswhere the testees do not have a specific purpose, the language that they are tested

on can be directed toward general social situations where they might be in aposition to use English

This basic assumption influences the tasks chosen to test language incommunicative situations A communicative test of listening, then, would test notwhether the testee could understand what the utterance, "Would you mind puttingthe groceries away before you leave" means, but place it in a context and see if thetestee can respond appropriately to it

If students are going to be tested over communicative tasks in anachievement test situation, it is necessary that they be prepared for that kind of test,that is, that the course material covers the sorts of tasks they are being asked toperform For example, you cannot expect testees to correctly perform such functions

as requests and apologies appropriately and evaluate them on it if they have beenstudying from a structural syllabus Similarly, if they have not been studyingwriting business letters, you cannot expect them to write a business letter for a test

The final aspect of communicative language test we would like to address isthat of assessment Tests intended to test communicative language are judged on theextent to which they simulate real life communicative situations rather than on howreliable the results are In fact, there is an almost inevitable loss of reliability as aresult of the loss of control in a communicative testing situation If, for example, atest is intended to test the ability to participate in a group discussion for studentswho are going to a British university, it is impossible to control what the other

participants in the discussion will say, so not every testee will be observed in thesame situation, which would be ideal for test reliability However, according to thebasic assumptions of communicative language testing, this is compensated for bythe realism of the situation.

In sum, any test that meets the requirements of a communicative test mustincorporate in itself such features as “context-specific”, “integrative”, “direct”,

“skills-based”, “use- emphasising”, “authentic input”, and “learners’ need-based”

2.4 Communicative Language Ability

The reason why communicative language tests can be set apart from otherforms of testing is that they are based on advanced models of communicativelanguage ability (or Communicative competence) proposed by (a) Canale andSwain, and (b) Bachman

According to Spolsky (1989:140):

Language tests involve measuring a subject s knowledge of, and

proficiency in, the use of a language A theory of communicative

competence is a theory of the nature of such knowledge and

proficiency One cannot develop sound language tests without a

method of defining what it means to know a language, for until you

have decided what you are measuring, you cannot claim to have

measure it

Several attempts have been made to define what it means to know alanguage, but we only propose to discuss two of the more influential models Thework of Canale and Swain began in an attempt to “determine the feasibility andpracticality of measuring what we shall call the ‘communicative competence’ ofstudents enrolled in ‘core’ French as a second language programmes in elementaryand secondary schools in Ontario” (Canale and Swain 1980:1) Canale and Swainproposed a set of three competences which combine to produce communicativecompetence The first, grammatical competence, included “knowledge of lexicalitems and rules of morphology, syntax, sentence grammar, semantics andphonology” (Canale and Swain 1980:29) The second was sociolinguisticcompetence This was made up of “sociocultural rules of use and rules of discourse”

(Canale and Swain 1980:29) The third competence they proposed was strategiccompetence, which related to “verbal and non-verbal communicative strategies thatmay be called into action to compensate for breakdown in communication due toperformance variables or to insufficient competence” (Canale and Swain 1980:29).

In 1983 Canale updated this model by subdividing sociolinguistic competence,which still relates to sociocultural rules, but he introduced a further competence,that of discourse Discourse competence concerns mastery of cohesion andcoherence in different genres

The main implication this model had for communicative language testingwas that since there was a theoretical distinction between competence andperformance, the learner had to be tested not only on his/her knowledge oflanguage, but also on his/her ability to put it to use in a communicative situation(Canale and Swain, 1980)

Bachman’s framework (1990) was an extension of earlier models “in that itattempt to characterize the processes by which the various components interact witheach other and with the context in which language use occurs” (Bachman 1990:81).The interaction of the components in the framework is presented in figure 2 on thenext page

The framework included three components: language competence, strategic

competence, and psycho-physiological mechanisms (Bachman 1990) Language

competence comprises two further competencies, organizational competence andpragmatic competence, each of which he further breaks down, with organizationalcompetence covering grammatical and textual competence, and pragmaticcompetence covering illocutionary and sociolinguistic competence Bachmandefined language competence as “a set of components that are utilized incommunication via language” (Bachman 1990:84)

Strategic competence consists of three components: assessment, planningand execution It is the mental capacity to implement language competenceappropriately in the situation in which communication takes place, and involvessociocultural and real world knowledge Psycho-physiological mechanisms refer tothe neurological and psychological processes involved in producing andcomprehending language.

Figure 2: Components of communicative language ability

in communicative language use (Bachman 1990:85)

One notable advance on the Canale and Swain model is that Bachmanacknowledges that test design and scoring might have a significant effect on thetestee’s performances as a result of strategic competence Certain tasks lendthemselves to use of strategic competence to compensate for a lack of competence

in other areas, while tests which are assessed according to the “practical effect ofthe language performance” (Bachman 1990:105) may be affected by the strategiccompetence factor

PSYCHOPHYSIOLOGICAL MECHANISMS

STRATEGIC COMPETENCE

CONTEXT OF SITUATION

LANGUAGE COMPETENCE Knowledge of language

KNOWLEDGE OF STRUCTURE Knowledge of the world

2.5 Bachman’s framework of Test Method Facets (TMF)

The two models of Canale & Swain’s and Bachman’s have provided a usefulframework for designing the ‘what’ of communicative language tests (Weir 1990)with the distinguished characteristics as mentioned in the earlier part, the subsection2.3 However, it is Bachman’s belief that performance on language tests is not onlyaffected by the content factors of the tests but also by the way test tasks aredesigned He insists, “if we are to develop and use language tests appropriately, forthe purposes for which they are intended, we must base them on clear definitions ofboth the abilities we wish to measure and the means by which we observe andmeasure these abilities.” (Bachman 1990:81) That is why together with theframework describing the components of communicative language ability, hedevelops the framework of TMF for test makers’ better understanding of how thedesign of a test task or a test as a whole can affect test performance

Vu Thi Phuong Anh (1996) provides a thorough description and discussion

of Bachman’s TMF framework In her description, the framework consists of fivedifferent aspects of test method, organized into five categories: (1) facets of thetesting environment, including familiarity of the place and equipment, personnel,time of testing, and physical conditions; (2) facets of test rubric, including testorganization, time allocation, and instructions; (3) facets of the input, includingformat, nature of language, and restrictions on response; and (5) the relationship(s)between input and response (see Figure 3)

The first two of these five categories testing environment and test rubric are categorised in the discussion to “belong to the administrative side of the testingevent”(p.86) Testing environment includes elements such as place and equipment,personnel, time of testing, and physical conditions such as lighting, temperature,humidity, and seating arrangement Test rubric includes test organization, timeallocation, and instructions These two facets of test methods are obviously notrelated to the language ability that a language test is meant to tap However, theycertainly have an important effect on language test performance, as they can changethe way a test taker responds to a test (i.e., response validity), as well as his/herdegree of success on the test (i.e., test scores) and therefore could invalidate the

-interpretation of test scores as measures of communicative language ability Forexample, if the time allocation in a test is too limited, it may cause some test takers

to rely on blind guessing; other to give up attempting to accomplish the test task;and a third group to rely on background knowledge to solve the test task In thiscase, differences in test scores no longer reflect differences in the abilities the test ismeant to measure, and cannot be interpreted as such Thus, these two facets of thetest method need to be considered carefully in test validation

The other categories of TMF in the Bachman model - test input, testexpected response, and the relationship between the two - are said by Vu ThiPhuong Anh to constitute the context side of communicative language performance,and can be used to monitor/assess the situational aspect of authenticity Input andexpected response refer to characteristics of the language used in the test texts andtasks such as channel, mode, type, length of language, topic, etc (e.g., a multiple-choice reading test would belong to a selected type of response, visual channel ofpresentation, receptive mode of presentation; the topic of the text might be technical

or general) The relationship between input and response can be one of three kinds:reciprocal (i.e., with two-way interaction between test input and response, as in anoral interview), nonreciprocal (without any interaction, as in a reading or writingtest), and adaptive (response affects subsequent input, but no feedback is provided,

as in computer-adaptive testing)

Bachman’s framework of test method facets (TMF) complements theframework of communicative language ability (CLA) discussed in the previous part(2.4) in providing a theoretical basis for test development and evaluation Theinclusion of test method effects in a theory of language test performance isconsidered by several authors as the innovation in the Bachman model As Skehanstates:

This aspect of the model implies a recognition of the fallibility of testing

of the way in which part of the test result may be the result of the test format effects rather than underlying ability, and most ambitiously, that testers need

to know about systematic effects of these sorts if they are to allow for them in actual test results, or, better still, to avoid them. (Skehan 1991:10)

1 FACETS OF TESTING ENVIRONMENT

Familiarity of the place and equipment

Language (native, target)

Channel (aural, visual)

Specification of procedures and tasks

Explicitness of criteria for correctness

3 FACETS OF THE INPUT

Format

Channel of presentation (aural, visual)

Mode of presentation (receptive)

Form of presentation (language, nonlanguage, both)

Vehicle of presentation ( live , canned , both)

Language of presentation (native, target, both)

Identification of problem (specific, general)

Degree of speededness

Nature of language

Length

Propositional content

Vocabulary (frequency, specialization)

Degree of contextualization (embedded / reduced)

Distribution of new information (compact / diffuse)

Type of information (concrete/abstract,

Nature of language

Length Propositional content

Vocabulary (frequency, specialization) Degree of contextualization (embedded/reduced) Distribution of new information (compact/diffuse) Type of information (concrete/abstract,

positive/negative, factual/counterfactual) Topic

Genre

Organizational characteristics

Grammar Cohesion Rhetorical organization

Pragmatic characteristics

Illocutionary force Sociolinguistic characteristics

Restrictions on response

Channel Format Organizational characteristics Propositional and illocutionary characteristics Time or length of response

5 RELATIONSHIP BETWEEN INPUT AND RESPONSE

Reciprocal Nonreciprocal Adaptive

Figure 3: Categories of test method facets (Bachman 1990:119)

A further discussion of the subcategories of TMF will be presented inChapter 4, where the content of the tests under evaluation is analyzed usingBachman model

However, it is worth noting at this point that the categories in the twoframeworks of Bachman’s also have close relationship The test method of the test

can control the ability tapped For example, the third and the fourth categories of theTMF framework closely relate to the first component of the CLA framework as theformat, the channels, the language characteristics of the input and the expectedresponse(categories 3 and 4 of the TMF) can control the assessment of the differentsub-competence of the test taker’s language competence (component 1 of the CLA).

In a more general view, it is noticeable that the analysis of the components in theCLT framework helps investigate the ‘what’ of the tests; meanwhile, the analysis ofthe five categories in the TMF framework provides a look into the ‘how’ of thetests The ‘how’ can control the ‘what’ Therefore, it should be wiser to start with ananalysis of the ‘how’ and end with conclusions about the ‘what’ of the test.Obviously, a discussion of these two aspects of the school’s tests can help figure out

a thorough picture of the school’s testing practices

2.6 Summary

Communicative language testing cannot be applied appropriately without athorough understanding of its importance and its underlying theory (i.e., abilitiesbeing measured) and principles (i.e., test qualities) Chapter 2 has reviewed all therelevant literatures necessary for theoretical basis of the research First of all, the

term ‘washback has been explained to see the importance of testing in teaching and learning Later, the principles of the Communicative Language Testing have been defined under terms of reliability, validity, practicality and authenticity These

principles have been then discussed more practically through the description of thetest qualities Finally, the chapter has ended with the most advanced frameworks ofthe components in a test’s content and method that can affect test performance Thereview of these relevant literatures will be of great use for the understanding of thestudy design and the analysis of the data collected during the research

CHAPTER 3 METHODOLOGY

This chapter discusses the methodology employed in the study reported inthis thesis The chapter begins with a presentation of the research questions,followed by a discussion of the research design and a description of the subjects, theinstruments, and finally with information about the implementation including datacollection procedures and preliminary results

3.1 Research questions

As explained earlier, this thesis is an attempt to investigate the reality ofEnglish language testing at Thai Nguyen LSS, with a desire to gain betterunderstanding of the extent to which testing practices at the school have beenchanged to support the new teaching trend _ the CLT The theoretical position ofthe CLT provides a basis for formulating research questions for the study and forselecting appropriate methodologies to arrive at an answer to these questions Theresearch questions posed in this thesis include:

1 What are the objectives, the TLU domain and TLU tasks of Grade-6 Englishcourse at Lower Secondary Schools?

2 What are Thai Nguyen LSS teachers’ perceptions of the CommunicativeLanguage Teaching (CLT) in general and of the Communicative LanguageTesting in particular?

3 Do real communicative language teaching and testing take place in classes atThai Nguyen LSS?

4 How do the teachers deal with informal progressive assessment?

5 Do current achievement tests at the school match the course objectives, i.e.with focus on communication skills?

6 Are the products of current teaching and testing practices at the school _ thestudents _ qualified in communication skills?

These questions reflect the view of language testing as an inseparable part oflanguage teaching, which cannot be divorced from the course syllabus and the

course objectives The answers to the questions, therefore, can serve as a basis forclear understanding of the school’s teaching renovation and a forecast of its future.

Based on the theoretical background and the context of the study, theresearch questions can be answered through the collection and analyses of fourdifferent sources of data: (1) guiding documents from the MOET and DOET on thecourse orientation and objectives (the TLU domain and TLU tasks), especially thoseconcerning testing; (2) the school teachers (their qualifications, their perceptions ofthe new teaching trend, their teaching and testing practices); (3) the progressive andsummative achievement tests in practice at the time of the research (test content,test methods, and test scoring); and (4) the school’s contemporary Grade-6 students(their communicative language ability) Section 3.2.1, 3.2.2, 3.2.3 and 3.2.4 belowexplain how each of the four sources of data contributes to the study

3.2.1 For course orientation and course objectives

Guiding documents by course book’s writers, the MOET, and Khanh HoaDOET could serve as an important source of data to define the objectives, the TLUdomain and the TLU tasks of the current Grade-6 English course (Researchquestion 1) The information served as the rationale for any comparison andconclusions arising from the analysis of other data because it would be impossible

to evaluate the teachers’ teaching and testing practices without understanding therequirements posed for them

To do this, it was necessary to examine the course books, the teacher’sbooks, the course descriptions and the tests specifications issued by the MOET andthe DOET

3.2.2 For the school teachers’ perceptions and their teaching-testing practices

Information about the school teachers needed seeking for to answer the nextthree questions of the research The answers to question 2 and 3 helped confirm theextent to which the school teachers had reached in their teaching reform Theanswer to question 4, on the other hand, provided the researcher with informationabout an important part of the teachers’ testing practice that could not be observed

in test papers, i.e how they conducted oral tests and how they carried out informalassessment

First, teachers’ perceptions (research question 2) had to be explored toprovide grounds for any explanation on their teaching performance, for it is evidentthat perceptions can navigate actions Moreover, a look into the teachers’perceptions could also help evaluate the effectiveness of their prior pre-serviceand/or in-service training courses This piece of information will be useful for laterdiscussion and recommendations based on the results of the study To understandthe teachers’ perceptions, a questionnaire was tailored All the information obtainedwas then transformed into numerical values to be analyzed with statistical methods.Detailed description of the questionnaire will be provided in part 3.3 of this chapter

Second, teachers’ teaching practice had to be investigated to see how theymanipulate on their perceptions of the CLT (research question 3) This could bedone through the observation of their classes with focus paid on the level ofcommunicativeness in the activities, such as how new language items werepresented and practised; how skills were developed; and whether the students hadchances to communicate on the language acquired These pieces of information canserve for two purposes: (1) to confirm whether the teachers were following theCLT Only if they were, could their testing practice be analyzed in the light of theCLT If not, it could be concluded at this stage that there was no CLT at the school,and thus no more study would be needed; and (2) to allow a comparison betweenthe contemporary teaching practice with previous one (as could be found in theresult of another study mentioned in part 2.2 of this thesis) This comparison couldprovide some evidence of the backwash effects that the contemporary testingpractice had caused to the teaching practice

Another aspect that the researcher could examine from classroomobservation was how the teachers conducted informal assessment (research question4) This aspect could be observed from the system of questions they used to checkthe student’s understanding, their reactions on the student’s feedbacks, the way theycorrected the student’s mistakes; and the way they consolidated the lessons Theteachers’ focus in these things could reveal part of their testing practice.

Finally, the most significant purpose of classroom observation was to seehow students’ speaking skills were tested (research question 4) As could be seenearlier in the background to the research, the testing of speaking skills could not beadministered formally in the summative tests due to the large number of students ineach class These skills were, therefore, were supposed to be assessed during classtime Oral tests, either formal or informal, were then a crucial focus of theresearcher’s class observation

In brief, the school teachers were an important source of data for informationabout the teaching and part of the testing practice at the school With anunderstanding of the teachers’ perceptions and their performance in class, theresearcher found complementary evidence of the school’s teaching and testingobjectives, which could not be found through a mere analysis of the test papers Inaddition, this understanding contributed much to later explanation of the students’communicative language ability

3.2.3 For the evaluation of test qualities

The contents and methods of the school’s test papers in use at the time wasanother important source of data contributing to a thorough understanding of theteaching-testing reality at the school Following the observation of informalassessment in class, test analysis could provide evidence for concluding whether thetests could measure what were expected to be measured, i.e whether the constructsbeing measured in the tests match the course objectives (research question 5)

For the purpose of the study, Bachman’s frameworks of CommunicativeLanguage Ability (CLA) (Bachman 1990:84) and of Test Method Facets (TMF)(Bachman 1990:119) were used The CLA framework provides a means for

“characterizing the traits, or constructs, that constitute the ‘what’ of language