Investigating students attitides towards teaching to the test in their toeic preparaton courses a study at hcmc university of technology m a

VIETNAM NATIONAL UNIVERSITY-HOCHIMINH CITY UNIVERSITY OF SOCIAL SCIENCES & HUMANITIES FACULTY OF ENGLISH LINGUISTICS & LITERATURE INVESTIGATING STUDENTS’ ATTITUDES TOWARDS “TEACHING TO

VIETNAM NATIONAL UNIVERSITY-HOCHIMINH CITY

UNIVERSITY OF SOCIAL SCIENCES & HUMANITIES

FACULTY OF ENGLISH LINGUISTICS & LITERATURE

INVESTIGATING STUDENTS’ ATTITUDES TOWARDS “TEACHING TO THE TEST” IN THEIR TOEIC PREPARATION COURSES: A STUDY AT HCMC UNIVERSITY OF TECHNOLOGY

A thesis submitted to the Faculty of English Linguistics & Literature

in partial fulfillment of the Master’s degree in TESOL

VIETNAM NATIONAL UNIVERSITY-HOCHIMINH CITY

UNIVERSITY OF SOCIAL SCIENCES & HUMANITIES

FACULTY OF ENGLISH LINGUISTICS & LITERATURE

INVESTIGATING STUDENTS’ ATTITUDES TOWARDS “TEACHING TO THE TEST” IN THEIR TOEIC PREPARATION COURSES: A STUDY AT HCMC UNIVERSITY OF TECHNOLOGY

A thesis submitted to the Faculty of English Linguistics & Literature

in partial fulfillment of the Master’s degree in TESOL

By

NGUYỄN THÁI SƠN

Supervised by

LÊ HOÀNG DŨNG, PhD

Dr Phó Phương Dung

Secretary HCMC University of Social Sciences &

Humanities

Assoc.Prof Dr Nguyễn Thanh Tùng

Reader 1 HCMC University of Pedagogy

Dr Đoàn Huệ Dung

Reader 2 Nông Lâm University

Dr Lê Hoàng Dũng

Member HCMC University of Social Sciences & Humanities

The final copies have been examined by the supervisor of the thesis and the signature that appears below verifies the fact that any necessary changes have been incorporated and that the thesis is now given final approval by the committee with reference to content and form

The thesis is therefore accepted in partial fulfillment of the requirements for the degree of Master of Arts in TESOL

Date Supervisor’s signature

ACKNOWLEDGEMENTS

Many people have assisted me in the preparation of the current thesis, but, of course I am alone responsible for any shortcomings First and foremost, I would like to express my sincere thanks and special gratitude to Dr Lê Hoàng Dũng, my supervisor, without whose excellent guidance, patience, and encouragement, the completion of this thesis could never have been possible

I am grateful for the support provided by University of Technology, Ho Chi Minh City: both its students and teachers who responded and have been instrumental in shaping the contents of the study I would like to give special thanks to Ms Mai Phuong, Ms Kim Phung, and Ms Xuan Tien, who enthusiastically permitted data collection through observation and shared insights into test preparation in their classrooms

A serious problem that could have obstructed the feasibility of this thesis was inadequate access to materials; consequently, my ultimate gratitude goes to Mr Huy Cuong, PhD candidate, Massey University and Mr W James Popham, Professor Emeritus, University of California-Los Angeles for kindly sending me their valuable articles upon my request

I would like, in addition, to thank the faculty members at USSH-EF who have contributed to my professional development and thank the staff who provided me with the full assistance whenever I needed it

Also, a special word of thanks is due to Ms Hoang Yen and Mr Nguyen Han, who generously dedicated their time and patience to supporting me with their excellent technical knowledge

Last but not least, I would like to express my heartfelt gratitude to my family for their deep love, tolerance, and understanding that have and will always follow me

on every path of my life

STATEMENT OF ORIGINALITY

I certify my authorship of the Master’s Thesis in TESOL submitted today entitled: INVESTIGATING STUDENTS’ ATTITUDES TOWARDS “TEACHING TO THE TEST” IN THEIR TOEIC PREPARATION COURSES: A STUDY AT

HCMC UNIVERSITY OF TECHNOLOGY

In terms of the Statement of Requirements for Theses in Master’s Programs issued

by the Higher Degree Committee This thesis has not been submitted for the award

of any degree or diploma in any other institutions

Ho Chi Minh City, May 2014

Signature

NGUYỄN THÁI SƠN

RETENTION OF USE

I hereby state that I, NGUYỄN THÁI SƠN, being a candidate for the degree of Master of Arts (TESOL), accept the requirements of the University relating to the retention and use of Master’s theses deposited in the library

In terms of these conditions, I agree that the original of my Master’s thesis deposited in the library should be accessible for purposes of study and research, in accordance with the normal conditions established by the library for the care, loan and reproduction of theses

Ho Chi Minh City, May 2014

Signature

NGUYỄN THÁI SƠN

TABLE OF CONTENTS

APPROVAL SHEET i

ACKNOWLEDGEMENTS ii

STATEMENT OF ORIGINALITY iii

RETENTION OF USE iv

LIST OF TABLES ix

LIST OF FIGURES x

ABSTRACT xi

CHAPTER 1 INTRODUCTION 1

1.1 Background to the study 1

1.1.1 “Teaching to the test” and the ELT reform in Vietnam 1

1.1.2 Utilization of TOEIC as a high-stakes test in Vietnam 3

1.1.3 TOEIC preparation courses at HCMC-UT 4

1.2 Objectives of the study 5

1.3 Research questions 5

1.4 Significance of the study 5

1.5 Scope of the study 6

1.6 Definition of terms 6

1.7 Organization of the study 8

CHAPTER 2 REVIEW OF RELATED LITERATURE 9

2.1 TOEIC as a high-stakes English test 9

2.1.1 Introduction to the TOEIC test 9

2.1.2 Issues around the TOEIC test and test preparation 10

2.2 “Teaching to the test” 12

2.2.1 General perceptions of “teaching to the test” 12

2.2.2 Discussion over “teaching to the test” 14

2.2.3 Ethical continuum for evaluation 25

2.3 A synthesis of the review of related literature 28

2.4 Towards an operational definition 29

CHAPTER 3 METHODOLOGY 37

3.1 Research method 37

3.2 Research setting 39

3.3 Sampling 40

3.4 Research participants 41

3.4.1 Teacher respondents 41

3.4.2 Student respondents 42

3.5 Research instruments 43

3.5.1 Questionnaire to teacher respondents 44

3.5.2 Questionnaire to student respondents 44

3.5.1 Structured interview 46

3.5.2 Classroom observation 46

3.6 Data collection procedure 47

3.7 Data analysis procedure 49

CHAPTER 4 DATA ANALYSIS AND DISCUSSION OF RESULTS 51

4.1 Analysis of data 51

4.1.1 Results from questionnaires to teacher respondents 51

4.1.2 Results from questionnaires to student respondents 63

4.1.3 Results from interviews with students 89

4.1.4 Results from classroom observation 94

4.2 Discussion of results 98

4.2.1 Application of “teaching to the test” at HCMC-UT 98

4.2.2 Learners’ attitudes towards “teaching to the test” at HCMC-UT 99

4.2.3 Practical problems in TOEIC preparation courses at HCMC-UT 101

CHAPTER 5 SUMMARY OF FINDINGS, SUGGESTIONS & CONCLUSION 105

5.1 Summary of major findings 105

5.2 Suggestions 107

5.2.1 Suggestions to the academic administrators at HCMC-UT 107

5.2.2 Suggestions to the teachers at HCMC-UT 111

5.3 Limitations of the current study 117

5.5 Conclusion 118 REFERENCES 119 APPENDICES 127

LIST OF ABBREVIATIONS

CA : Communicative Approach

CEFR : Common European Framework of Reference

CET : College English Test

CETQA : Center for Educational Testing and Quality Assessment

CLT : Communicative Language Teaching

CIV : construct-irrelevant variance

EFL : English as a Foreign Language

ESP : English for Specific Purposes

ETS : Educational Testing Service

HCMC : Ho Chi Minh City

HCMC-UT : Ho Chi Minh City University of Technology

IELTS : International English Language Testing System

ITBS : Iowa Test of Basic Skills

TOEIC : Test of English for International Communication

TOEFL : Test of English as a Foreign Language

W : Writing (skill)

LIST OF TABLES

Table 2.1 TOEIC and redesigned TOEIC (adapted from ETS, 2008) 10

Table 2 2 A reviewed concept of “washback” proposed by Pan (2009) 13

Table 2 3 Summary of “teaching to the test” discussion 24

Table 2 4 A continuum of test preparation activities by Haladyna et al (1991) 26 Table 2 5 Indicators of “Teaching to the test” 35

Table 2 6 Indicators of “Teaching to the test” (continued) 36

Table 3.1 Data from the questionnaire to teacher respondents at HCMC-UT 41

Table 3 2 Demographics of the teacher respondents at HCMC-UT 42

Table 3 3 Demographics of student respondents at HCMC-UT 43

Table 3 4 Description of questionnaire to student respondents at HCMC-UT 45

Table 3 5 Reliability statistics of question items in the questionnaire to student respondents at HCMC-UT 49

Table 4 1 Frequency of teaching test content and format 52

Table 4 2 Frequency of test-wiseness training 55

Table 4 3 Frequency of curriculum narrowing 58

Table 4 4 Frequency of motivational activities employed by teachers 61

Table 4 5 Summary of the extent to which “teaching to the test” is employed 63

Table 4 6 Students’ attitudes towards teaching test content and format 71

Table 4 7 Students’ attitudes towards test-wiseness training 75

Table 4 8 Students’ attitudes towards curriculum narrowing 78

Table 4 9 Students’ attitudes towards motivational activities 82

Table 4 10 Learners’ preferred test preparation activities 87

Table 4 11 Summary of learners’ attitudes towards “teaching to the test” 89

Table 4 12 Students’ suggestions for improving TOEIC preparation course 93

Table 4 13 A comparison between teachers’ practices and students’ attitudes towards “teaching to the test” at HCMC-UT 103

LIST OF FIGURES

Figure 2 1 Revised schematic representation of “communicative competence” 11

Figure 2 2 A synthesis of the review of related literature 28

Figure 2 3 Conceptual framework of “Teaching to the test” 30

Figure 3 1 Triangulation of methods and data collection 39

Figure 3 2 Data collection procedure at HCMC-UT 47

Figure 4 1 Students’ preference for learning topics at HCMC-UT 64

Figure 4 2 Students’ learning method preference at HCMC-UT 65

Figure 4 3 Students’ preference for practice activities out of the class 66

Figure 4 4 Students’ preference for checking progress at HCMC-UT 67

Figure 4 5 Students’ priority for language improvement at HCMC-UT 69

Figure 4 6 One-Sample t-Test for students’ attitudes towards teaching test content and format at HCMC-UT 74

Figure 4 7 One-Sample t-Test for students’ attitudes towards test-wiseness training at HCMC-UT 77

Figure 4 8 One-Sample t-Test for students’ attitudes towards curriculum narrowing at HCMC-UT 81

Figure 4 9 One-Sample t-Test for students’ attitudes towards motivational activities at HCMC-UT 84

Figure 4 10 Independent Sample t-Test for students’ attitudes and test pressure 86 Figure 4 11 Learners’ preferred test preparation activities at HCMC-UT 88

Figure 5 1 Suggestions derived from major findings and implications 107

Figure 5 2 Metacognitively-based Approach to teaching the TOEIC test (proposed by Pan, 2010) 111

ABSTRACT

“Teaching to the test” is commonly perceived as a practice of preparing students closely to the test under pressure of high-stakes testing This tendency is becoming more popular in EFL classes, particularly those in Asian context where standardized English tests are currently utilized as highly selective tools in education as well as recruitment A considerable volume of published literature has discussed the defending and opposing viewpoints of experts and practitioners towards this issue; yet little has been mentioned from EFL learners’ perspectives

As the issue became increasingly difficult to ignore, this study was conducted to investigate students’ attitudes towards “teaching to the test” in their TOEIC preparation courses at HCMC University of Technology (HCMC-UT) A triangulation of questionnaires, interviews, and observation was carried out over a sample of 188 students from 5 classes whose instructors claimed to have frequently employed “teaching to the test” in 2013 summer semester

The results reveal that teaching test content and format and test-wiseness training are highly favorable due to its convenience and ethicality whereas curriculum narrowing, which sacrifices communicative teaching for the tested domain, is least

supported This apparently calls for a revision to guarantee a matching between students’ score achievement and their communicative competence Based on its findings and implications, the study then provides suggestions for ELT improvement with regard to preparing students for English standardized test at HCMC-UT

Key words: teaching to the test, language assessment, TOEIC preparation courses

CHAPTER 1 INTRODUCTION

This chapter initiates the current study by presenting some basic introductory information In particular, it discusses background of the problem, objectives of the study, research questions, significance of the study, definition of terms, and organization of the thesis report

1.1 Background to the study

1.1.1 “Teaching to the test” and the ELT reform in Vietnam

The last decade has seen an urgent need to reform English language teaching (ELT) in Vietnam English, as an indispensable tool for success in the inevitable era of globalization, holds a special emphasis in schools’ curriculum Most undergraduates, prior to their college graduation, are expected to have received an average of ten years of English instruction The current national curriculum of foreign language teaching for general and tertiary education, in fact, allocates a minimum of 1,000 hours for ELT (Le HungTien, 2011) More specifically, English

is taught as a compulsory subject at both lower secondary and secondary schools from grade 6 to grade 12, taking up to 700 hours Then, at tertiary level, the regular program for foreign languages, as determined by the Ministry of Education and Training (MOET), is 240 hours for general English and from 60 to 90 hours for ESP

Nonetheless, Hoang Van Van et al (2006), in their extensive study, report that a very modest number of students can use English for communication after finishing the high school program In fact, empirical evidence from a research conducted on

59 universities by MOET reveals a shocking figure: over a half of Vietnamese students (51,7%) cannot get the required basic English proficiency although most

of them have passed all English tests with significantly high scores (Tuoi Tre News Online, Dec 6, 2008)

Then, what has to be blamed for such an undesired outcome? The answers can be varied However, a remarkable comment can be found in the observation of Hoang Van Van et al (2006) which points out a considerable mismatch between teaching and assessment objectives in Vietnam high schools Although testing aims to generalize students’ ability of productive language use, teaching seems to focus on merely discrete-point test items of grammar, vocabulary, reading and sentence transformation that are generally presented in the tests

According to Dr Vu Thi Phuong Anh, the ex-director of CETQA (Center for Educational Testing and Quality Assessment-Vietnam National University, Ho Chi Minh City), testing is the weakest component of language teaching that leads to that frustrating consequence (Tuoi Tre News Online, Dec 6, 2008) Although reformed English textbooks have been introduced with equal focus on all language skills and aspects, major “high-stakes” exams traditionally rely on reading and grammar with multiple-choice format which encourages only rote memorization, guesswork and cramming This means a tendency of “teaching to the test” has so far been conducted to meet the demand of tests, and therefore, distorted the curriculum which primarily aims to develop communicative competence among EFL learners in Vietnam Here arises the so-called “test score pollution” consequence in which “many examinees become expert in taking language tests, but do not learn how to use the language” (Gilfert, 1996) Le Hung Tien (2011:2),

on outlining the ELT in Vietnam general and tertiary education from second language education perspectives, also confirms this problem as he reports:

Many school leavers cannot read simple texts in English nor communicate

with English speaking people in some most common cases… Actually the

real focus of teaching and learning English in both lower secondary and

secondary schools is completing English grammar and vocabulary

exercises, in order to pass the final exams (even the entrance exams of

many universities and colleges in Vietnam) which mainly consist of

grammar and vocabulary tasks only

In agreement with these authors, Duong Thu Mai et al (2012) even go further since they indicate that the quality of ELT in contemporary Vietnam is “a source

of criticism”, directly blaming this issue on the poor conduct of assessment practices by administrators and educators Knowing that testing has certain washback effects on teaching and learning, more attention should be paid to reforms of language assessment as well

In fact, the 2020 Project, which was launched in 2008 in response to the ELT reform in Vietnam, has put a great emphasis on language assessment by issuing the Common Framework of Levels of Foreign Language Proficiency, based on the Common European Framework of Reference for Languages: Learning, Teaching, and Assessment (CEFR) As it is stated, the Project targets a remarkable advance

in the improvement of the foreign language proficiency for the Vietnamese human resource during the period from 2008 to 2020 so that the graduates from vocational colleges and universities would be proficient users of foreign languages, which would be an advantage of the Vietnamese workforce (Decision 1400-QD-TTg)

1.1.2 Utilization of TOEIC as a high-stakes test in Vietnam

Along with other standardized tests, TOEIC (Listening & Reading) has held an especially significant role in language assessment in our education system According to China Daily (Jan 28, 2011), the test has been used by 10,000 organizations in 120 countries around the world with a variety of purposes such as

“screening people for job placement and promotion, tracking the learning progress

of students, employees, and even executives” Vietnam has adopted TOEIC since

2001 and two years later, the test was being used at about 45 companies (ETS, 2005) Then more and more universities and colleges throughout the country have used TOEIC as a standardized test to measure students’ English proficiency for both entrance and graduation requirements (Tuoi Tre News Online, Dec 6, 2008) The more recent data indicates that TOEIC scores are employed for progress tracking as well as selective purposes at 127 colleges and more than 350 nationwide business organizations (ICTPress, Dec 6, 2013) Also, test preparation

courses are widely offered in many educational institutions to both ESP and EFL learners And again, such extensive preparation for a high-stakes test is facing criticism for its primary focus on increasing test scores and neglect of developing learners’ communicative competence which the test claims to measure (Phillips, 2006)

1.1.3 TOEIC preparation courses at HCMC University of Technology

(HCMC-UT)

What has been discussed above is quite applicable to the situation at Ho Chi Minh City University of Technology (HCMC-UT), where students are taught general English as EFL rather than ESP As its mission states, the university is a leading center for training, scientific research and technology transferring in Vietnam Consequently, English language teaching is a major emphasis in its curriculum which offers main-stream students 240 periods of English equally incorporated into 4 courses during the whole training programme The first three courses (AV 1,

2, 3) are assigned to teach general English, helping students build up language areas and skills for communication Then the last module (AV 4) is designed in form of TOEIC preparation course to prepare students for a minimum score of

450 However, the final tests that measure students’ achievement at the end of all 4 courses are still merely multiple-choice questions which regardless of their practicality, just focus on discrete points and cannot promote communicative language use (see appendix 5) TOEIC also holds extremely high-stakes use since its certificate with a required score is a must for graduation and course completion Students, in fact, have to achieve at least 400 scores for thesis acceptance and 450 scores for thesis defense (Decision 958/QĐ-ĐHBK-ĐT) Consequently, TOEIC preparation course (AV4) at HCMC-UT is commonly considered susceptible to

“teaching to the test”

Taken altogether, the situations above indicate that ELT reform in Vietnam is facing the issue of “teaching to the test” at both middle-school and higher education, requiring improvement in language assessment Notwithstanding

criticism, those who support this tendency of teaching for tests claim that it is a convenient and practical approach in response to high-stakes and helps their students to overcome pressure of testing Moreover, as long as English language tests hold such powerful effects, “teaching to the test” should be considered fruitful and advantageous The debate around this issue has yet to seem settled (Phillips, 2006; Pan, 2010)

So far, however, there has been little discussion over how the students perceive the practice of “teaching to the test” employed in their classrooms In regard with learner-centered approach, students’ needs and beliefs which consciously or unconsciously direct their own learning should be seriously considered as a basis for reformation and improvement in ELT

1.2 Objectives of the study

As the issue around “teaching to the test” is becoming increasingly difficult to ignore, the objectives of this study are to (1) find out the extent to which “teaching

to the test” is employed in TOEIC preparation courses at HCMC-UT, and (2) investigate students’ attitudes towards the practice of “teaching to the test” in their classrooms This study also expects to contribute to an awareness of learners who

as the center of education cannot be ignored in the ELT reform

1.3 Research questions

To obtain those objectives, this study seeks to address the following questions:

1 To what extent is “teaching to the test” employed in TOEIC preparation courses at HCMC-UT?

2 What are students’ attitudes towards the practice of “teaching to the test” in their classrooms?

1.4 Significance of the study

There are several areas where this study possibly makes a unique and original contribution to the current ELT reform First, its findings extend our knowledge of

washback effects and provide better understanding of EFL learners’ attitudes towards “teaching to the test” under high-stakes condition Empirical evidence from this study, then, gives a practical platform to suggest what can be done for ELT improvement in regard to standardized English tests like TOEIC Finally, this research will serve as a base for future studies on other high-stakes tests in contemporary use such as TOEIC Speaking and Writing, IELTS, TOEFL, etc and add to the growing body of literature on language assessment which is strongly focused in the current ELT reform in our country

1.5 Scope of the study

The study, with an endeavor to promote leaner-centeredness in language classrooms, has its primary focus on students’ perceptions of “teaching to the test” practice on getting prepared for a high-stakes test like TOEIC, which then form the basis for ELT improvement and provides suggestions to academic administrators

as well as teachers at HCMC-UT The study, however, has no interest in confirming or rejecting the appropriateness of TOEIC utilization and test preparation courses in a more general context

1.6 Definition of terms

Several technical terms related to language assessment will be utilized throughout the current study Therefore, it is necessary to indicate what some essential terms refer to and provide a consensus for later discussion The following term definitions are mostly synthesized from Bailey (1998); Brown (2001); Crocker (2005); and Brown (2005)

Discrete-point test: test constructed to measure students’ knowledge of different

structures by assessing independent bits of language (Brown, 2005)

Distracters: those choices that will be counted as incorrect in a multiple-choice

item; they should distract or divert the students’ attention away from the correct answer (Brown, 2005)

High-stakes assessment: assessment in which significant awards are given or

sanctions imposed on individual students, teachers, or schools as result of performance (Crocker, 2005)

Measurement error: a source of variance, due to extraneous sources, e.g

personal problems, scoring procedures, test item problems, etc (Brown, 2005)

Normal distribution: a frequent distribution of test scores that takes the shape of a

bell-curved, concentrated near the center and decreasing symmetrically on both sides (Brown, 2005)

Norm-referenced test (NRT): a type of test where a student’s test scores are

interpreted relative to all the other students’ scores, usually with reference to normal distribution (Brown, 2001)

Test practicality: the degree to which a test is good regarding financial

limitations, time constraints, ease of administration, and scoring interpretation

(Brown, 2001)

Test reliability: the extent to which the test results can be considered consistent or

stable (Brown, 2001)

Test-wiseness: the ability to easily comprehend almost any test directions,

knowledge of guessing strategies, or strategies for maximizing the speed of task performance (Bailey, 1998)

Test validity: the degree to which a test measures what it claims, or supports, to be

measuring There are three types of validation: (1) content validity, (2) face validity, (3) construct validity (Brown, 2001)

Content validity: test requires the test-takers to perform the behavior that is

being measured

Face validity: test is valid in the students’ perception

Construct validity: test actually taps into the theoretical construct as it has been

defined

Washback: the effect test has on teaching and learning (Brown, 2005)

1.7 Organization of the study

The overall structure of the study takes the form of five chapters, including this

introductory chapter The remainder of the thesis report includes chapter 2 which

reviews the related literature and proposes an operational definition of “teaching to

the test” to guide the study, chapter 3 describing methodology employed in the current investigation with reference to the addressed research problems, chapter 4

presenting how the data collected is analyzed and discussed to seek the best

answer to the research questions, and chapter 5 providing a summary of major

findings and implications which in turn, offer a platform for suggestions of ELT improvement at HCMC-UT and recommendations for future research

Summary

This chapter has presented an overview about the current study with reference to background of the problem, research objectives, research questions, significance of the study, definition of terms, and organization of the study In the next chapter, some related literature that provides a theoretical base to the study will thoroughly

be reviewed

CHAPTER 2 REVIEW OF RELATED LITERATURE

In this chapter, related literature that provides a theoretical base to the current study is going to be reviewed In particular, it thoroughly discusses a considerable volume of educational assessment literature with reference to TOEIC as a high-stakes standardized English test and test preparation Then, an operational definition of “teaching to the test” is synthesized to guide the research

2.1 TOEIC as a high-stakes English test

2.1.1 Introduction to the TOEIC test

The TOEIC (Test of English for International Communication) is a choice instrument designed to assess “the everyday English skills of people working in an international environment… and scores indicate how well people can communicate in English with others in business, commerce, and industry” (ETS, 2008) The test was firstly requested by the Japanese government for making decisions in recruitment, placement, and training with Japanese workers in

multiple-1979 (Stoynoff, 2009) Since then, TOEIC has seen extremely rapid growth and been internationally accepted by 10,000 organizations in 120 countries around the world for a variety of selective purposes (Daily China, Jan 28, 2011)

Originally, the 1979 test was comprised of 2 sections (Listening and Reading), each of which contained 100 MCQs As its maker claimed, the test content was based on what the test developers expected examinees would encounter in workplace settings where English was used for general commercial purposes Then

a redesigned TOEIC Listening and Reading test was introduced in 2005 but it continues to be a paper-based assessment conducted in 2 hours Instead, listening input is made more international by including three varieties of English (Australian, British, and North America) and texts in Reading section is more authentic with double-passages corresponding to the target language use situation This revised TOEIC is the ongoing test at many countries including Vietnam

were launched in 2006 These are computer-based assessments that are administered separately and at different times from the TOEIC Listening and Reading test In brief, first developed in Japan in 1979, TOEIC has gone through several adaptations and been in widespread use as a high-skates test in English language assessment for highly selective purposes An illustration of TOEIC Listening and Reading formats (old and new) can be seen as follow:

Listening comprehension Section 1

2 Question-Response: 30 questions Question-Response: 30 questions

3

Short Conversations: 30 questions

(30 Conversations, 1 question each)

Short Conversations: 30 questions

(10 Conversations, 3 question each)

4

Short talks: 20 questions

(6-9 talks, 2-4 questions each)

Short talks: 20 questions

(10 talks, 3 questions each)

Reading comprehension Section 2

5 Incomplete Sentences: 40 questions Incomplete Sentences: 40 questions

6 Error Recognition: 20 questions

Text completion: 12 questions

(4 reading sets, 3 questions each)

7 Reading Comprehension: 40 questions

Reading Comprehension: 48 questions

(single passages: 28 questions, double passages: 20 questions)

Table 2.1 TOEIC and redesigned TOEIC (adapted from ETS, 2008)

2.1.2 Issues around the TOEIC test and test preparation

One of the most significant current discussions in language assessment is about the validity of TOEIC test Although TOEIC just consists of 2 sections for testing 2 receptive skills, its creators claim that “the scores indicate how well people can communicate in English” (ETS, 2008) This leads to some independent researches into whether the TOEIC test does indeed measure communicative competence (Cunningham, 2002; Sewell, 2005; Lawson, 2008)

While Sewell (2005) and Lawson (2008) state that TOEIC is a relatively reliable and valid test of listening and reading comprehension, they both doubt its validity

as a measure of speaking and writing abilities In fact, the construct validity of TOEIC as a communicative language test is challenged by Cunningham (2002) who questions how TOEIC, a mere multiple-choice test, can assess learners’ communicative competence which is quite complex and consists of several interrelated components (grammatical, socio-linguistic, discourse, and strategic competence)

Through a long chronological evolution, the newest model of Communicative Competence along with its role in language testing has been proposed by Celce-Murcia (2007:45) (see Figure 2.1) The development of Communicative Competence theoretical framework means better construct of language test is required Multiple-choice questions, as discrete-point testing, are normally criticized for its lack of authenticity and mismatch with the nature of productive language use However, due to its convenience and practicality, multiple-choice assessment has often been overused (Brown and Hudson, 1998)

Figure 2 1 Revised schematic representation of “communicative competence”

(adopted from Celce-Murcia, 2007)

Cunningham (2002) then conducted an empirical research to examine the correlation between students’ TOEIC scores and their communicative ability The study was conducted on a homogenous group of first-year university students in Japan Students’ communicative competence then was measured by TIC, a direct assessment of reading, listening, and writing abilities The results suggested no correlation between 2 variables and TOEIC turns out not to be a reliable test of language abilities The researcher, therefore, doubts its role as a placement test and

a measure of non-native speaker’s language ability

Regardless of controversial views on TOEIC, the test has widely been used in Asia and spawned a mass of TOEIC preparation courses However, preparing students for international standardized tests such as TOEFL, TOEIC, and IELTS is often considered “unethical because of concerns over the issues of narrowing the curriculum, overemphasizing test-strategy instruction and mechanical practice, neglecting high-order thinking skills, and causing test-wiseness” (Pan, 2010) In other words, many preparation courses of this kind, while aiming at students’ performance in TOEIC and test scores, have become test-oriented rather than language-focused Therefore, there have been a lot of debates about this tendency, which is normally addressed as “teaching to the test”

2.2 “Teaching to the test”

2.2.1 General perceptions of “teaching to the test”

Despite widely-discussed, the phrase “teaching to the test” has yet to be thoroughly defined, causing much confusion with other similar terms and over whether it is good or bad This part reviews a number of definitions of “teaching to the test” that have been both explicitly and implicitly mentioned in published research and literature on educational assessment

To begin with, it would be useful to examine the concept of “washback” which is generally perceived as a backward effect of test on teaching and learning (Buck, 1988; Hughes, 1989; Shohamy, 1992; Alderson and Wall, 1993; Messick, 1996;

Brown, 2005) Bailey (1996), after carefully reviewing the concept, proposes a basic model of how washback works This model helps to clarify “washback” by classifying the effects of test-derived information on test-takers as “washback to the learners” and those to teachers, administrators, curriculum developers, counselors, etc… as “washback to the programme” However, the most systematic study of this concept must be attributed to Pan (2009) who thoroughly examines washback at micro and macro levels as follows:

Micro-washback Narrow view

within classroom

The extent to which a test influences within the classroom, mainly in the change or innovation of curricula and teachers’ methodologies and the influence of students’ learning

Macro-washback Wider and more

holistic view beyond classroom

The extent to which a test influences within the society, ranging from government policymaking, school administration, publishing, and general opportunities, to parents’ expectations of their children

Table 2 2 A reviewed concept of “washback” proposed by Pan (2009)

“Teaching to the test”, as classroom activities driven by the test, can be perceived

as part of washback to the programme (Bailey, 1996) or micro-washback (Pan, 2009) Yet, it is worth noting that “teaching to the test” has long been discussed as

a consequence of negative washback in which “tests determine what teachers actually teach and what students will study for” (Wiggins, 1989) In fact, Brown (2005) on listing harmful aspects of washback implies test preparation closely geared towards the test that leads to considerable narrowing of the curriculum Similarly, Crocker (2005:165) highlights “teaching to the test” as a negative connotation as it generally refers to “targeting and delivering instruction geared solely at the content or format of a particular test for the express purpose of increasing examinee score”

Regarding language testing at secondary schools in Vietnam, Tran Thi Thu Huong (2010) emphasizes the relationship between testing and teaching in classrooms She adds that test would bring positive effects once it can honestly reflect the reality of teaching and learning, help participants realize their strengths and weaknesses, and therefore provide information to enhance educational quality However, a test can also bring harmful effects when introduced as authoritative or judgmental tools because test preparation can outweigh language-skill practice Taken altogether, the current literature of educational assessment tends to perceive

“teaching to the test” as negative washback resulting from a high-stakes test since test preparation is geared so closely to the test that the pre-determined curriculum will be considerably narrowed It is significant to note that the term “teaching to the test” can also be addressed as “item-teaching” (Popham, 2001), “curricular reductionism” or “measurement-driven instruction” (Crocker, 2005) and is absolutely different from “curriculum-teaching” (Popham, 2001) or “curriculum alignment” (Crocker, 2005) which means teaching the full set of knowledge or skills represented by the test

2.2.2 Discussion over “teaching to the test”

2.2.2.1 Arguments against “teaching to the test”

There is a large volume of published studies describing “teaching to the test” as an educational problem In fact, Crocker (2005) claims that this term generally has a negative connotation among teachers, students, and school administrators The discussion against “teaching to the test” must have been initiated by Cannell (1987) whose report questions the credibility and integrity of USA nationally normed standardized achievement test results His analysis points out that all 50 states claimed to be above the national average and reported inflated ITBS (Iowa Test of Basic Skills) scores where an estimated 70% of the students performed above average Regarding this as a statistical illogicality, he argues that inaccurate initial norms and “teaching to the test” must have accounted for such spurious scores

Cannell’s survey then set the stage for several studies investigating the relationship between test score validity and “teaching to the test” (e.g Mehrens and Kaminski, 1989; Shepard, 1990; Haladyna, 1991) These studies, in general, share a chief concern that some “teaching to the test” practices while aiming to maximize students’ performance for standardized tests can distort scores and prevent accurate inferences from test scores to the broader domain the test are meant to represent

Mehrens and Kaminski (1989) observe that no single standardized test can represent a complete map of the content domain that schools require their students

to achieve Test makers, in fact, also claim that their tests are merely a sample of the broader achievement domain of skills and knowledge which is typically too great to test everything Therefore, “teaching to the test” makes scores untruthful The scores then simply indicate how well students have studied for a particular test rather than a more general level of achievement

In an attempt to explain for the source of “inflated test score gains” in referenced test, Shepard (1990) argues that students who are taught closely to the test receive more focused instruction on the tested content and therefore, outperformed the students in norming sample group who did the test unannounced, unanticipated and of course, without having benefit of such excessive test preparation Students’ scores in comparison to the normative group then apparently become spurious and inflationary

norm-This view is supported by Haladyna et al (1991) who address such phenomenon as

“test score pollution” and blame “teaching to the test” for producing irrelevant variance (CIV) through inappropriate test preparation “Test score pollution”, generally defined as “factors affecting the truthfulness of a test score interpretation”, is generated by three main sources: (1) test preparation activities, (2) testing situations and (3) external conditions On examining the causes of this problem, the authors identify how public school personnel prepare students to take standardized test as one of the two major sources of score polluting practices

“Teaching to the test”, consequently, is considered dishonest and unethical in that

it makes students’ scores rise without actually improving their achievement in the broader domain from which the educators would like to infer

“Teaching to the test”, known as “washback to the programme”, also has a major influence on how the curriculum is run The earliest empirical study to assess the effects of high-stakes testing on instruction must be attributed to Shepard and Dougherty (1991) Data from questionnaires and direct interviews reveal that due

to the importance of standardized testing, half of the teachers admitted giving instruction focused on tested skills and therefore, put less emphasis on non-tested content Many teachers also said that they gave less attention to higher-order thinking and extended projects because of testing Instead, they increased activities which supported the test and distorted teaching because of extensive time given to test preparation In fact, the survey reports that 68% of the teachers claimed to conduct test preparation activities regularly and throughout the school year, not only a few days or weeks before testing Despite defending the use of standardized testing, Crocker (2005:159) highlights two types of concern about test preparation

in regard to curriculum narrowing:

(1) Test preparation requires drilling students on a narrow set of low-level

skills covered on the test, ignoring material that would have been covered

had the teacher been unfettered by demands of preparing students for the

assessment

(2) Instruction aimed at making students skillful test takers on

standardized assessment may be harmful to educational development and

thinking process.

Crocker (2003) points out that this problem is an “inherent conflict” in educational assessment Whereas the examiner aims to measure students’ ability to solve real-life problem using their acquired skills and knowledge, the examinees expect a certain obvious match between teaching and testing Therefore, although “teaching

to the test” can be inappropriate, it is sometimes a compromise between the participants Jerald (2006) even goes further since he addresses “teaching to the

test” as “drill and kill” instruction which restricts teaching to focus on low-level skills of rote learning and memorization The concern over test validity is not as much as how “teaching to the test” affects the curriculum and classroom instruction since he states: “drill-focused form of teaching to the test can crowd out opportunities to teach students more advanced cognitive skills, such as how to solve problems and communicate effectively” In a more prospective view, Jerald alarms that those students who are habitually taught to the test and denied the opportunity to develop such advanced skills will suffer an increasing disadvantage

in the current employment market which strictly requires complex ability and true qualification Therefore, by “teaching to the test”, he concludes that teachers are

“trading long-term benefits to students for short-term gains on standardized tests”

In his recent critique of “teaching to the test”, Phelps (2011) emphasizes the mutually exclusive relationship between test preparation and good teaching practices The act of completely aligning classroom instruction with test-oriented activities means that every teacher has no choice but teaches the same content and teaches to the test Phelps argues that this is an “anathema notion” to many education professors and those who hold a professional view that “each and every teacher is a skilled and creative craftsperson who designs unique instructional plans for unique classrooms” He blames “teaching the test” in that way for creating a kind of standardizing instruction which “de-skills” teachers and therefore, “teaching to the test must always be wrong”

As for language assessment, Hamp-Lyons (1998) criticizes extensive test preparation for preventing the utilization of Communicative Language Teaching (CLT) Investigating an array of TOEFL commercial textbooks, she comments that these self-study materials while containing practice tests, exercises, tape, tape script, and answer key which helps learners to score themselves, are inadequate for developing autonomy since they do not help students to diagnose problem areas, strengths or weaknesses In fact, as commercial test packages, these materials are built around the model of tests with focus on test-taking strategies, mastery of

The author then argues that such textbooks can lead to teaching and learning discrete chunks of language rules and vocabulary items without context, and therefore “have nothing to say to a pedagogy” Such commercial textbooks which encourage “teaching to test” will do more harm than good as they channel learners’ time and energy from true development of communicative proficiency into unproductive, test-mimicking exercises

“Teaching to the test” is particularly preferred and popular in Asian countries, where there exists a long-standing tradition of using testing for selective purposes Under such huge pressure of high-stakes condition, students are probably more eager to learn for tests This situation, on the downside, implies merely extrinsic motivation which does not have a lasting effect and help build up learners’ autonomy (Brown, 2001:77) In fact, several studies on washback in Asian context report that “teaching to the test” in language classroom, regardless of its practicality, results in test anxiety and tiredness, teacher-centered approach and lack of productive language skills among students whose communicative competence is primarily claimed to be measured (Nguyen Thai Son, 2012)

2.2.2.2 Arguments for “teaching to the test”

Those who support “teaching to the test” can claim this is not an illegitimate practice but teaching to the “agreed-upon goals” (Shepard, 1990) If tests are well-constructed, then “teaching to the test” means teaching to set standards and thus dramatically improving the efficiency and effectiveness of instruction They can also argue that students’ performance in a test, regardless of potential score pollution, still reflects somewhat of achievement at least through that particular test This means “teaching to the test” is acceptable as long as it is well-managed

so that score pollution is kept at a minimum (Phillips, 2006)

In a well-rounded view, Mehrens et al (1998) and Perlman (2003) argue that

“teaching to the test” can have both positive and negative effects on test validity, depending on how it is controlled The upside is that learning about test format can help students reduce anxiety and eliminate the potential sources of error variance

caused by students’ unfamiliarity with the test and therefore, improves the validity

of test scores In fact, Smith’s 1991 study reports that teachers can conduct a type

of test preparation in regard to stress inoculation The primary aim of this activity

is to boost students’ confidence and self-efficacy, not for the sake of increasing scores so that students can optimize their ability and complete the test at their best

On discussing the feasibility of authentic tests as a solution to the problem of inflated scores, Miller and Seraphine (1993:128) blame the shortcomings of

“teaching to the test” on high-stakes conditions as they come to the following conclusion:

The nature of high-stakes assessment will continue to narrow instruction

which, in turn, will narrow the interpretation of the construct measure,

regardless of the form of assessment As long as the assessment pressures

students, teachers, and administrators to do well, instruction will be

narrowed to more accurately reflect the sample of skills, methods, and

content measured by assessment, rather than broadened to cover the

domain the assessment is intended to sample from and generalize to in the

interpretation of test scores

This view is supported by Crocker (2005) who points out that under high-stakes conditions; teachers are shackled to measurement-driven instruction “Teaching to the test” then turns out to be a convenient and practical approach in response to high-stakes From teachers’ perspective, Burgess and Head (2005) point out several reasons why the English instructors can find teaching an exam class

“extremely satisfying”: (1) learners are more motivated thanks to a shared goal and desire to succeed in their study, (2) the syllabus is clearly defined and greatly supported with a lot of published materials, (3) teachers are rewarded by the feeling of achievement on contributing to their students’ success brought about by good exam results

2.2.2.3 Issues around test-wiseness training

However, a large and growing body of literature has concentrated on test-wiseness training as a common practice constituting “teaching to the test” Canale and Swain (1980), on proposing a theoretical framework for Communicative Competence, add strategic competence as a component of communicative language use They realize that students performing a language task tend to make use of strategies to compensate for problems or deficiencies in communication so that they can still successfully complete such tasks

Regarding language assessment, Brown (2005:173) considers this as students’ experience in test-taking which can be “developed by some students to an art form” Then, if the purpose of a test is to measure students’ acquired knowledge and skills, test-wiseness is a potential source of measurement error Phelps (2011) even finds it dishonest and unethical to excessively focus on test-taking techniques

in compensation for the subject matter What perfectly exemplifies this issue can

be found in advertisements for cramming courses and commercial textbooks which guarantee test scores for even last-minute preparation This means students do not have enough time and need not to master necessary content to do well on tests For

a fee of several hundred dollars, they would be drilled in test-taking techniques that the advertisements promised would increase scores In agreement with these authors, Derrick (2013) points out that these tricks can be workable for an attempt

to outsmart MCQs in poorly-written traditional exams, yet they are a “waste of time in terms of language acquisition”

The proponents of test-wiseness training argue that test-taking skills are not a “bag

of tricks” that can magically help ill-prepared students to get by (Crocker, 2005)

As a matter of fact, they define test-wiseness as “a sophisticated approach to taking tests the ability to use characteristics of tests and test-taking situations to reach the full potential of one's knowledge and aptitudes” (Millman, 1969: xiii) Scruggs and Mastropieri (1992) even confirm that students who are test-wise can outperform those of equal ability but lacking test-wiseness Regarding that such

disadvantage increases if combining with test-anxiety, it seems unfair and unethical to subject students to a test without preparing them for that particular test This can be compared to a scenario where someone is put into a competition without being notified of the rules and regulations

According to Cohen (1994), test-taking strategies can simply be viewed as learning strategies, a set of cognitive skills that learners consciously use to accomplish a task This view is supported by Celce-Murica (2007:50) who confirms that learners with effective use of strategies “tend to learn languages better and faster than those who are strategically inept” Moreover, data from several sources suggest that some types of language test tasks are particularly susceptible to the effects of test-takers’ strategic competence and to a certain extent measure test-wiseness (Bachman, 1990; Cohen, 2006) Cohen (2006) provides an example of language test in which students were warned that they would encounter vocabulary they had not seen before and that they could still answer the questions using skill and initiative This means that some tests, by the nature of its construct, allow test-takers to arrive at correct answers on the basis of test-wiseness rather than knowledge and skills of the tested content

Concerning principles for designing effective classroom tests, Brown (2001: 408) addresses training strategies for test-takers as the first principle which guarantees that students are able to “demonstrate their competence through an optimal level of performance” Elsewhere, Crocker (2005) adds that teaching test-taking skills, in fact, improves test validity since it helps diminish CIVs rooting from students’ lack of comprehension of directions and inability to manage time efficiently Indeed, due to this aspect, test-wiseness training is generally considered defensible and appropriate practice in the perceptions of many assessment experts, administrators, educators and teachers (Mehrens and Kaminski, 1989; Popham, 1991; Haladyna et al., 1991; Moore, 1994; Lai and Waltman, 2008)

Nonetheless, a crucial question is whether test-wiseness training works and truly supports test-takers to successfully complete a test Dolly and Williams (1986)

conduct an empirical study to seek answers to 2 questions: (i) can cognitive strategies to increase test-wiseness be taught? and (ii) can the test-wiseness strategies be generalized by students so that they can improve scores in examinations other than those measuring test-wiseness? The results from 54 undergraduates show that the experimental group who received one hour of lecture

on strategies significantly outscored the control group in measurement of wiseness and test questions susceptible to the acquired strategies However, there

test-is little evidence to support that those students can generalize such test-wtest-iseness strategies to highly score in other examinations This obviously implies that test-wiseness can be trained to improve students’ performance for a particular susceptible test but hardly compensate for knowledge and skills of the tested content The authors then comment that test-takers, all in all, have to use their cognitive strategies to recognize the correct answer whenever they lack information concerning the test items But how can a test-taker decide which option is the best guess? Without instruction in strategies, they will probably either leave the item blank or guess wildly

Several attempts have also been made to investigate the effects of test-wiseness training on students’ performance (Bangert et al., 1983; Samson, 1985; Scruggs et

al 1986) Bangert’s analysis (1983) shows that coaching progammes indeed boost students’ score in achievement test up to 0.25 standard deviation However, the effect differs with the level of training intervention: (i) smaller for short test-taking orientation session, (ii) larger for more extensive programmes of drill and practice, and (iii) largest in a single lengthy programme designed to improve broad cognitive skills This result is confirmed by later empirical studies conducted by Samson (1985) and Scruggs et al (1986) who conclude that longer training programmes will be more effective Despite limited evidence, Scruggs ‘research suggests that training in test-taking skills also positively affects students’ anxiety, self-esteem, and attitude towards tests The fact that test-wiseness has positive effects on test-takers is then confirmed by Yang (2000:172) who conducted an

empirical study on Chinese TOEFL candidates and reached the following conclusion:

Test-wise students had a more meaningful, thoughtful, logical and less

random approach to the items than did the test-nạve students In addition,

they were more academically knowledgeable and used that knowledge to

assist them in figuring out answers to questions

The debate around “teaching to the test”, in sum, has attracted a considerable amount of literature in various related areas such as educational assessment, language testing, educational psychology, etc The inappropriate practices of

“teaching to the test” generally faces criticism for its harm to the validity of test scores and preventing accurate inferences from scores to broader domain of achievement Extensive focus on test preparation also means that the systematic, carefully-planned curriculum can be distorted and narrowed to serve the testing purposes Testing instead of playing a supportive role turns out to drive teaching and learning, obstructing learners from acquiring more advanced skills “Teaching

to the test”, in addition, elicits teacher-centeredness, test-anxiety, and issue of fairness in testing

In contrast, those who defend “teaching to the test” claim that they are teaching to agreed-upon standards constructed in the test Preparing students for tests also helps diminish CIVs caused by students’ unfamiliarity with tests From teachers’ perspective, teaching an exam class can be “extremely satisfying” thanks to its convenience and rewarding feeling of achievement on contributing to students’ success Test-wiseness training is defensible and appropriate practice in perceptions of most educators and its positive effects on test-takers and their performance have been proved by empirical studies The debate reviewed so far has provided quite a clear picture about “teaching to the test” which in turn offers useful suggestions to the instructors who wish to help their students truly develop necessary skills and knowledge as well as highly achieve in standardized tests A summary of “teaching to the test” debate is presented in Table 2.3

Table 2 3 Summary of “teaching to the test” discussion

TEACHING TO THE TEST

a 1 Harmful to test score validity

 Preventing accurate inferences (Mehrens &

2 “De-skilling” teachers (Phelps, 2011)

1 Legitimate teaching toward set standards

(Shepard, 1990; Phillips, 2006)

2 Convenient and practical teaching in response

to high-stakes tests (Miller & Seraphine, 1993;

Crocker, 2005)

3 Improving test validity by diminishing CIVs

(Mehrens et al., 1998; Perlman, 2003)

4 Possibly “extremely satisfying” from teachers’

perspective (Burgess & Head, 2005)

2 Misleading belief that test-taking

techniques can compensate for lack of

knowledge (Phelps, 2011; Derrick, 2013)

3 Eliciting drilling approach (Phelps,

2011)

4 Workable for poorly-written traditional

exams only (Derrick, 2013)

1 Constructing learning strategies/ cognitive abilities (Millman, 1969; Cohen, 1994, Celce- Murcia, 2007)

2 Some tests actually measure test-wiseness

(Bachman, 1990; Cohen, 2006)

3 Eliminating CIVs (Brown, 2001; Crocker, 2005)

4 Possible to be trained (Dolly & Williams, 1986)

5 Raising Ss’ achievement (Bangert et al., 1983;

Samson, 1985; Scruggs et al 1986)

6 Eliciting positive affective factors (Samson,

2 “Drill and kill” teaching (Jerald, 2006)

3 Promoting teacher-centeredness

(Hamp-Lyons, 1998; Nguyen Thai Son, 2012)

Eliciting test anxiety and extrinsic

motivation (Brown, 2001; Nguyen Thai Son,

2.2.3 Ethical continuum for evaluation

By and large, the discussion over “teaching to the test” has not been settled, yet test preparation courses continue in widespread use in response to high-stakes standardized tests In an attempt to provide a guideline of what constitutes appropriate student preparation for testing, several ideas about an “ethical continuum” have been put forward

The first serious discussion and analysis of this emerged in late 1980s with an

“ethical continuum” of “teaching to the test” proposed by Mehrens and Kaminski (1989) This includes seven descriptive points, ranging from the most to the least ethical as follows:

i general instruction on objectives not determined by looking at the objectives measured on standardized tests

ii teaching test-taking skills

iii instruction on objectives generated by a commercial organization where the objectives may have been determined by looking at objectives measured by a variety of standardized tests

iv instruction based on objectives (skills and sub-skills) that specifically match those on the standardized test to be administered;

v instruction on specifically matched objectives (skills and sub-skills) where the practice or instruction follows the same format as the test questions;

vi practice or instruction on a published parallel form of the same test vii practice or instruction on the test itself

The study then concludes that the ethical boundary fell between points iii and v, with points i and ii being ethical and points vi and vii being unethical This

guideline apparently implies that the more closely test preparation is geared towards the test, the less ethical and inappropriate it is

Haladyna et al (1991), on examining sources and causes of “test score pollution”, provide an appraisal of the ethics of test preparation practices Compared to Mehrens and Kaminski’s ethical continuum, this guideline (Table 3.6) more explicitly indicates whether a practice is ethical and appropriate although the authors emphasize that their view is not shared universally

Test preparation activities Degree of ethicality

Training in test wiseness skills Ethical

Checking answer sheets to make sure

that each has been properly completed Ethical

Increasing student motivation to

perform on the test through appeals to

parents, students, and teachers

Ethical

Developing a curriculum based on the

Preparing objectives based on items on

the test and teaching accordingly Unethical

Presenting items similar to those on

Using Scoring High or other score

Dismissing low-achieving students on

testing day to artificially boost test

scores

Highly unethical

Presenting items verbatim from the

Table 2 4 A continuum of test preparation activities by Haladyna et al (1991)

However, regardless of their explicit indication, these guidelines seem rather confusing since the continuum is not well defined yet In fact, what criteria are actually used to determine a practice is ethical or unethical? Popham (1991:13) argued for a straightforward evaluation of preparation practices with reference to two evaluative criteria: (1) professional ethics (no test-preparation practice should violate the ethical standards of the education profession) and (2) educational defensibility (no test preparation practice should increase students’ scores without

actually simultaneously increasing student mastery of the content domain tested Then, based on these proposed criteria, he evaluated 5 generally defined test preparation practices as follows:

i Previous-form preparation: special instruction and practice with the

current test (violates both standards  inappropriate)

ii Current-form preparation: special instruction and practice with the

current test (violates both standards  inappropriate)

iii Same-format preparation: regular instruction covering current test content

and uses practice items of the same format as test items (not educationally defensible  inappropriate)

iv Generalized taking preparation: special instruction covering

test-taking skills for a variety of test format (appropriate)

v Varied-format preparation: regular instruction covering current test

content and uses a variety of practice item formats (appropriate)

Taken together, these researchers generally agree that test-wiseness training and motivational activities are appropriate and ethical whereas practices which reduce

or narrow the curriculum are unethical or even highly unethical As for teaching test content and format, previous-form preparation and current-form preparation are inappropriate and educational indefensible while varied-format preparation is considered ethical Same-format preparation could be regarded as “grey area” since this activity is marked inappropriate by Popham (1991) for its educational indefensibility but undecided by the other authors (Mehrens and Kaminski, 1989; Haladyna et al., 1991)

Subsequently, Moore (1994) goes further by examining whether teachers and testing specialists could reach a consensus on appropriate test preparation with reference to those guidelines suggested above Empirical evidence reveals that teachers consistently rate practices to be more appropriate than do testing specialist In fact, significant differences between two groups of subjects were found for 6 out of 8 categories of preparation activities Then the author concludes