CHUYÊN đề bồi DƯỠNG học SINH GIỎI assessment and testing in EFL class at high school level

In the scope of this paper, we will examine some general principles of constructing multiple-choice items and try to find out some common problems in designing multiple-choice questions,

HỘI THẢO KHOA HỌC CÁC TRƯỜNG THPT CHUYÊN KHU VỰC DUYÊN HẢI & ĐỒNG BẰNG BẮC BỘ

LẦN THỨ VIII

CHUYÊN ĐỀ BỒI DƯỠNG HỌC SINH GIỎI Assessment and Testing in EFL Class at High School Level

Người thực hiện: Nguyễn THỊ CẨM TÚ

Đơn vị: Trường THPT Chuyên Nguyễn Bỉnh Khiêm - tỉnh Quảng Nam.

I INTRODUCTION

he classroom test is one of the most important aspects of the teaching -learning process, and designing the classroom test is one of the most demanding responsibilities facing school instructors Unfortunately, most of teachers of English at Vietnam’s upper-secondary school have had little, if any, preparation

in the craft of wring tests; consequently, the process is not only difficult, it is also frustrating and often ineffective

Writing test questions will always be demanding, even for experienced instructors, but it will be less frustrating for those who know the techniques for writing specific types of items and have some guidelines for general test constructions

The objective test is only of many ways in which students can be evaluated Tests can be formal or informal, oral or written; and no one form of testing is necessarily better or worse than another

It is useful to consider multiple-choice items in some detail, as they are undoubtedly one of the most widely used types of items in objective tests However, since the multiplechoice item is one of the most difficult and time -consuming types of items to construct, numerous poor multiple-choice tests now abound

In the scope of this paper, we will examine some general principles of constructing multiple-choice items and try to find out some common problems

in designing multiple-choice questions, then we will give some suggestions for designing multiple-choice questions

II THE GENERAL PRINCIPLES OF CONSTRUCTING

MULTIPLE-CHOICE ITEMS

A multiple-choice item is composed of two parts: a stem that identifies the question or problem, and a set of alternatives or possible answers that contain a

key that is the best answer to the question, and a number of distractors that are

plausible but incorrect answers to the question Students respond to multiple-choice items by indicating the alternative that they believe best answers or completes the stem The task of a distractor is to distract the majority of poor students (i.e those who do not know the answer) from the correct option

According to J B Heaton (1988) the general principles should be observed when multiple-choice items are constructed:

1 Each multiple choice item should have only one answer This answer must be absolutely correct unless the instruction specifies choosing the best option (as in some vocabulary tests) Although it may seem an easy matter, it is sometimes extremely difficult to construct an item having only one correct answer

2 Only one feature at a time should be tested: it is usually less confusing for the testees and it helps to reinforce a particular teaching point

3 Each option should be grammatical correct when placed in the stem, except of course in the case of specific grammar test items For example, stems

ending with the determiner a, followed by options in the form of nouns or noun

phrases, sometimes traps the unwary test constructor

4 All multiple-choice items should be at a level appropriate to the proficiency level of the testees The context, itself, should be a lower level than the actual problem which the item is testing: a grammar test item should not contain other grammatical features as difficult as the area being tested, and a vocabulary item should not contain more difficult semantic features in the stem than the area being tested

5 Multiple-choice items should be as brief and as clear as possible (though it is desirable to provide short contexts for grammar items)

6 In many tests, items are arranged in rough order of increasing difficulty It is generally considered important to have one or two simple items

to ‘lead in’ the testees, especially if they are not too familiar with the kind of test being administered Nevertheless, areas of language which are trivial and not worth testing should be excluded from the test

III SOME COMMON PROBLEMS IN DESIGNING

MULTIPLE-CHOICE QUESTIONS (MCQs)

1 Common problems in designing the stem

a There appears to have grammatical clues in the options e.g capital letters, a or an, etc.

Example:

He was badly injured in an yesterday

A traffic jam B accident C crash D collision

The article an in the above question helps the testee easily know the key

(B)

b The stem does not have sufficient information and it contains words or phrases which repeated in each option.

Example:

Choose the best option A, B, C or D to complete the following sentence by circling the letter you choose

He is

A the man whom I talk about B the man which I talk about

C the man that I talk about D the man that I talk

c The stem is not clear and this makes the testees confused.

Example:

What is not mentioned in the text as one of her excuses?

NOT must be written in capital letter so that the testee can pay attention to the requirement of question

d The stem contains unnecessary information.

Example:

All of the following sentences, which contain relative clauses, are correct

EXCEPT for

A The suitcase which my father bought was upstairs.

B The suitcase my father bought was upstairs

C The suitcase bought by my father was upstairs

D The suitcase buying my father was upstairs

“which contain relative clauses” is an unnecessary information, because

the testees can understand the stem without it

In short, the primary purpose of the stem is to present the problem clearly and concisely The testee should be able to obtain from the stem a very general idea of the problem and the answer required At the same time the stem should not contain extraneous information or irrelevant clues, thereby confusing the problem being tested Unless students understand the problem being tested, there

is no way of knowing whether or not they could have handled the problem correctly Although the stem should be short, it should convey enough information to indicate the basis on which the correct option should be selected

2 Common problems in designing the lead-in

The lead-in does not present the problem clearly, sufficiently and concisely.

Example:

Choose the best answer

Choose the letter of the best answer

Choose the correct answer

Circle the best answer

These lead-in parts do not provide the criteria for the testees to answer the questions

Let’s consider the following lead-in from the TOEFL sample test:

Questions 1-15 are incomplete sentences Beneath each sentence you will see four words or phrases, marked A, B, C, and D Choose the one word or phrase that best completes the sentence Then, on your answer sheet, find the number of the question and fill in the space that corresponds to the letter of the answer you have chosen Fill in the space so that the letter inside the oval cannot

be seen

This lead-in part is long but sufficient The testees will know how to do the test The lead-in is not necessarily long as in the example above but the sufficient criteria are prior

3 Common problems in designing the options (key and distractors)

3.1 The item has more than one correct option (double key).

Example:

Choose from the prepositions marked A, B, C and D the correct one to fill in the blank by circling the corresponding letter A, B, C or D

weekends, John often goes fishing with his friend George

3.2 The item has no correct option (no key).

Example:

Choose one word whose underlined part is pronounced differently from the others by circling the corresponding letter A, B, C or D

A celebration B certain C century D excellent

3.3 The distractors or incorrect options are reasonably unattractive and implausible.

Example:

He is a very honest man; he never lies

Choice B in the above item is much below the level being tested and will

be eliminated by testees immediately: their chances of selecting the correct option will then be one in three

3.4 The options are unparallel.

Every atlas has its on legend

The word legend in this sentence probably means:

C A part that explains meanings of symbols D special color

Choice C attracts the testees’ attention because it is longer and its structure is completely different from the others Thus, the testees tend to choose this option for their answer (in this case, it is the choice C that is the correct answer)

IV CONCLUSION - SUGGESTIONS

The following suggestions for designing MCQs are organized into three sections: 1) General Strategies, 2) Designing Stems, and 3) Designing Alternatives

General Strategies

- Write questions throughout the term Multiple-choice question exams

are challenging and time-consuming to create The teacher will find it easier if he/she writes a few questions each week, perhaps after a lecture when the course material is still fresh in his/her mind

- Instruct students to select the “best answer” rather than the

“correct answer” By doing this, the teacher acknowledges the fact that the

distractors may have an element of truth to them and discourage arguments from students who may argue that their answer is correct as well

- Use familiar language The question should use the same terminology

that was used in the course Avoid using unfamiliar expressions or foreign language terms, unless measuring knowledge of such language is one of the goals of the question Students are likely to dismiss distractors with unfamiliar terms as incorrect

- Avoid giving verbal association clues from the stem in the key If the

key uses words that are very similar to words found in the stem, students are more likely to pick it as the correct answer

- Avoid trick questions Questions should be designed so that students

who know the material can find the correct answer Questions designed to lead students to an incorrect answer, through misleading phrasing or by emphasizing

an otherwise unimportant detail of the solution, violate this principle

- Avoid negative wording Students often fail to observe negative

wording and it can confuse them As a result, students who are familiar with the material often make mistakes on negatively worded questions In general, avoid having any negatives in the stem or the options In the rare cases where the teacher uses negatives be sure to emphasize the key words by putting them in upper case and bolding or underlining them

Designing Stems

- Express the full problem in the stem When creating the item, the

teacher wonders if the students would be able to answer the question without looking at the options This makes the purpose of the question clear

- Put all relevant material in the stem Do not repeat in each of the

alternatives information that can be included in the stem This makes options easier to read and understand, and makes it easier for students to answer the question quickly

- Eliminate excessive wording and irrelevant information from the stem Irrelevant information in the stem confuses students and leads them to

waste time

Designing Alternatives

- Limit the number of alternatives Use between three and five

alternatives per question Research shows that three-choice items are about as effective as four or five-choice items, mainly because it is difficult to come up with plausible distractors

- Make sure there is only one best answer Avoid having two or more

options that are correct, but where one is “more” correct than the others The distractors should be incorrect answers to the question posed in the stem

- Make the distractors appealing and plausible If the distractors are

farfetched, students will too easily locate the correct answer, even if they have little knowledge When testing for recognition of key terms and ideas keep the distractors similar in length and type of language as the correct solution When testing conceptual understanding, distractors should represent common mistakes made by students

It is unlikely that students would choose options a, b, c or d, even if they

didn’t know the answer A better question would have plausible links between the stem and the distractors:

- Make the choices grammatically consistent with the stem Read the

stem and each of the choices aloud to make sure that they are grammatically correct

- Place the choices in some meaningful order When possible, place the

choices in numerical, chronological or conceptual order A better structured question is easier to read and respond to

- Randomly distribute the correct response The exam should have

roughly the same number of correct answers that are a's, b's, c's, and d's (assuming there are four choices per question)

- Avoid using “all of the above” If “all of the above” is an option and

students know two of the options are correct, the answer must be “all of the above” If they know one is incorrect, the answer must not be “all of the above”

A student may also read the first option, determine that it is correct, and be misled into choosing it without reading all of the options

- Avoid using “none of the above” The option “none of the above” does

not test whether the student knows the correct answer, but only that he/she knows the distractors aren’t correct

- Refrain from using words such as always, never, all, or none Most

students know that few things are universally true or false, so distractors with these words in them can often be easily dismissed

- Avoid overlapping choices Make the alternatives mutually exclusive.

It should never be the case that if one of the distractors is true, another distractor must be true as well

- Avoid questions of the form “Which of the following statements is correct?” There is no clear question being asked, and the choices are often

heterogeneous Such questions are better presented in the form of True/False questions

TỔ NGOẠI NGỮ

THPT CHUYÊN NGUYỄN BỈNH KHIÊM QUẢNG NAM

REFERENCES

1 Do, T M (2005) Designing an Objective English Test at

Upper-Secondary Schools, VNU, Journal of Science, Foreign Language, T.XXI, No

4AP

2 ELIN 823 Module 3 (2003) Assessment in the Language Curriculum.

Victoria, University of Wellington

3 Harris, M and McCann, P (1994) Assessment Macmillan Publishers

Limited

4 Heaton J B (1988) Writing English Language Test Longman Group

UK Limited Group

5 Nguyen, Q T (2005) Chuẩn đánh giá trong dạy và học ngoại ngữ, cited in Kỷ yếu Hội thảo Quốc gia “Chuẩn đánh giá và công cụ đo trong đào

tạo cử nhân ngoại ngữ - Lý luận và thực tiễn” Danang, Vietnam.

6.http://www.adm.uwaterloo.ca/infotrac/tips/

designingmultiplechoicequestions.pdf

7 http://www.lts.rmit.edu.au/renewal/assess/faq_mcq.doc

8 http://www.uwe.ac.uk/elearning/mcq.pdf

9 http://edu.georgianc.on.ca/teaching/pub/pedagoggles/MCQs_2_4.pdf