Slide Issues in language testing

Slide 1 Issues in language testing Group 7 – K18C Nguyen Thi Bich Ngoc (12 08) Do Thu Phuong Do T Phuong Thanh Tran Thi Thu Trang (30 03) Tran Thi Thu Trang (18 10) Tran Thi Thu Hien Nguyen Thi Nhai Le Thi Nhung Nguyen Thi Thanh Tam Nguyen Thi Thanh Le Huong Thao Ngo Thi Thom AN INTRODUCTION TO LANGUAGE TESTING 1 1 Testing and Teaching 1 2 Why Test? 1 3 Types of Test 1 DIFFERENT APPROACHES TO LANGUAGE TESTING 2 1 The essay translation approach 2 2 The structuralist approach 2 3 The integrative a.

Tran Thi Thu Trang (30.03)

Tran Thi Thu Trang (18.10)

Tran Thi Thu Hien

Nguyen Thi Nhai

Le Thi Nhung

Nguyen Thi Thanh Tam

Nguyen Thi Thanh

Le Huong Thao

Ngo Thi Thom

AN INTRODUCTION TO LANGUAGE TESTING

1.1 Testing and Teaching 1.2 Why Test?

1.3 Types of Test

1

DIFFERENT APPROACHES TO LANGUAGE TESTING

2.1 The essay-translation approach 2.2 The structuralist approach

2.3 The integrative approach 2.4 The communicative approach

2

PRACTICAL ASPECTS OF DESIGNING A LANGUAGE TEST

3.1 Quality of a good test 3.2 Stages of designing a language test

3

1 AN INTRODUCTION TO LANGUAGE

TESTING

1.1 TESTING AND TEACHING

 “ A test is a method of measuring a person’s ability or

knowledge in a given domain”

(Brown, D.H.2001 Teaching by principles Long man)

 Testing is when you want to know how much someone knows

by asking questions

(Long man's Active Study Dictionary )

 Teaching is when you teach someone something, you give them instructions so they know about it or how to do it; You make them think, feel or act in a new or different way; You explain or show students how to do something

(Collins' COBUILD Dictionary )

1.1 TESTING AND TEACHING

TEACHING TESTING

Closely interrelated

The teaching may determine the test => Tests = devices to reinforce learning and to motive students

The test decides the teaching => Tests = a means of assessing students’ performance

1.1 TESTING AND TEACHING

Mechanical

test of

structure

Tests of grammar, translation or language manipulation

Negative and even harmful effect on teaching

Communicati

ve test of

language

A focus for purposeful everyday communication

- Much more positive effects

on learning and teaching

- Improved learning habits.

own effectivene

ss =>

Students benefit more

*Diagnose the

student’s weaknesse

s and difficulties

=> Render assistance

*Evaluate the

effectiveness of the syllabus as well as the methods and

materials being used

*Motiva

te students

1.3 TYPES OF TESTS

1.3.1 Achievement/ Attainment tests

1.3.2 Proficiency tests

1.3.3 Placement tests

1.3.4 Aptitude tests

1.3.5 Diagnose tests.

1.3.1 Achievement/

Attainment test

 Formal tests designed to show

mastery of a particular syllabus or corpus

of language

2 kinds

CLASS PROGRESS

TESTS

STANDARDISE

D ACHIEVEMENT

TESTS

CLASS PROGRESS TESTS

 measure how well learners have mastered the material [or skills] taught in class and [or] how well they have improved in a given area.

-Constructed by the class’s teacher.

- Based on the language program which the class has been following

-Each test situation is unique.

-It is as important as an assessment of the teacher’s own work as the student’s own learning.

a teaching device

CLASS PROGRESS TESTS

•A good progress test should:

+ Encourage sts’s performance and

STANDARDISED ACHIEVEMENT

TESTS

intended to measure achievement

on a larger scale.

E.g.: University/ college/ school

entrance exam; school certificate

examination; annual school examinations.

-The constructers of such tests rarely teach any of the students being tested.

- The test is often based on a published

syllabus

- It exerts a strong influence on the teaching

in schools

A good achievement test

should reflect the particular

approach to learning and teaching that has previously been adopted

STANDARDISED ACHIEVEMENT

TESTS

Proficiency Tests are not usually based

on a particular syllabus but are used to measure achievement in relation to a

specific task that the candidates may be required to perform at a subsequent

point of time.

 ‘Proficient’ = having sufficient

command of the language for a

particular purpose.

 Other proficiency tests do not have

any occupation or course of study in

1.3.3 Placement tests

Placement tests

-used to place students at

appropriate level of instruction

1.3.4 Aptitude tests

- used to measure capacity for

perceive sound and spelling correspondences

and the ability to identify patterns in language ].

1.3.5 Diagnostic tests

Diagnostic Tests enable teachers to

identify any areas of weakness or

difficulty, so that they can then plan and implement an appropriate

remedial teaching program.

 They may be used to assess the

knowledge and skills of learners in

particular aspects of language before the start of a course [ and consequently may

be used for placement as well as course design

purposes ]

2.1 The essay-translation approach

2.2 The structuralist approach 2.3 The integrative approach

2 4 The communicative approach

2.1 The essay-translation approach

- It is commonly referred to as the pre-scientific

stage of language testing.

- Tests usually consists of essay writing,

translation and grammatical analysis.

- The tests also have a heavy literary and

cultural bias.

- Language learning is chiefly concerned with the systematic acquisition of a set of habits.

- It draws on the importance of contrastive

analysis and the need to identify and measure the learner’s mastery of the separate elements

of the target language The skills are also

separated.

2.2 The structuralist approach

- This approach involves the testing of

language in context.

- Tests are designed to assess the learner’s

ability to use two or more skills

2.3 The integrative approach

- They measure the reader’s

ability to decode ‘interrupted’

or ‘mutilated’ messages by

making the most acceptable

substitutions from all the

contextual clues available

(example)

- Texts neutral in both

content and language should

be used.

- Cloze testing is a good

indicator of general linguistic

ability.

- The integrated skills involved are auditory discrimination, the auditory memory span, spelling, the recognition of sound

segments, a familiarity with the grammatical and lexical patterning of the language, and overall textual

comprehension

- Teachers should dictate

in meaningful units of sufficient length (example)

Cloze tests Tests of dictation

2.4 The communicative approach

- Communicative tests are concerned primarily with how language is used

in communication.

- Success is judged in terms of the effectiveness of the communication.

- Communicative testing results in an attempt to obtain different profiles of

a learner’s performance in the language.

- They must reflect the culture of a particular country.

- The most important criterion for communicative tests is they should be based on precise and detailed specifications of the needs of the learners.

- Qualitative modes of assessment in preference to quantitative ones should

be used (example)

PRACTICAL ASPECTS OF DESIGNING A

2 Stages of test construction

2.1 Writing test specification

2.2 Writing the test

2.3 Piloting the test/ Pretesting

3 Washback 

QUALITIES OF A GOOD TEST

1.1 Reliability

1.2 Validity

1.3 Practicality

1.1 1 WHAT IS RELIABILITY?

“Reliability is a measure of accuracy, consistency,

dependability, or fairness of scores resulting from the administration of a particular examination”.

(Henning, 1987)

1.1 1 WHAT IS RELIABILITY?

 A reliable test: measures consistently whatever it is supposed

to measure (consistency of measurement).

 Reliability: necessary, but not sufficient, condition of validity (discussed later)

WHICH ONE IS MORE RELIABLE?

1.1.2 FACTORS THAT INFLUENCE THE

1.1.2 FACTORS THAT INFLUENCE THE

RELIABILITY OF A TEST

1.1.2.2 Test Item Level of Difficulty

- Unreliable test items: too hard or too easy

- Reliable test items: 1/3- 2/3 of the candidates will get it right

- Over the whole test, especially with achievement tests:

• 1/3 scoring less than 45% (weaker candidates)

• 1/3 scoring 45%- 65% (average candidates)

• 1/3 scoring 65%- 100% (good candidates)

1.1.2 FACTORS THAT INFLUENCE THE

• immediately after a very good composition?

• immediately after a very poor composition?

1.1.2 FACTORS THAT INFLUENCE THE

RELIABILITY OF A TEST

1.1.2.4 Syllabus

- How much the test is based on passages and questions taken

directly from a textbook

- How much the test is based on the syllabus within the text book

1.1 3 HOW TO MAKE TESTS MORE RELIABLE

1 Take enough samples of behaviour: the more items a test has,

the more reliable it is

2 Do not allow candidates too much freedom: candidates should

not be given a choice, and the range over which possible

answers may vary should be restricted.

3 Write unambiguous items achieved by pretesting

4 Provide clear and explicit instructions

5 Ensure that tests are well laid out and perfectly eligible

6 Candidates should be familiar with format and testing

techniques

1.1 3 HOW TO MAKE TESTS MORE

RELIABLE

7 Provide uniform and non-distracting conditions of administration

8 Use items that permit scoring which is as objective as possible

:MCQs, open-ended questions that has a unique, possibly one-word, correct response

9 Provide a detailed scoring key

10 Train scorers

11 Agree acceptable responses and appropriate scores at outset of

scoring

12 Identify candidates by number, not name

13 Employ multiple, independent scoring

1.2 VALIDITY

1.2.1 Aspects of validity

 Face validity: the test looks as if it measures what it is supposed to measure.

 Content validity: the content of the test constitutes a representative sample

of the language skill, structures, etc with which it is meant to be concerned.

 Response validity: describes the extent to which examinees responded in the manner expected by the test developers.

 Statistical or empirical validity: the validity obtained as a result of

comparing the results of the test with the results of some criterion measure.

 Construct validity: it can be demonstrated that a test/ part of a test/ a

testing technique measures just the ability which it is supposed to measure.

1.3 PRACTICALITY

1.3.1 Definition

Practicality: the relationship between the required resources in the design, development and use of the test, and the available resources for these

1.3 PRACTICALITY

1.3.2 Condition of a practical test

Available resources Practicality =

Required resources

 If practicality ≥ 1, the test development and use is practical.

 If practicality < 1, the test development and use is not

practical.

2 STAGES OF TEST

CONSTRUCTION

2.1 Writing test specification

2.2 Writing the test

2.3 Piloting the test/ Pretesting

2.1 WRITING TEST SPECIFICATIONS

2.1.1 What are Test Specifications ?

2.1.2 Who need Test Specifications?

2.1.3 Checklist for Test Specifications

2.1.1 WHAT ARE TEST

SPECIFICATIONS?

 Provide the official statement about what the

test tests and how it tests it.

 Are the blueprint for test writers to follow.

 Should be distinguished from Syllabus

2.1.1 WHAT ARE TEST

 Public usage

 For teachers and students to prepare for the test.

Test specifications vs Syllabus

2.1.2 WHO NEED TEST

 Publishers of textbooks and practice tests

 Teachers and Students

2.1.3 CHECK LIST FOR TEST

SPECIFICATIONS?

Test Specifications should include ALL or MOST of the

following:

1 The test purposes

2 Descriptions of test takers

3 Test level

4 Construct (theoretical framework)

5 Description of suitable language course

6 Number of sections/papers

7 Time allocation

8 Weighting for each section

9 Target language situation

2.1.3 CHECK LIST FOR TEST

SPECIFICATIONS?

10 Text –types

11 Text length

12 Tested language skills and elements

13 Test task and methods

15 Rubrics

16 Criteria for marking

17 Descriptions of typical performance

18 Sample papers

2.2 WRITING THE TEST

2.2.1 SAMPLING

 Choose widely from the whole area of content for content validity and for beneficial backwash

 Should not concentrate on those elements

known to be easy to test

 Succeeding versions of the test should also sample widely and unpredictably.

2.2.2 ITEM WRITING AND

 requires good personal relations in a test writing team

 Should administer the test to native speakers of a

similar educational background to future test candidates These native speakers should score 100%, or close to it.

2.2.2 ITEM WRITING AND

MODERATION

CRITICAL QUESTIONS THAT MAY BE ASKED:

 Is the task perfectly clear?

 Is there more than one possible correct response?

 Can candidates show the desired behaviour (or arrive

at the correct response) without having the skill

supposedly being tested?

 Do candidates have enough time to perform the

task(s)?

2.2.3 WRITING AND MODERATION

SCORING KEY

 Where there is intended to be only 1 correct response a perfectly straightforward matter

 Where there are alternative acceptable responses, or

where partial credit may be given for incomplete

responses greater care  the criticism of colleagues

should be sought.

2.3 PILOTING THE TEST/ PRETESTING

 Aim: to identify possible problems before the test is

administered to the group for which it is intended

 What to do:

 Try the test items out on a few friends or colleagues, of whom

at least 2 are native speakers of the language being tested, to

see whether the instructions are clear, the language of the items

acceptable and the answer key accurate

 Administer the test to a group as similar as possible to the one for which it is intended

 Note problems in administration and scoring

 Calculate the reliability coefficients of the whole test and its

components

 Analyse individual items

3.1 DEFINITION OF WASHBACK

 According to Buck (1988), there is a natural tendency for both

teachers and students to tailor their classroom activities to the demands of the test, especially when the test is very important to the future of the students, and pass rates are used as a measure of teacher success

 Washback effect refers to the impact that testing has on teaching and learning.

 Washback can be either harmful or beneficial

3.2 HOW TO ACHIEVE BENEFICIAL

WASHBACK

 Test the ability whose development you want to

encourage

 Sample widely and unpredictably

 Use direct test

 Make test criterion-referenced

 Base achievement tests on objectives

3.2 HOW TO ACHIEVE BENEFICIAL

WASHBACK

 Ensure test is known and understood by teachers and

students

 When necessary, provide assistance to teachers

 Counting the cost

Alderson, J., Clapham, C & Wall, D (1995) Language Test Construction and Evaluation Cambridge University Press Bachman, L.F & A.S Palmer (1996) Language Testing in

Practice Oxford: OUP

Henning, G (1987) A guide to language testing: Development, evaluation and research Cambridge, Mass: Newbury House Hughes, A (1989) Testing for Language Teachers Cambridge: Cambridge University Press

To, H & Nguyen, M (2008) ELT Methodology III: Language Assessment & ELT Materials Development Hanoi.