Assessing Vocabulary

 Does vocabulary consist of single words or as larger lexical items?. Components of word knowledge I: Form: Nation, 1990: 31Spoken Form  R What does the word sound like?. Components of

Workshop Explorations

 What is a word?

 Does vocabulary consist of single words or as larger lexical items?

 What does it mean to know a lexical item?

 What should be measured in a vocabulary

test?

 How should vocabulary be measured?

 What is better – discrete-item testing or

contextualized-item testing?

 And more…

What is a word?

 Single words: chair

 Phrasal verbs: put on

 Short phrases: as a matter of fact

 Longer phrases: it has come to my attention that…

 idioms: out to lunch

Larger lexical items

Lexical phrases

 Polywords: short fixed phrases

at any rate, so to speak, as a matter of fact

 Institutionalized expressions: proverbs, formulaic

utterances

once upon a time, long time no see

 Semi-fixed expressions: basic frame + options

a day/month/year ago, yours sincerely/truly, as far as I know/can tell/ am aware

 Sentence builders: framework for complete sentence with slots for whole ideas

I think that…, not only… but also…, I regret to inform you that…

More lexical phrases

 I thought you’d never ask.

 Call me as soon as you get home.

 It’s on the tip of my tongue.

 Have you heard the news?

 Never say never.

 I’ll believe it when I see it.

 She never has a bad word to say about anyone

Characteristics of lexical

sequences

 Relatively fixed in form

 The meanings of the individual words do not provide the meaning for the phrase

 Familiar expressions for everyday

communication

 Pragmatic function

Components of word knowledge I: Form: (Nation, 1990: 31)

Spoken Form

 (R) What does the word sound like?

 (P) How is the word pronounced?

Written Form

 (R) What does the word look like?

 (P) How is the word written and spelled?

Components of word knowledge II: Position (Nation, 1990: 31)

Grammatical Patterns

 (R) In what patterns does the word occur?

 (P) In what patterns must we use the word?

Collocations

 (R) What words or types of words can be

expected before or after the word?

 (P) What words or types of words must we use with this word?

Components of word knowledge III: Function (Nation, 1990: 31)

Frequency

 (R) How common is the word?

 (P) how often should the word be used?

Components of word knowledge IV: Meaning (Nation, 1990: 31)

Concept

 (R) What does the word mean?

 (P) What word should be used to express this

meaning?

Associations

 (R) What other words does this word make us

think of?

 (P) What other words does this word make us

use instead of this one?

Four stages of knowing a word

Stage 1: I never saw it before.

Stage 2: I’ve heard of it, but I don’t know what it means.

Stage 3: I recognize it in context – it has

something to do with…

Stage 4: I know it.

(Stage 5: I can distinguish this word from others that are closely related to it.)

Dale (1965: 898)

What is vocabulary ability?

 The context of vocabulary use

generational usage

language variations

everyday usage/specialized terminology

 Vocabulary knowledge and fundamental processes

vocabulary size

knowledge of word characteristics

lexical storage

fundamental vocabulary processes

 Metacognitive stategies for vocabulary use

strategic competence (paraphrazing, avoiding, language

switching, appealing to authority, using general terms)

Objective testing of vocabulary

“Tests in which the learning material is divided into small units, each of which can be assessed by means of a test item with a single correct answer that can be

specified in advance.”

Reed, J (2000) Assessing vocabulary Cambridge University Press

 Words treated as independent linguistic units >

multiple-choice, matching test, discrete item testing etc.

 Tests often based on word lists of frequency

 Multiple-choice tests discriminate well among levels of ability and are highly reliable

 Valid indicators of overall language ability

Multiple-choice vocabulary items

Little ongoing research available, except for TOEFL

 Difficult to construct; require field-testing, analysis, refinement

 Learner may know an alternate meaning of the word, but not the one sought

 25% chance of guessing correctly

 Items may test knowledge of distractors instead of exact

meaning of target word

 Wrong answer may result from lack of understanding of syntax

 Permits only limited sampling of total vocabulary knowledge

Weshe & Paribakht (1996: 17).

BUT: convenient to administer; well-established procedures to

analyze them

things useful liquid

to change into steam exciting event

Write one suitable word in each blank

a. Modern planes fly at an a of

Multiple-choice matching

Last week while Jack and Linda were having

lunch in a café, thieves (1)broke down the front door of their house, went inside, and (2)broke into their safe Now they’re (3)broke.

_ a poor

_ b made into two or more pieces

_ c entered to steal something

_ d entered by force

Assessing quality of vocabulary knowledge

Little research available on in-depth study

of word-knowledge, except in young

children

BUT: proposals to recognize

 Partial vs precise knowledge

 Depth of knowledge

 Receptive vs productive knowledge

Write a sentence to show you know what

the word means and how it is used

Testing depth of processing

3 Write the correct ending for each word:

a Someone who interprets is an interpret .

b Something that can be interpreted is interpret _.

The word-associates test

Circle the word(s) which are associated with the underlined word

Edit

arithmetic film pole publishing

Assessing vocabulary size

Native-speaker

Anderson (1996): the average 12 th

grader knows about 80,000

words in L1; children learn up to

4,000-6,000 new words each

year

Miller (1977): children between 6-8

pick up an average of 14 new

words a day

Nagy & Herman (1987): children

acquire 3,000 words per year

Nation (1990): a vocabulary of at least 3,000 word families is necessary to read an academic text with 95% comprehension

Nurweni & Read (1999): first-year university students in Indonesia (w/ 6 years of English) had a vocabulary of about 1,200 words.

How to measure vocabulary

size?

Wide variation: students in U.S universities know 15,000 - 200,000 words!

Issues

What counts as a word?

How do we choose which words to test?

How do we find out whether the selected

words are known?

The Vocabulary Levels Test

(Nation: 1983; 1990)

Test design

 5 levels of word frequency: 2,000, 3,000, 5,000, University word level (5,000+), 10,000.

 Word-definition matching (matching

the words to the definition)

 Each level has 36 words and 18 definitions

 Designed to involve as little reading as possible and to minimize chances of guessing correctly

 Intended as a broad measure of word knowledge

 Words selected from frequency list on a random basis

 Often used as an informal diagnostic tool by teachers; little

research done

The Vocabulary Levels Test

_ make

Productive Levels Test

1 I’m glad we had this opp to talk.

2 He takes cr and sugar in his coffee.

3 The telegram was deli two hours after

it had been sent.

4 Suddenly he was thru into the dark

room.

5 The victim’s shirt was satu with blood.

The Eurocenters Vocabulary

 Initially developed as a placement test

 Points subtracted for ‘yes’ answers to non-words

 Problematic with low levels of English and French

speakers

The Eurocenters Vocabulary

The Vocabulary Knowledge

Scale

Test design

 Can be used with any set of words

 Learners are asked to rank words

The Vocabulary Knowledge

Elicitation Scale

Self-report categories

I I don’t remember having seen this word before.

II I have seen this word before, but I don’t know what it means.

III I have seen this word before, and I think it means _

(synonym or translation)

IV I know this word It means _ (synonym or translation)

V I can use this word in a sentence: _.

(Write a sentence.) (If you do this section, please also do

Secton IV.)

(Paribakht & Wesche, 1997: 180)

The Test of English as a Foreign Language (TOEFL)

1996-mid 1970s : separate vocabulary test;

discrete-point approach (synonym matching and sentence

completion)

1976 : words in context vocabulary items included in

Reading section

1995 : separate set of vocabulary items eliminated;

integrated and contextualized in the reading section

1998 : computer test; clicking on synonyms within

reading

2005 : iBT TOEFL; clicking on synonyms within reading; assessing vocabulary in writing and speaking sections

TOEFL: until the mid-1970s:

(A) according to

(B) before

(C) because of

(D) during

TOEFL: 1995

Many of the computing patterns used today in elementary

arithmetic, such as those for performing long

multiplications and divisions, were developed as late as the fifteenth century Two reasons are usually advanced

to account for this tardy development, namely, the

mental difficulties and the physical difficulties

encountered in such work.

The word ‘tardy’ in line 3 is closest in meaning to

(A) historical

(B) basic

(C) unusual

(D) late

TOEFL: 1998

The Southwest has always been a dry

country, where water is scarce, but the

Hopi and Zuni were able to bring water

from streams to their fields and gardens

through irrigation ditches Because it is so rare, yet so important, water played a major role in their religion.

Look at the word rare in the passage Click on

the word in the bold text that has the same

meaning.

iBT TOEFL

The expenditure of energy is essentially a process of

budgeting, just as finances are budgeted If all of one’s

money is spent on clothes, there may be none left to buy

food or go to the movies Similarly, a plant or animal cannot

squander all its energy on growing a big body if none would

be left over for reproduction, for this is the surest way to

VOCABULARY MULTIPLE CHOICE ITEMS: PROBLEM AREAS

Evaluate the following multiple choice items for vocabulary

 Identify problem areas

 Correct/improve problem areas

6 The of penicillin has opened

a new era for medicine

A invention

B inspection

C discovery

D combination