ELT journal oct 09

An international journal for teachers of English to speakers of other languagesin association withC ArticlesPerspectives on spoken grammar Two ways of presenting vocabulary Preparing lea

An international journal for teachers of English to speakers of other languages

in association withC

ArticlesPerspectives on spoken grammar Two ways of presenting vocabulary Preparing learners for the workplace How good is your test?

Teaching discourse intonation Developing teacher training skills Online corpora and writing skills Promoting student autonomy Readers respond

Grammar as a communicative resource

An outcomes-based approach Comment

ELT and the global recession Online forum report Class-centred teaching Key concepts in ELT Innovation in ELT

ReviewsWriting Stories: Developing Language Skills through Story Making Teaching Unplugged: Dogme in English Language Teaching The Oxford Guide to Practical Lexicography

Teaching Children English as an Additional Language Teaching English as an Additional Language Introducing English as an Additional Language Tasks in Action: Task-Based Language Education Practical Classroom English

Survey review Writing academic English Websites review Comics

NEW for February 2010

into the young learner

classroom

INTO THE CLASSROOM

www.oup.com/elt

NEW SERIES

INTO THE CLASSROOM

Make sense of new teaching tools, techniques, and educational policy.

Have confidence introducing them into your classroom.

NEW

also by

Gordon Lewis

Volume 63/4 October 2009

Every effort has been made to trace the owners

of copyright material in this issue, but we shall be

pleased to hear from any copyright holder whom we

have been unable to contact If notified, the publisher

will attempt to rectify any errors or omissions at the

earliest opportunity

© Oxford University Press 2009

All rights reserved; no part of this publication

may be produced, stored in a retrieval system, or

transmitted in any form or by any means, electronic,

mechanical, photocopying, recording, or otherwise,

without either the prior written permission of the

Publishers, or a licence permitting restricted copying

issued in the UK by the Copyright Licensing Agency

Ltd, 90 Tottenham Court Road, London W 1 PS 9 HE ,

or in the USA by the Copyright Clearing Center,

222 Rosewood Drive, Danvers, Massachusetts 01923,

USA

UK ISSN 0951-0893 (print); 1477-4526 (online)

Typeset by TnQ Books and Journals Pvt Ltd.,

Oxford Journals Advertising

60 Upper Broadmoor Road

Article titles, abstracts, and Key concepts appear free

online through the ELT Journal website:

http://eltj.oxfordjournals.org

Abstracting and Indexing

British Education Index covers ELT Journal

The Advisory Board

IATEFL Graham Hall Northumbria University Éva Illés

Eötvös Loránd University, Budapest Barley Mak

The Chinese University of Hong Kong Jonathan Marks

Leba, Poland Alice Oxholm Sheffield Hallam University Annamaria Pinter University of Warwick Barbara Skinner University of Ulster Jane Spiro Oxford Brookes University Melinda Tan

University of Central Lancashire

Key Concepts Editor

Alan Waters University of Lancaster

Text Messages Editors

Jill and Charles Hadfield

Editorial Front Office

E LT Journal is a quarterly publication for all those involved in the field

of teaching English as a second or foreign language The journal linksthe everyday concerns of practitioners with insights gained from relatedacademic disciplines such as applied linguistics, education, psychology,and sociology

E LT Journal aims to provide a medium for informed discussion of theprinciples and practice which determine the ways in which the Englishlanguage is taught and learnt around the world It also provides a forumfor the exchange of information among members of the professionworldwide

The Editor ofE LT Journal is supported by an Editorial Advisory Panelwhose members referee submissions Their decisions are based upon therelevance, clarity, and value of the articles submitted

The views expressed inE LT Journal are the contributors’ own, and notnecessarily those of the Editor, the Editorial Advisory Panel, or thePublisher

ELT Journal Volume 63 Number 4 October 2009

Articles

Christine Goh Perspectives on spoken grammar 303

Evagelia Papathanasiou An investigation of two ways of presenting vocabulary 313

David Wood PreparingESP learners for workplace placement 323

Funda Ku¨cxu¨k and

JoDee Walters

How good is your test? 332Mike Beaken Teaching discourse intonation with narrative 342

Christopher Stillwell The collaborative development of teacher training skills 353

Alex Gilmore Using online corpora to develop students’ writing skills 363

Jesu´s A´ngel Gonza´lez Promoting student autonomy through the use of the European Language

Portfolio 373

Readers respond

Anthony Bruton Grammar is not only a liberating force, it is a communicative

resource 383Phillip D Jones Giving a voice to teachers and students in an outcomes-based

approach 387

Comment

Richard S Pinner ELT and the global recession 390

Online forum report

Rose Senior Class-centred teaching 393

Key concepts in ELT

Martin Wedell Innovation in ELT 397

Survey review

Christopher Tribble Writing academic English—a survey review of current published

resources 400

Reviews

Alan Maley Writing Stories: Developing Language Skills through Story Makingby

A Wright and D A Hill 418Nick McIver Teaching Unplugged: Dogme in English Language Teachingby

L Meddings and S Thornbury 419Robert Lew The Oxford Guide to Practical Lexicographyby B T S Atkins and

M Rundell 421

Joanne Glasgow and

Barbara Skinner

Teaching Children English as an Additional Language: Meeting theChallenge in the Classroomby L Haslam, Y Wilkin, and E Kellet,Teaching English as an Additional Language: A Programme for 7–11 YearOldsby C Scott, and Introducing English as an Additional Language toYoung Childrenby K Crosse 425

Thomas A Williams Tasks in Action: Task-Based Language Education from a

Classroom-Based Perspectiveby K Van den Branden, K Van Gorp, and

M Verhelst (eds.) 429Alex Case Practical Classroom Englishby G Hughes and J Moate with

T Raatikainen 433

Websites for the language teacher

Diana Eastment Comics 436

Correspondence 439IATEFL 442

Annual index: Volume 63 (2009) 443Cumulative index 446

Please visitELT Journal’s website athttp://eltj.oxfordjournals.org

Perspectives on spoken grammar Christine Goh

English language teachers’ opinions on the pedagogic relevance of spokengrammar are beginning to be reported, yet the voices of teachers in East Asia arerarely heard In this article, the views of teachers from China and Singaporeexpressed in an online discussion are compared The discussion, which was part of

a taught postgraduate course, focused on the usefulness of British spokengrammar norms and the potential value of spoken grammar knowledge forlanguage learners There is a broad consensus of opinion about its importance forraising learners’ language awareness, but Chinese and Singaporean teachersgenerally had different attitudes to native speaker norms, while opinions on somepedagogical issues vary more at the individual level The similarities anddifferences are attributed to the teachers’ sociolinguistic concerns, understanding

of learner needs, and beliefs about grammar that are influenced by the writtenlanguage The implications of these teacher perspectives for teacher education arehighlighted

Introduction Many academic papers have been published on spoken grammar, and the

special features of speech have recently been included in several Englishgrammars (see, for example, Biber, Conrad, and Leech 2002) While there isgeneral recognition that language teaching should take account of thedifference between spoken and written language, the pedagogic relevance

of spoken grammar is still a matter of much discussion McCarthy andCarter (2001: 57) argue its importance to language teaching as follows:Language pedagogy that claims to support the teaching and learning ofspeaking skills does itself a disservice if it ignores what we know about thespoken language Whatever else may be the result of imaginativemethodologies for eliciting spoken language in the second languageclassroom, there can be little hope for a natural spoken output on the part

of language learners if the input is stubbornly rooted in models that owetheir origin and shape to the written language

But is a model of spoken grammar derived from a corpus of British spokenEnglish appropriate for all language learners? Given the status of English as

a lingua franca for intercultural communication, and a call by somelinguists to teach only its ‘core’ features, can we justify teaching a spokengrammar based mainly on spontaneous everyday speech in the BritishIsles? This issue of using real data from British native speaker (NS)language was debated by Carter (1998) and Cook (1998) and furtherexamined empirically by Timmis (2002) in his survey of about 600 teachersand learners from various countries Timmis concluded that whileteachers in general thought it was not always necessary for learners toconform to NS norms, learners themselves were in fact keen to do so

E LT Journal Volume 63/4 October 2009; doi:10.1093/elt/ccp004 303

ª ª The Author 2009 Published by Oxford University Press; all rights reserved.

In responding to specific questions on spoken grammar, two thirds of theteacher respondents thought it was important to expose learners to features

of spontaneous NS speech, but nearly a quarter of them expressedreservations about the grammaticality of certain spoken grammar forms.Timmis (2005) further showed that UK-based teachers were favourablydisposed to the idea of teaching spoken grammar forms Clearly, a grammar

of speech based on British English1data would be relevant to the UKcontext The question remains, however, as to whether teachers outsidethe UK would find spoken grammar just as useful for their learners

Views from China

and Singapore

For some preliminary answers to the question, I present here the views of

75 English language teachers from China and Singapore They werepostgraduate students doing applied linguistic courses in the sameuniversity Of the total, 37 were Chinese college and high school teachers,while the remaining 38 were mainly primary and secondary teachers fromSingapore In an online forum, the teachers discussed the questions below:

n Is linguistic information from British English data revealed in the

C A N C O D Eproject useful for teaching learners about spoken language?

n Can knowledge of spoken grammar forms improve learners’ spokenlanguage performance?

The questions were meant to be open-ended prompts that gave teachersroom to explore related issues and perspectives The discussion was notoriginally intended to be a procedure for collecting research data I had set it

up as a learning task for a teaching methodology course on listening andspeaking When reading the teachers’ responses, however, I found theircomments refreshing, interesting, and insightful It was particularlyexciting to see that the teachers were making their voices heard on

a discussion topic that had so far been dominated by linguists I thereforedecided to analyse their written comments in a systematic manner andreport my observations

Methods Before they engaged in the online discussions, I introduced the teachers to

spoken grammar in class They also read The Grammar of Talk: SpokenEnglish, Grammar and the Classroom by Carter (2003) individually It wasselected because it was relevant for bothE F Land ‘mainstream’ Englishlanguage teachers Furthermore, it contained concise explanations anduseful examples of key spoken grammar forms: heads, tails, modalexpressions, discourse markers, deixis, ellipsis, tags, flexible positioning ofadverbs and adverbials, purposefully vague language, and coordination ofclauses The teachers were also asked to visit a website on theC A N C O D Eproject (http://www.cambridge.org/elt/corpus/cancode.htm) To ensurethat they had enough time to read and respond to one another’s comments,teachers were organized into small discussion groups consisting of six toseven members They were also instructed to post their individualresponses to the two questions before responding to the views of othergroup members At the end of the course, I obtained the teachers’

permission to analyse and use their comments

The analysis was done in two stages First, each person’s initial individualresponses to the two questions were examined and allocated to one of three

304 Christine Goh

categories, with the Chinese and Singaporean participants distinguishedthroughout Second, these and the teachers’ subsequent responses to othergroup members’ views were examined for specific issues regarding theteaching and learning of spoken grammar The analyses focused on features

of saliency, frequency, and distribution, and the results were checked twice,once after a three week interval and then five weeks later Perspectivesidentified in the responses were subsequently selected

China and Singapore Before I present the results of the comparisons, it is useful to highlight some

similarities and differences about China and Singapore A commonsituation in both countries is the ever increasing demand for Englishinstruction at all levels and forms of learning China, an emerging worldsuperpower, needs English to consolidate its current economic influenceand efforts at modernization (Hu 2002), while the resource-scarce city-state

of Singapore needs citizens with a good command of the language toreposition itself as a centre for knowledge, learning, and service industries,and to participate in overseas trade (Alsagoff 2007) Whereas ChineseE F Lstudents receive little authentic input from their immediate educational andsocial environments, students in Singapore have English as a medium ofinstruction and also study it as a curriculum subject from preschool years.About half of Singaporean students come from English-speaking homes.Many, however, speak a colloquial variety known popularly as ‘Singlish’, thesyntax, phonology, and lexical items of which are heavily influenced byvernacular languages in the community

British spoken

grammar norms

The first question in the discussion invites teacher opinion on theusefulness of British spoken grammar norms (see Figure 1) Chineseteachers are on the whole happy to embrace the exonormative featuresthat British spoken grammar provides and welcome more linguisticinformation from other NS varieties They consider the ability to speaknaturally and accurately like a NS from countries such as Britain to be

a distinct advantage Among Singaporean teachers, however, opinion isdivided, revealing the complexities in language choice in societies wherethe local English variety competes with Anglo models for allegiance andacceptance

figure 1

British spoken grammar

norms

is considered especially helpful toE F Llearners who have few opportunities

to develop features of authentic English speech

It reveals an authentic picture of language use to students, a world full

of incomplete sentences, phrases, vague language, discourse markers,etc (C1)

In contrast, more than 60 per cent of their Singaporean colleagues findspoken grammar based on British English to be less directly useful Themain reason is that Singaporean speakers already have their own naturalspoken grammar forms for casual speech through Singlish Nevertheless,

a few teachers do see the British model as a useful point of reference for thelearning of standard English

It is fine to use British English as the basis for teaching spoken grammar

as we need a set of standard rules to follow (S1)

‘Useful, but ’ More than half of the Singaporean teachers say that spoken grammar forms

should be introduced to students only for comparison purposes:

It would be useful to use the corpora for spoken Singapore Englishalongside the findings of theC A N C O D Ecorpus (S2)

Perhaps students are speaking Singlish partly because they observed

‘everybody around me’s speaking like this, what’s wrong of me in doingso?’ The corpus examples would expose them to the reality that ‘it’s a realbig world out there!’ (S3)

Some Chinese teachers feel that NS models should not be limited to justBritish English:

We could add more variety, for example, by using American, Australian,Canadian English, etc to the teaching of spoken grammar (C2)

‘Not useful’ Around five per cent of the Chinese teachers argue that learners should

focus on acquiring written grammar, which can then be modified forspeaking

Given the limited classroom exposure, priority should be given to theinstruction of written grammar instead of spoken grammar [ ] If

a student has a good knowledge of written language, his/her spokenlanguage can be improved more easily provided s/he is exposed in thereal-world conversation (C3)

A Singaporean teacher’s comment reflects sociolinguistic sensibilities,particularly the issues of ownership and identity, implying that her studentsmay resist exonormative models:

It is rather artificial and an uphill task to force down the throats of ourpupils to speak English using the linguistic information based on BritishEnglish [ ] English does not belong to the people in Britain (S4)

306 Christine Goh

Increasing learner confidenceThe teachers feel that many ChineseEFLlearners have the misconceptionthat they should construct ‘perfect’ sentences modelled after the writtenlanguage These learners become anxious and their performance is affected

It is very necessary [ ] to reduce their anxiety in speaking in English Forexample, they may feel relaxed to know there are lot of phrasal utterancesand ellipsis in British daily conversations (C5)

For some Singaporean teachers, confidence for their students comes from

an understanding of different speech registers through spoken grammar:Having knowledge of spoken grammar not only enables pupils todifferentiate between written and spoken language but also the differenttypes of speech used in different contexts With this knowledge, pupilscan select and use the appropriate forms in constructing their speech andspeak with confidence (S4)

Producing natural spoken outputSome Chinese teachers remark that the speech of manyE F Llearners(including themselves) often sounds stilted because it has been modelled on

figure 2

Can spoken grammar

improve spoken

language performance?

formal written grammar and believe that knowledge about the grammar ofspeech will help them sound more natural when they speak.

Speaking was taught to me by using written forms as models [ ] I spokelike a TV announcer, always complete sentences with ‘perfect’ grammar

I began to feel uncomfortable when some of the girls imitated myspeaking (C6)

L2 learners are bound to sound bookish without using these features ofspoken grammar This is one reason why many of us were shocked anddisappointed when we used English for the first time in real interaction.(C7)

In contrast, some Singaporean teachers think that British spoken grammarforms such as tails and heads will make their students sound ‘foreign’ andunnatural They nevertheless welcome the idea of discourse markers fororganizing speech

It improves

performance, but

Some teachers are more cautious about teaching spoken grammar to alllearners as they believe that it will only benefit the advanced learners.But can students really imitate some features of spoken grammar in theirown speaking? I guess no way until they are so good at speaking and theycan monitor their speaking, produce continuous utterances, and respond

to others by adjusting their own at the same time It requires reallyadvanced speakers to do that (C8)

Some also believe that it is better to introduce spoken grammar to learnersafter they have acquired formal written grammar

If exposure of these sorts can help us to be more effective speakers andlisteners, then there surely cannot be any harm in exposing our students

to spoken grammar forms especially when they already are well versed intheir written forms (S5)

The Chinese teachers are also concerned that students who adopt spokengrammar features during high-stake oral examinations may be

disadvantaged They agree that many examiners are not familiar with thistype of natural spoken output in English-speaking countries and maytherefore expect candidates to produce utterances that are constructedaccording to written English structures:

I have given some remedial courses for students who are taking the oralexams ofC E T-4 and C E T-6 In preparation for such a course, I havetalked to my colleagues who participate in marking the official exam [ ]students are required to talk as fluently as possible in a formal way andexaminers do not value some of the spoken grammar features mentioned

in this article (C9)Some teachers in Singapore raise similar issues:

We marked students down for not answering in complete sentences (S6)Students are expected to speak Standard English which mimics a scriptedmonologue or dialogue where utterances are well-formed with specificstructures (S7)

308 Christine Goh

It does not improve

The main worry amongst some Singapore teachers is the effect it mighthave on the quality of their students’ writing, as the comments belowdemonstrate:

I am against explicit teaching of such characteristics as it would confusestudents’ knowledge of written grammar [ ] As it is, in Singaporeneighbourhood2schools, we are struggling to help students eradicatetraces of spoken grammar3in their written essays (for example,contractions, abbreviations, flexible positioning of clauses, and evenellipses) Teaching explicit features of spoken grammar would probably

do more harm than good, at least with respect to written work (S8)These teachers consider most British spoken grammar forms to beinappropriate for formal communication in Singapore They would prefertheir students to produce utterances based on the neat grammar of writingwithout the ‘messiness’ of spontaneous speech

NS norms and

spoken grammar

One issue that the teachers focused on was the relevance of NS norms ofspoken grammar Timmis (2002) had earlier reported that many teachersbelieved that it was unnecessary for learners to conform to NS norms It hasalso been argued by others that using such a model could frustrate learnersbecause it sets them up for a standard they may not attain Many of theteachers involved in the discussion, however, seem to think otherwise TheChinese teachers are firm in their view on using NS models for theirteaching They consider the models important for their learners as ChineseEFLlearners do not get sufficient spoken English input that can help themproduce speech that is natural and authentic These teachers have taken

a sociolinguistics course and are aware of issues about language variation,standardization, and challenges in applying NS norms Nevertheless, whenthey reflect on their own learning experiences and their students’ needs(‘People do not speak English except in special contexts, such as meetingforeigners, so there is no authentic Chinese English’), they assert that NSmodels are not only crucial (‘If they don’t have a model to follow, they will be

in confusion’) but also empowering (‘When your language is not standard,your voice cannot be heard’) and motivating (‘When we find our orallanguage echoes much of native speakers’ language, our confidence rises’)

A likely reason for this attitude is that the Chinese teachers are themselvessuccessfulE F Llearners who have benefited from NS models in an input-poor environment This is a situation that may not be appreciated fully bysome NS teachers and teachers from Singapore Interestingly, the teachers’attitude is similar to those of the learners in Timmis’ (2002) study Their

opinions also lend support to Kuo’s (2006) argument that NS models areappropriate for second language pedagogy as well as appealing to languagelearners Furthermore, the teachers’ endorsement of the pedagogicrelevance of spoken grammar lends credence to Kuo’s conclusion that

a comprehensive NS model should ‘serve as a complete and convenientstarting point’ (ibid.: 220) Zhang (2004) also reported the overwhelmingpreference amongst some ChineseE F Lteacher trainees for an NS variety ofspoken English In Korea, Taiwan, and Japan, elementary schoolE F Lteachers have expressed the hope that a proportion of their students willeventually achieve NS proficiency (Butler 2004) In light of what non-nativespeaker teachers want for their learners, there is clearly a need to re-examinecurrent assertions that learners need to be exposed to only selected linguisticfeatures of English

In Singapore, the role of traditional NS models (for example, British spokenEnglish) is less prominent though not altogether unimportant Theteachers’ discussion illustrates the tension that exists in Singaporeans’language choice attributed to opposing orientations towards social-culturalidentity and economic capital (Alsagoff 2007) Some teachers argue thatsince English in Singapore has its own linguistic features and conventions

of use, they must be careful not to impose an exonormative model thatstudents may reject The presence of Singlish which has its owngrammatical features means that local students do not need British normswhen interacting in a casual style British spoken grammar features arenevertheless welcomed as a means of raising language awareness (‘Theinformation can be used to complement existing information on grammar .they will also attain a more balanced perspective’.) See also Teacher S3#s

comment Others see no harm in referring to British English norms in theSingapore classroom (‘We do need some guiding models to follow and wehave been using British English forms as a guide to teaching English inSingapore’.) Interestingly, when discussing foreignES Lstudents studying

in Singapore (for example, Chinese, Vietnamese, and Indonesian students),the teachers recommend that spoken grammar be taught to help thesestudents speak more naturally

The dilemma that Singaporean teachers face is not lost on their Chinesecolleagues during the discussion, with one of them calling on the former togive due emphasis to traditional NS models in the classroom (‘Otherwise,when they grow up with a good command of their English dialect (Singlish),they will go far away from what is called ‘‘good English’’’.) This concern isreflected in Alsagoff’s observation that for Singaporeans to be globallyrelevant and economically competitive, they need English that is of

‘international currency’ which she calls ‘International Singapore English’,

a variety that is exonormatively defined, benchmarked against features ofanglo-English, and creates an image of high education attainment,formality, and authority for its speakers (Alsagoff 2007: 35) TheSingaporean teachers’ views further illustrate issues about languagecompetence and language standards inE LTcontexts where a newindigenous English variety exists (Davies 2003)

On the whole, the ChineseE F Lteachers welcome the idea of introducingspoken grammar to their students Where they and their Singaporean

310 Christine Goh

counterparts are ambivalent, it is in respect of its apparently ungrammaticalnature Spoken grammar forms such as heads, tails, and ellipsis, forexample, are seen as deviations from established notions of ‘proper’grammar where sentences follow typical structures, such as subject-verb-object and subject-verb-complement Some teachers are also constrained by

a narrow interpretation of the term ‘grammar’ They find it confusing thatlexical items in the form of modal expressions (for example, ‘hopefully’,

‘perhaps’) are a part of the grammar of English Coordination of multipleclauses with ‘and’ as well as the use of vague language (for example, ‘sort of’,

‘kind of like’) are also perceived to be speech habits that learners shouldavoid, particularly in formal interactions

In addition to conceptual issues, practical concerns have been raised.Assessment of oral skills in national examinations is said to value a form ofspoken output that is based on the written model Flexible positioning ofadverbials, heads, tails, and certain modal expressions are, as one teacherputs it, ‘big mistakes in China’ Another view is that given limitedcurriculum time for English in colleges and schools, it may be better forteachers to focus on formal written grammar, which could then serve as

a basis for producing spoken English Some Singaporean teachers are alsoconcerned that teaching spoken grammar could exacerbate some students’poor written grammar In spite of these reservations, nearly all the teachersagree that spoken grammar has a part to play in their students’ languageawareness development

Conclusion The teachers show a broad consensus of opinion that spoken grammar

knowledge is useful for raising awareness about spoken and writtenlanguage The Chinese teachers also believe that by learning to use thegrammar of British speech, their learners will speak English morenaturally and confidently Singapore teachers, on the other hand, areunderstandably more cautious when considering British spokengrammar norms Although the teachers come from different sociolinguisticlandscapes, they share a common goal for their learners’ speakingdevelopment—to speak English that is recognized as ‘good’ or ‘standard’

by speakers outside their countries In China, and perhaps more so inSingapore, mastery of an internationally accepted standard variety ofEnglish is seen as a way of increasing the country’s economic andhuman capital It is clear that the teachers want a comprehensive model

of standard English for teaching spoken English, albeit for differentpurposes, and British spoken grammar has a place in this scheme

of things

The debate about the pedagogic relevance of spoken grammar by teachers isset to continue as they become more informed about new linguisticdevelopments Teacher educators can facilitate this by helping teachers andteachers-to-be understand the grammar of speech in the same way they dowith written grammar This can be facilitated through the use of teacherlanguage awareness activities in methodology classes It is also importantthat teachers’ knowledge about spoken grammar is not limited only torecognition of its forms or categories In addition to developing soundconceptual knowledge about spoken grammar, teachers should also exploreways in which the knowledge can be applied to teaching learners how to use

the spoken language more effectively to express their communicative needsand to understand what is said by people they interact with Timmis’ (2005)framework for teaching about spoken grammar offers insights on howsome of this can be done, but ideas that work forE LTpractitioners in the UKneed to be evaluated for their relevance for teachers elsewhere Alternativetechniques should be explored so as to address sociolinguistic concerns thatare unique to each country As this article has shown, teachers understandthat their learners’ spoken English needs are influenced by the linguisticand social environments in which English is learnt Any attempts atdeveloping teachers’ insights into the pedagogic relevance of spokengrammar must therefore allow teachers to examine their understanding

of it in the context of their own teaching

Final revised version received September 2008Notes

1 ‘British English’ is used in its broadest sense to

refer to the source of theC A N C O D Edatabase

from which the features of spoken grammar were

derived (see Carter and McCarthy 1997)

2 These are schools where most of the students

come from non-English-speaking homes or

homes where mainly Singlish is spoken

3 Spoken grammar of Singlish (for example, the

omission of the copular or auxiliary verb: ‘She not

nice’; ‘Why you like that?’)

References

Alsagoff, L.2007 ‘Singlish: negotiating culture,

capital and identity’ in V Vaish, S Gopinathan, and

Y Liu (eds.) Language, Capital, Culture Rotterdam,

The Netherlands: Sense Publishers

Biber, D., S Conrad,and G Leech 2002 Longman

Student Grammar of Spoken and Written English

Harlow, UK: Pearson Education Limited

Butler, Y G.2004 ‘What level of English proficiency

do elementary school teachers need to attain teach

EF L? Case studies from Korea, Taiwan, and Japan’

T ES O LQuarterly 38/2: 245–78

Carter, R.1998 ‘Orders of reality:C AN C O D E,

communication, and culture’.E LTJournal 52/1:

43–64

Carter, R.2003 ‘The grammar of talk: spoken

English, grammar and the classroom’ in New

Perspectives on Spoken English in the Classroom

London: Qualifications and Curriculum Authority

Carter, R.and M McCarthy 1997 Exploring Spoken

English Cambridge: Cambridge University Press

Cook, G.1998 ‘The uses of reality: a reply to Ronald

Carter’.ELTJournal 52/1: 57–63

Davies, A.2003 The Native Speaker: Myth and Reality.Clevedon, UK: Multilingual Matters

Hu, G W.2002 ‘Recent important developments insecondary English language teaching in the People’sRepublic of China’ Language, Culture and

Curriculum 15/1: 30–49

Kuo, I C V.2006 ‘Addressing the issue of teachingEnglish as a lingua franca’.E LTJournal 60/3: 213–21.McCarthy, M R.and R Carter 2001 ‘Ten criteria for

a spoken grammar’ in E Hinkel and S Fotos (eds.).New Perspectives on Grammar Teaching in SecondLanguage Classrooms Mahwah, NJ: LawrenceErlbaum Associates

Timmis, I.2002 ‘Native-speaker norms andinternational English: a classroom view’.E LTJournal56/3: 240–9

Timmis, I.2005 ‘Towards a framework for teachingspoken grammar’.E LTJournal 59/2: 117–25.Zhang, L J.2004 ‘Awareness-raising in theT E F L

phonology classroom: student voices andsociocultural and psychological considerations’.I T L

Review of Applied Linguistics 145/1: 219–68

The authorChristine Gohis an Associate Professor of AppliedLinguistics at the National Institute of Education ofthe Nanyang Technological University, Singapore.She is interested in listening, speaking andpronunciation development, metacognition andsecond language learning, and the influence ofteacher cognition onELT She teaches

undergraduate and postgraduate courses on first andsecond language acquisition,E LTmethodology forlistening, speaking, and pronunciation andsupervises research in her areas of interest.Email: christine.goh@nie.edu.sg

312 Christine Goh

An investigation of two ways of presenting vocabulary

Evagelia Papathanasiou

The use of semantic links or networks in L2 vocabulary acquisition has been

a popular subject for numerous studies On one hand, there is a strong theoreticalbackground stating that presenting words in related fashion facilitates thelearning of L2 vocabulary On the other hand, research evidence indicates thatsemantically related vocabulary seems to hinder rather than ease the learning ofL2 vocabulary The aim of the present study is to examine which manner of L2vocabulary presentation is more helpful for L2 learners It was conducted inE F Lclassrooms with GreekE F Lstudents The subjects were 31 intermediateE F Lchildren and 32 beginnerE F Ladults The two different ways of organizing newvocabulary for presentation were tested The article will focus on the mainconclusion that semantically related clustering impedes L2 vocabulary learning

at beginners’ level

Introduction In recent years, contradictory advice to teachers has been emerging from

studies into the use of semantic links or networks in classroom materialsand activities for L2 vocabulary learning There is some experimentalevidence which suggests that learning semantically related words (forexample, body parts) at the same time makes learning more difficult(Tinkham 1993, 1997; Waring 1997; Finkbeiner and Nicol 2003) There

is also a theoretical framework that strongly supports the idea that it isvery useful to present words of related meaning together so that learners cansee the distinctions between them and gain a complete coverage of thedefined area of meaning The experimental evidence mainly derivesfrom research using artificial language and not a natural L2 The purpose

of our research was to investigate which of the two ways of vocabularypresentation would prove to be a useful tool in L2 vocabulary learning.The present study was influenced by principles and theories of actionresearch

E LT Journal Volume 63/4 October 2009; doi:10.1093/elt/ccp014 313

ª ª The Author 2009 Published by Oxford University Press; all rights reserved.

There are three main arguments for the presentation of vocabulary insemantically related sets:

1 The first argument is that the presentation of semantically relatedvocabulary makes the meaning of these words clearer by seeing how theyrelate to and are different from other words in the set This is importantbecause, according to Channell (1981), the mind uses semantic similarity

in classifying words

2 The second argument is that there is evidence for the usability andeffectiveness of presenting related vocabulary in classroom activities.Jullian (2000) refers to a classroom activity which incorporates an explicitapproach towards the presentation of semantically related vocabulary.The writer points out that this type of classroom activity helps students tounderstand the full semantic content of the related words and detect whatmakes them similar and different from each other

3 The emphasis on the ability to distinguish differences between wordswith related meaning is also present inE F Lcoursebooks Coursebookwriters are driven to present semantically related vocabulary items mostlybecause of their own perceptions of the communicative needs of theirstudents As a consequence, these coursebooks are divided into variousunits responding to any situation in which students might find itnecessary to communicate in the L2 (for example, visiting a doctor).However, there is some experimental evidence against the presentation ofsemantically related vocabulary in sets Tinkham (1993, 1997) and Waring(1997) investigated interference effects for word learning using ‘artificial’words The subjects listened to lists of English words paired with imaginarywords Their task was to try to learn the word pairs in as few trials as possible.The data collected by these researchers suggest that the presentation of newvocabulary items to L2 learners in clusters of semantically and syntacticallysimilar words impedes rather than facilitates learning This means that ittakes students more time to learn new lexical items when these lexical itemsare presented in related sets rather than in unrelated sets

Nevertheless, there are some limitations (Waring: ibid.) on thegeneralizability of the results found in the above studies There are fourmain points we have to consider:

a very few words were tested;

b the artificial words that were used were strictly controlled;

c there was no clear definition of what semantic relatedness might mean;and

d it was not made clear whether the same effects would hold for learnerswho already had part of the semantic set being tested

It is also important to mention here research by Schneider, Healy, andBourne (1998) that used natural L2 words rather than artificial ones andwhich gave very different results Their findings initially appeared tosuggest that learning related words together (for example, parts of the body)was easier than learning unrelated words However, when a test of long-term (LT) retention was administered, the researchers found that theparticipants in the mixed-order acquisition condition (presented with

314 Evagelia Papathanasiou

unrelated vocabulary) were faster and made fewer errors than those in thegrouped-order acquisition condition (presented with related vocabulary).

In a more recent study, Finkbeiner and Nicol (2003) used 32 new words andeach was paired with a picture of a familiar concept The results revealed thatparticipants translated L2 labels learnt in semantic sets significantly moreslowly than they did L2 labels learnt in random order

The arguments for and against presenting new vocabulary in lexical setsreported above suggest a need for further research We have two contrastingviews on the presentation of vocabulary in an L2 (experimental evidenceversus theoretical framework) However, we do not have enoughconvincing evidence to decide which of the two contrasting approaches

to learning vocabulary is the more useful and appropriate for L2vocabulary teaching The best way for us to make a decision is to apply bothapproaches inE F Lclassrooms and compare the results

Methodology

Overview of research

We conducted a study of alternative ways of teaching vocabulary to GreekEFLstudents This was a classroom study which employed two differentways of organizing new vocabulary for presentation:

1 presenting semantically related words (words that share certain semanticand syntactic similarities, for example, topic-related vocabulary, such as

‘knife’, ‘fork’, ‘spoon’, synonyms, antonyms, or homonyms) together atthe same time, and

2 presenting vocabulary in an unrelated (mixed) fashion (words that are notsemantically related, for example ‘book’, ‘hospital’, ‘freedom’)

The aim of the study was to evaluate the relative claims of the two differentprocedures by using two different groups of students (Class A and Class B)

At the end of the research period, all students were tested to determinewhich of the two competing methods was the more effective

Research question The research question was as follows: Which of the two ways of

presenting and organizing the teaching of new L2 vocabulary (specificallyrelated word sets or unrelated word sets) produces better retention ofthose words when scored in short-term (SHT) andLTvocabulary translationtests?

Motivation for using

GreekE F Lstudents

in the research

Our motivation for using GreekE F Lstudents as subjects in our studyderives from a paper by Scholfield and Gitsaki (1996) The main point isthat most Greek students seem to have relatively poor vocabulary knowledgeand tend to use a small number of words in their writing We wanted toexamine if learning semantically related (topic-related vocabulary,synonyms, antonyms, and homonyms) or unrelated words would helpstudents learn and recall more vocabulary items

Timing and stages of

the study

The study was influenced by principles and theories of action research.The subjects were 31 intermediateEFLchildren and 32 adult beginners.Two different ways of organizing new vocabulary for presentation wereemployed:

1 presenting semantically related words (topic-related vocabulary, forexample, ‘mugging’, ‘terrorism’, ‘forgery’, synonyms, antonyms, orhomonyms) together at the same time, and

2 presenting vocabulary in an unrelated fashion (for example, ‘carpenter’,

‘tornado’, ‘sage’)

SHTandLTtests were administered to the students Since we were usingvocabulary translation tests, the knowledge of the meanings of the words intheir L1 was taken into consideration For this reason, we provided allsubjects (both children and adults) with a list of the words (used in the study)translated into Greek in order for them to tick the words they did not know.This procedure ensured that the study was not affected by unknown L1vocabulary

Procedure Two intermediate classes of juniors and two groups of adult beginners

participated in this study One class at each level was labelled A and the other

B The subjects in Class A were taught the association between 60 Englishwords (see Appendix 1) and their Greek equivalents with words that weresemantically related (topic-related vocabulary, homonyms, synonyms, andantonyms) for a period of three weeks There were two lessons per week.Each vocabulary lesson lasted for 45 minutes and took place at the end of thenormal class that students attended every Monday and Friday At the sametime, the subjects in Class B were taught the association between 60English words and their Greek equivalents with words that were not relatedsemantically (see Appendix 2) The words were presented in a mixed(unrelated) order At the end of the third week, an immediate (SHT)vocabulary test was administered to both classes Two weeks later, thesubjects in both classes were tested on aLTvocabulary test

For the next three weeks, Class A was taught the association betweenEnglish words and their Greek equivalents with the words grouped in

a mixed (unrelated) order The vocabulary items were the same used forClass B In the meantime, Class B was taught the association betweenEnglish words and their Greek equivalents with the words grouped in

a related fashion (semantically related words) The words were the samewords used for Class A At the end of the third week, an immediate (SHT)vocabulary test was administered to both classes Two weeks later, thesubjects in both classes were tested in anLTvocabulary test

It is worth mentioning that during the presentation of new vocabulary,attention was paid to one particular aspect of knowing a word: the form–meaning connection According to Nation (2001), strengthening the form–meaning connection involves having to recall a meaning when seeing orhearing a particular word or having to recall a spoken or written form whenwanting to express a meaning The subjects must be able to recognize

a word and link it to its meaning by using L1 translations

Teaching The teaching procedure was the same for both children and adult groups

Both groups were exposed to the same teaching material Each lesson lastedfor 45 minutes The teacher (myself as a practitioner) first introduced thestudents to the new vocabulary and then elaborated, expanded, andconsolidated these words into classroom exercises (see Steps 1, 2, and 3 of

316 Evagelia Papathanasiou

the teaching procedure below) We followed the same teaching procedurefor both related and unrelated words.

Step 1Duration: 10 minutes (noticing) At first, the students saw a list of tenEnglish words written on the board The teacher then read aloud the wordsone by one and provided their Greek translations The students wrote theEnglish word on one side of a card and the meaning (using L1 translation)

on the other to encourage recall Small cards (around 5 · 4 cm) were used so

that they could be easily carried around The students were encouraged tolearn words, receptively, for example, to see the L2 word and recall themeaning using L1 translation

Step 2Duration: 15 minutes (retrieval) Each of the students went through the set ofcards looking at each foreign word and trying to retrieve its meaning If thestudent did not remember the Greek equivalent, he or she would turn thecard over The students repeated this process for each of the new words Theteacher ensured that the word cards were used repeatedly by practising theword card strategy with the whole group The purpose of the repetitions wassimply to facilitate learning Tinkham (1993) found that most learnersrequired five to seven repetitions for the learning of a group of six pairedassociates Thus, the teacher went through the set of cards with the students

at least five to six times The students, then, were asked to give (orally) theGreek translation for each new English word, for example, answeringquestions like ‘What is the Greek for ‘‘priest’’?’ The students had to say theGreek equivalent The questions helped them instantiate and apply thewords

Step 3Duration: 20 minutes (generation) During the third phase of the teachingprocess, the students were asked to do two different exercises to encouragerepetition of the new vocabulary in each lesson The same format ofexercises was used for both Class A and Class B

Characteristics of the

test

The way we presented the target words in a test was related to the purpose ofthe assessment The selected target words were presented in isolationbecause we wanted to assess the students’ ability to supply the meaningwhen given the target word Meaning and word (written) form were themain types of word knowledge to be tested We used a ‘definition recall test’,which meant that the students were given a list of English words and asked

to write the Greek equivalent (L1 translation) There were three maincharacteristics of the test:

1 It was a pen and paper test taking 15–20 minutes to complete

2 It tested receptive knowledge (passive recognition test)

3 It tested vocabulary only

Results and

discussion

In this section, we describe and present the results from the t-test analysis ofthe performance of everyone (all students together) and children and adults(separately) on related and unrelated vocabulary, both onSH TandLTtests.Figure 1 presents the results of the most important pairs for discussion

It is clear from the table above that

n adult beginners performed significantly better on the unrelatedvocabulary test than on the related vocabulary test

n children (intermediate level) showed no significant difference in testscores between related and unrelated vocabulary

This suggests tentatively that the presentation of unrelated vocabulary mayassist learning of new L2 words more than related vocabulary only atbeginners’ level (adults) The result above is compatible with the results ofprevious research (Tinkham 1997; Waring 1997; Schneider, Healy, andBourne 1998; Finkbeiner and Nicol 2003) illustrating that presenting L2students (beginners) with their new vocabulary grouped together in sets ofsyntactically and semantically similar words impedes rather than facilitatesthe learning of those words It is crucial to mention that these resultsreinforce the positions stated by the researchers mentioned above since theywere extracted from natural language in anE F Lclassroom through

a teaching procedure

Extensive research into ‘interference theory’ (Baddeley 1990) suggests that

as similarity increases between targeted information and other informationlearnt either before or after the targeted information, the difficulty oflearning and remembering the targeted information also increases(Tinkham 1997) Similarly the ‘distinctiveness hypothesis’ (see Hunt andMitchell 1982), which relates ease of learning to the distinctiveness (non-similarity) of the information to be learnt, also validates the above argument

It is important to point out that these results apply to beginner-levelEF Ladults and not to intermediateEFLchildren where there is no significantdifference between related versus unrelated vocabulary test scores Based onthe high mean scores for the adults, especially in unrelated vocabulary, wemade the following assumptions:

One probable reason for the adults achieving higher scores was motivation

It seemed that adults were highly motivated and more conscientious

tailed)

(2-[1] Everyone-re-SHT vs 1 63 60 21.1746 9.48435 1.19492 -2.501 124 0.014 Everyone-un-SHT 2 63 60 25.6984 10.78176 1.35837

[2] Everyone-re-LT vs 1 63 60 16.8095 7.97346 1.00456 -3.114 124 0.002 Everyone-un-LT 2 63 60 21.7460 9.73515 1.22651

[3] All-Children-re-SHT vs 1 31 60 18.8065 9.96467 1.78971 -0.620 60 0.538 All-Children-un-SHT 2 31 60 20.4194 10.51277 1.88815

[6] All-Children-re-LT vs 1 31 60 14.0323 7.24101 1.30052 -1.106 60 0.273 All-Children-un-LT 2 31 60 16.3871 9.39034 1.68655

[9] All-Adults-re-SHT vs 1 32 60 23.4688 8.53072 1.50803 -3.469 62 0.001 All-Adults-un-SHT 2 32 60 30.8125 8.40675 1.48612

[12] All-Adults-re-LT vs 1 32 60 19.5000 7.82469 1.38322 -4.032 62 0.000 All-Adults-un-LT 2 32 60 26.9375 6.90459 1.22057

re=related, un=unrelated, SHT=short-term test, LT=long-term test, Group 1=related group, Group 2=unrelated group, Mean=mean of test scores, Sig.=significance, t=obtained value for t, df =degree of freedom

318 Evagelia Papathanasiou

learners for personal and professional reasons Motivation has to do with theemotional dimension of L2 learning The main reason they joined theEnglish seminars was to acquire a certificate in English in order to use itprofessionally and for personal interest Children on the other handprovided quite low scores both in related and unrelated vocabulary, possiblydue to lack of motivation.

Another possible reason for the adults’ higher scores was that adults, ingeneral, can master certain aspects of a foreign language even well intoadulthood Adult L2 learners routinely achieve high levels of proficiency inthese aspects of a foreign language Lexical and syntactical competencebecomes easier for them in contrast to phonology, which becomes verydifficult to acquire

Conclusion This study differs from similar ones in having been carried out in a natural

setting The use of a natural L2 combined with the teaching procedure in

a real classroom environment makes this research generate results thatmight apply to natural L2 learners On the contrary, previous research(Tinkham 1997; Waring 1997) was tightly controlled to benefit theresearcher, not the learner, as Waring (ibid: 271) points out

This research complements previous studies and suggests the need forsome re-evaluation of current pedagogical practice The results of this studymay come as a surprise to many current writers ofE S Lcoursebooks whorely heavily upon the employment of semantic clusters in their presentation

of new vocabulary They may want to explore the possibility of simplifyingL2 vocabulary learning by incorporating a semantically unrelated form ofpresentation (at least at beginner level)

An intermediate (or more advanced) learner would probably alreadyknow many words from the semantic groups, and when presentedwith new words may only need to add new words to an existing store,rather than create a new one from scratch It may therefore be thatactivities grouping words with related meaning are best used at

a secondary stage when the words can be recognized, somemeanings have been acquired, and learners have reached a pointwhere they will benefit from further opportunity to makeconnections and distinctions (Hedge 2000: 122–3) For initialpresentation, we can present unrelated vocabulary and later at a moreadvanced level present semantically related vocabulary We shouldpresent related vocabulary in a way that does not create an environmentfor interference effects

Even though the present study was conducted in Greece, it offers usefulfindings for L2 learners and practitioners worldwide.E F Lteachers indifferent countries could use the results of this research in order to assisttheir students with L2 vocabulary learning The present study sets a positiveframework for internationalEFLpractitioners

Future research The present study can also be considered as a useful starting point for

similar research or replication in order to see if the findings apply in othercontexts as well Further research with intermediate and more advancedstudents seems to be necessary in order to clarify whether related vocabulary

plays a prominent role in L2 learning at this level It probably made nodifference with our intermediate subjects because they were not motivatedenough.

Another aspect that might require further research is the teaching andtesting procedure We need to consider whether we would obtain thesame results if both teaching and testing methods were different

Keeping in mind that the present study used only nouns to examine theinfluence of vocabulary presentation, it would be interesting to observethe results if we taught and tested verbs, adjectives, and other parts ofspeech

In addition, the productive dimension of L2 vocabulary learning in relation

to the manner of vocabulary presentation could also be examined Thepresent testing focused on the receptive use (seeing the L2 word and having

to provide the L1 translation) of the words Another study could test if theeffect also occurred productively (subjects to be given the L1 word and toproduce the L2 word)

It is hoped that the present article has offered some evidence of how themanner of organizing words for presentation may be important for learningnew L2 vocabulary

Final revised version received December 2008

References

Baddeley, A.1990 Human Memory London:

Lawrence Erlbaum Associates

Channell, J.1981 ‘Applying semantic theory to

vocabulary teaching’.ELTJournal 35/2: 115–22

Finkbeiner, M.and J Nicol 2003 ‘Semantic category

effects in second language word learning’ Applied

Psycholinguistics 24/3: 369–83

Hedge, T.2000 Teaching and Learning in the

Language Classroom Oxford: Oxford

University Press

Hunt, R R.and D B Mitchell 1982 ‘Independent

effects of semantic and nonsemantic

distinctiveness’ Journal of Experimental Psychology:

Learning, Memory and Cognition 8/1: 81–7

Jullian, P.2000 ‘Creating word-meaning

awareness’.E LTJournal 54/1: 37–46

Nation, I S P.2001 Learning Vocabulary in Another

Language Cambridge: Cambridge University Press

Schneider, V I., A F Healy,and L E Bourne 1998

‘Contextual interference effects in foreign language

vocabulary acquisition and retention’ in A F Healy

and L E Bourne (eds.) Foreign Language Learning:

Psycholinguistic Studies on Training and Retention

Mahwah, NJ: Lawrence Erlbaum

Scholfield, P J.and C Gitsaki 1996 ‘What is the

advantage of private instruction? The example of

English vocabulary learning in Greece’ System 24/1:117–27

Tinkham, T.1993 ‘The effect of semantic clustering

on the learning of second language vocabulary’.System 21/3: 371–80

Tinkham, T.1997 ‘The effects of semantic andthematic clustering on the learning of secondlanguage vocabulary’ Second Language Research 13/2:138–63

Waring, R.1997 ‘The negative effects of learningwords in semantic sets: a replication’ System 25/2:261–74

The authorEvagelia Papathanasiouhas a PhD in L2 vocabularyacquisition and she is an instructor inEA Pin theEnglish Language Support Unit at City College,Thessaloniki, Greece, affiliated institution of TheUniversity of Sheffield, England, UK (the results ofthis article are part of her research at this university).Her current main interests are L2 vocabularyacquisition, the effects of the Common EuropeanFramework in the language learning community,and students’ attributional beliefs in languagelearning

Email: epapathanasiou@city.academic.gr

320 Evagelia Papathanasiou

Appendix 1

Semantically related

nouns

Appendix 2

Semantically

unrelated nouns

Excess Dough Landing

322 Evagelia Papathanasiou

Preparing E S P Learners for workplace placement

David Wood

Engineering students in North American universities often participate incooperative education placements in workplaces as part of the requirements fortheir degrees and professional certification Students for whom English is an L2often experience difficulties in these placements due to the fact that while theiracademic language ability may be sufficient to manage their coursework, theystruggle to cope with the communication demands of a workplace context Thispaper is a report of a course designed to assist these types of students inaugmenting their workplace communication abilities Students were required toanalyse the fluency features and formulaic language of native speaker (NS)models of speech in genres relevant to the professional workplace and to conductethnographic research and analysis with NSs in face-to-face communication onsimilar themes Results of the course show that fluency and proficiency in generalwere improved for the majority of the students

Background This paper is a report of a speech fluency-focused pedagogical intervention

conducted with a group ofE S Lmixed first language (L1) students ofengineering in a university setting The students were enrolled in advancedlevels of the university’s engineering programmes and were applying forcooperative education placements in private enterprises in the community.Engineering faculty and the university cooperative education office, incollaboration with host site managers in the community, recommendedspecific spoken English assistance for such students, noting that, whileoften highly skilled and proficient in their field of study, they tended tostruggle with the job interview process and dealing with spoken Englishwhile in the workplace

In response to this, the university established specifications and assessmenttasks for spoken English for the workplace, and a course was designed tohelp the target students to improve their English speech proficiency Thespecifications and assessments are described in detail below, and the coursewas established as a 48-hour series of workshops to meet twice a week for

6 weeks Half of this time, or 24 classroom hours, was devoted to speechfluency development, and the rest was designed to concentrate onpronunciation and other aspects of speech

In the fluency-focused component of the course, attention was paid to thetemporal markers of speech which indicate fluency, as well as the importantrole played by multiword language phenomena called ‘formulaic

sequences’ These are generally defined as multiword strings or frameswhich appear to be mentally stored and retrieved as single words and which

E LT Journal Volume 63/4 October 2009; doi:10.1093/elt/ccp005 323

ª ª The Author 2009 Published by Oxford University Press; all rights reserved.

have a wide range of meanings and serve important functions in spokendiscourse In order to understand the body of knowledge about fluency andformulaic sequences which informed the spoken English course, it isnecessary to review some of the key research in these areas.

Fluency Research over the decades from the 1960s through the 1990s established

that speech fluency is largely a function of temporal variables of speech,which are speech rate, repairs, amount and frequency of hesitation, location

of pauses, and length of runs of fluent speech between pauses (see Wood

2001 for a review) Research also shows that the ability to speak fluentlyinvolves facility in producing a repertoire of formulaic speech units,multiword strings, or frames which are retrieved from long-term memory

as if they were single words (Wray 2002; Wood 2006) These formulaicsequences include, among others, two-word collocations, such as ‘goodtime’ or ‘first step’, phrasal verbs, such as ‘run into’ or ‘come across’, idioms,routine expressions with social pragmatic functions, such as ‘have a goodday’ or ‘how are you’, whole clauses, discourse markers, such as ‘on the otherhand’ or ‘in summary’, and frames with fillable lexical slots, such as ‘a(year/day/week ) ago’ or ‘a (one/two/three ) step process’

Formulaic language The consensus in definitions seems to be that formulaic language

sequences are multiword units of language which are stored in long-termmemory as if they were single lexical units (Pawley and Syder 1983; Wrayand Perkins 2000) Hickey (1993) puts it most succinctly by noting thatdefinitions of formulaic language units are often expressed in terms ofprocesses, referring to multiword or multiform strings produced andrecalled as a chunk like a single lexical item rather than being generatedfrom individual items and rules

In speech production, formulaic sequences may be uttered withphonological coherence (Coulmas 1979; Wray 2002, 2008), with nointernal pausing and a continuous intonation contour Phonologicalreduction may be present as well, such as phonological fusion, reduction

of syllables, and deletion of schwa These are common features of theutterance of the most high-frequency phrases in English, but much less

so in low-frequency or more novelly constructed utterances, according toBybee (2002) Phonological reduction can be taken as evidence that ‘much

of the production of fluent speech proceeds by selecting prefabricatedsequences of words’ (Bybee 2002: 217)

Researchers have identified other conditions necessary for wordcombinations in speech to be considered formulaic (Coulmas 1979; Wray

2002, 2008) Two conditions, that the unit must be at least two morphemeslong and cohere phonologically, are seen as necessary for formulaicity Aswell, formulas may outstrip other output in terms of length and complexityand be invariant in form and be used for specific situational purposes Wray(2002) adds structure or form of the sequences as an additional criterion,and she notes that such strings often begin with conjunctions, articles,pronouns, prepositions, or discourse markers (p 31) According to Wray andPerkins (2000), many formulaic sequences are not composed semantically,but are holistic items which may exhibit syntactic irregularity This may bemanifest in two qualities: a restriction on manipulation, for example, one

324 David Wood

cannot pluralize ‘beat around the bush’ or passivize ‘face the music’ or say

‘you slept a wink’ or ‘feeding you up’; the fact that, in formulaic languagenormal restrictions are flouted, such as the sequences which contain an

intransitive verb + direct object, for example, ‘go the whole hog’ or other

gross violations of syntactic laws like ‘by and large’ Of most relevance to theidentification of formulaic sequences in spoken language is Wray’sstatement that ‘it may simply be that identification cannot be based on

a single criterion, but rather needs to draw on a suite of features’ (p 43).Indeed, formulaic sequences cross a wide range in terms of form andfunction and may include idioms, proverbs, phrasal verbs, entire phrases orclauses, and collocations of varying length

One key issue in the examination of formulaic language is the question ofwhat is meant by ‘produced or recalled as a whole’ According to Weinert(1995), perhaps formulas are recalled based on the linear surface order oftheir parts, or by their phonological units, following a particular type oflexical recall first and last words of a unit might be most prominent inmemory and act as triggers for total recall Alternatively, it could be thatformulas are stored as ‘cognitive bundles’, retrieved in various ways,depending on their form, pragmatic aspects, and so on It may be that, inaccordance with the variability of automatic and controlled processingsuggested by Kahnemann and Treisman (1984), there is a continuum ofcreative/formulaic–holistic processing, for example, different for differentcategories of formulas Or perhaps a formula is sometimes retrievedautomatically, sometimes partially so, and sometimes by controlledprocessing

Formulaic sequences

and language

learners

There appear to be three major benefits to language learners from having

a substantial repertoire of formulaic sequences First of all, the sequencesare generally conventionalized or institutionalized means of expressingparticular meanings or language functions, reflecting what Sinclair (1991)calls the idiom principle This means that control of formulaic sequencescan assist learners in having a native-like mastery of language, soundingnatural and producing language appropriate to particular contexts andgenres For students inE S Pcontexts or training to enter the workplace, thisparticular function of formulaic language is of high value Second, asoutlined above, use of formulaic sequences has a processing benefit and canallow speakers to produce more fluent language, by, for example, reducingthe hesitations in speech and permitting attention to various aspects oflanguage production simultaneously The third major benefit of usingformulaic sequences is that they can provide chunks of accurate language;

as Boers, Eyckmans, Kappel, Stengers, and Demecheleer (2006: 247) state,

‘these prefabricated chunks constitute ‘‘zones of safety’’ and appropriate use

of them may thus confine the risk of ‘‘erring’’ to the spaces in between theformulaic sequences in one’s discourse’ While fluency was a key element ofthe course described here, all these three proficiency-related benefits offormulaic sequences were built into the specifications for the course and itsdesign

It has been noted by Wray (2002: 174, 175, 182) that there appears to be

a great deal of individual variation in L2 learner use of formulaic sequencesand that they tend to be acquired readily at early stages of acquisition, but not

at later stages She observes that the individual variation may have to dowith the interactional purposes for speaking, as well as the internal andexternal pressure on the learner to achieve particular speech goals Wray(2002) also points out that likely reasons for the decline in acquisition offormulaic sequences at higher levels of language acquisition may be thereduced and simplified language input that learners receive, along with thefact that collocational relationships among words tend not to be noticed orimmediately salient to learners The fluency course described here is aneffort to address these phenomena First, by raising learner awareness of theneed to attend to the language requirements linked to particular speechpurposes, it is expected that they will gain a heightened perception of theformulaic and collocational conventions used to accomplish particularfunctions Second, the course is designed to draw learner attention to thefunctions and the communicative power of formulaic language andcollocation that they may miss in everyday communication Beyond theinstillation of awareness, the course is also designed as a means of training

in language production involving formulaic language

The fluency course The design of the fluency course was undertaken with two pedagogical foci

in mind and geared towards the spoken English for the workplacespecifications which were developed at the university for these particulartypes of students One focus was the value of spontaneous native speaker(NS) input and student ethnographic analysis of that input, following theanalysis–observation–analysis cycle outlined by Riggenbach (1999) Theother focus was the need for activity which can help to automatize pieces oflanguage such as formulaic sequences, informed to a large extent by fluencyworkshop activities described by Wood (2001) in the context of generalES Linstruction

The specifications for

spoken English for

the workplace

The means used to place the learners into the spoken English course was

a set of simulations consisting of three tasks related to the types ofspeech activities engineering cooperative education students mightencounter in their job interviews and placements The first task requiredcandidates to describe their interests in their field of study, within atwo-minute time limit Candidates were instructed to talk about theirinterest in the field, the contributions they felt they could offer, and thebenefits they expected from their cooperative work placement The secondtask simulated a meeting in which the candidate was to listen to a managerannounce a change in the work environment, covering three main points:what the planned change was; reasons for the change; and contributionsexpected from employees After listening to the manager, the candidate was

to orally relay the information to a colleague The third task requiredcandidates to read an inter-office memo and answer oral questions from

a customer

The simulation was scored using the rubric below (Table 1), which coveredcontent, organized presentation of material, overall comprehensibility,grammatical and phonetic accuracy, and fluency In the rubric, 1 representsthe lowest score and 5 the highest The learners who scored below a 4 overallwere enrolled in the spoken English course in an effort to boost theirperformance

326 David Wood

Low HighClarity of expression

Course structure The structure of the course was a blend of analysis of fluency features of NS

input and fluency-focused activities

NS model analysisThe 16 learners registered in the course listened to and analysed threesamples of NS speech related to the themes of the test The first was a shortspontaneous monologue from a colleague who was a language teacher, inwhich he explains why he chose this field of work, what benefits he getsfrom it, what he expects to contribute to it This relates thematically to thefirst part of the speech specifications The second NS speech sample wasfrom a person who was working in managing language teachingprogrammes, in which she explains upcoming changes to the programmesand what the implications are for the work of the teachers This relatesthematically to the second part of the specifications The third NS model wasfrom a manager of software design and programming in a governmentdepartment, in which he discusses how lack of funding has had an impact

on particular services and how this can be addressed to meet a particulardeadline This relates thematically to the third part of the specifications

A transcription of each NS sample was presented to the group, and the classlistened and marked hesitation patterns and intonation contours on thetranscripts The instructor drew the attention of learners to the formulaicsequences used in the discourse, after which the class shadowed the texts

a number of times

Classroom activitiesFollowing this, several fluency-focused activities were conducted in an effort

to automatize and encourage memory of the formulaic sequences First

in this task sequence was dictogloss, in which five key utterances fromthe transcript were dictated to the class Following standard dictoglossprocedures, the five utterances were read aloud to the class at normal speed,with a five-second delay between each piece Learners jotted down what they

could catch and worked in groups to reconstruct the text as accurately aspossible They referred back to the transcript to check their final productswith the original This was followed by a mingle activity, in which eachlearner was given a key utterance from the transcript and told to memorize

it Then, each class member approached a classmate and recited his or herutterance until the classmate could recall it verbatim and vice versa The twolearners could then return to their seats and write down the utterance theyhad heard from the other This continued until all members had collected allthe distributed utterances through repeating and transcribing The groupthen had the opportunity to check what they had written with the originals.Two other activities followed from these, both building on the awareness offormulaic sequences from the NS transcripts The first of these was a chatcircle, in which the class stood in two concentric circles, the inner circlefacing outwards and the outer circle facing inwards so that learners werepaired Topics related to the speech model were then announced and themembers of the pairs took 90-second turns speaking about it After bothmembers of the pairs had a chance to speak, they shifted partners and dealtwith a new topic The next activity was Nation’s (1989) four/three/twoactivity to boost fluency Here, learners were to choose one of the threemodel speech sample topics and prepare a four-minute talk about it Afterdelivering this talk to a partner, they were required to give another talk usingexactly the same content to another partner, but in a three-minute time, thenshift again, and deliver it in two minutes Learners followed all this activitywith delivery of their talks in small groups, receiving feedback on fluencyand formulaic sequences from their peers

After this, learners recorded themselves describing their interests,contributions, and expected benefits from their careers The instructorprovided fluency and formulaic sequence-based feedback on theserecordings

Ethnographic activity In the next section of the course, learners were paired with teacher

education students from a university education faculty While one mighttake the view that the learners would best have met with engineers, insituations such as this instructors and course planners will be constrained

by practicality issues such as the availability of motivated NSs The mainpurpose of the contact activity in this situation was to expose the participants

to native or native-like spontaneous speech in English in an interviewcontext, independent of discipline-specific language They were required toconduct an interview with the education students on topics related to those

of the simulations in the initial cooperative education placement test Theinterview guide is discussed further below

The education students were required to interview the cooperativeeducation students to hear their English language learning experiences and

to determine what strategies and classroom activity types they had foundvaluable or engaging as students The cooperative education students were

to converse with the teacher education students using a template (seeAppendix 1) and had to record their responses and make transcripts of theiranswers At the next class meeting, learners worked to pool their transcripts,recordings, and findings, using the instructions in Appendix 2

328 David Wood

Learners used all the resources, including instructor feedback, theethnographic data they collected from the university teacher trainees, andin-class experiences and peer feedback to refine their own speech on thethree themes related to the test of speaking: discussing why they chose

a particular career and what they expected to contribute to it and benefitsthey expected from it; workplace changes and their causes and the expectedcontributions of employees; challenges in meeting deadlines, reasons forthat, and plans to rectify the situation

Throughout, learners were required to pay careful attention to fluencyfactors and formulaic sequences which were relevant to the speaking tasks

Final presentation The entire sequence of work culminated in an in-class performed repeat of

the simulation tasks used initially in placing the learners in the course.Their peers, along with three faculty members from the departmentresponsible for the language training all completed the evaluation template(see Table 1) for each learner

The learners generally scored higher in criteria related to speed, hesitation,intonation, and rhythm and vocabulary, phrases, and grammar in the finalpresentation At the beginning, overall scores for six of the learners were 3,with the other ten scoring 4 In the final presentation, all scores were in the 4range, with six of them advancing from 4 to 4.5 This shows someimprovements in expression after only three weeks of bi-weekly classmeetings, along with transcribing and analysis outside class

Key formulaic sequences used in the NS models in class appeared in thefinal presentation performances, such as

n ‘We would have to make changes’

n ‘We’ve noticed that’

n ‘That’s sort of my main reason for’

n ‘Have left some very deep impressions’

n ‘Trying to get through different hoops’

n ‘To figure out how to use’

n ‘There are a number of contributions I could make’

n ‘To weave together my interests and skills’

Conclusion The course described here is an example of how knowledge about fluency

and formulaic sequences can help in the design of effective languagetraining for advanced learners ofE S P The use of NS models andethnographic activities with NSs within specific genres provided thestudents with a repertoire of language and an awareness of the expectations

of the contexts in which they were to communicate in English The higherscores in evaluation at the end of the course indicate language improvementover just three weeks of training

The procedures outlined in this paper are readily adaptable to otherlanguage teaching contexts Virtually anyE S PorE A P(English forAcademic Purposes) programme can make use of the model analysis,fluency tasks, and ethnographic activities described here, although this type

of programming may be constrained somewhat by access to NS models, or,for that matter, NSs themselves Use of technology including chat programs

and Voice Over Internet Protocol could assist in making links betweenlearners and NSs, or highly proficient L2 learners might also stand in forNSs, depending on the needs and constraints of the programme and itsgoals.

Further classroom applications of our knowledge of formulaic sequencesare essential; as research into this language phenomenon grows, manypotential classroom applications follow along The growing awareness ofhow fundamental formulaic language is to communication, and thenature of the sequences themselves opens up many possibilities forsyllabus and curriculum design as well as specific types of classroomactivities to facilitate greater speech proficiency for L2 learners

Final revised version received December 2008References

Boers, F., J Eyckmans, J Kappel, H Stengers,and

M Demecheleer.2006 ‘Formulaic sequences and

perceived oral proficiency: putting a lexical approach

to the test’ Language Teaching Research 10/3: 245–61

Bybee, J.2002 ‘Phonological evidence for exemplar

storage of multiword sequences’ Studies in Second

Language Acquisition 24/2: 215–21

Coulmas, F.1979 ‘On the sociolinguistic relevance

of routine formulae’ Journal of Pragmatics 3/3–4:

239–66

Hickey, T.1993 ‘Identifying formulas in first

language acquisition’ Journal of Child Language

20/1: 27–41

Kahnemann, D.and A Treisman 1984 ‘Changing

views of attention and automaticity’ in

R Parasuraman and D R Davies (eds.) Varieties of

Attention New York: Academic Press

Nation, P.1989 ‘Improving speaking fluency’

System 17/3: 377–84

Pawley, A.and F H Syder 1983 ‘Two puzzles for

linguistic theory: nativelike selection and nativelike

fluency’ in J C Richards and R W Schmidt (eds.)

Language and Communication New York: Longman

Riggenbach, H.1999 Discourse Analysis in the

Language Classroom, Volume 1: The Spoken Language

Ann Arbor, MI: University of Michigan Press

Sinclair, J.1991 Corpus, Concordance, Collocation.Oxford: Oxford University Press

Weinert, R.1995 ‘The role of formulaic language insecond language acquisition: a review’ AppliedLinguistics 16/2: 180–205

Wood, D.2001 In search of fluency: what is it andhow can we teach it? Canadian Modern LanguageReview 57/4: 573–89

Wood, D.2006 ‘Uses and functions of formulaicsequences in second language speech: anexploration of the foundations of fluency’ CanadianModern Language Review 63/1: 13–33

Wray, A.2002 Formulaic Language and the Lexicon.Cambridge: Cambridge University Press

Wray, A.2008 Formulaic Language: Pushing theBoundaries Oxford: Oxford University Press.Wray, A.and M R Perkins 2000 ‘The functions offormulaic language: an integrated model’ Languageand Communication 20/1: 1–28

The authorDavid Woodteaches Applied Linguistics andT ES LatCarleton University in Ottawa, Canada His researchinterests centre on the role of formulaic language insecond language acquisition and implications forteaching

Email: david_wood@carleton.ca

Appendix 1 You will meet with several NS visitors from another university They will

have some research questions to ask you

You should prepare to discuss with them the following points:

1 Why they chose teaching as a career

2 What contributions they feel they can make to the teaching profession

330 David Wood

3 What benefits they expect to gain from a teaching career.

4 When to schedule another appointment to meet

5 The history of their studies until now

6 An example of changes in their studies or work and what they had to do

Before next class, make a transcript of their answers Bring the transcriptand recording to class

Appendix 2 Using your transcripts and referring to the tapes you made from discussion

with the university teacher trainees, join with a partner and find somesample phrases, words, and expressions used for these purposes:

n explaining a personal choice

n listing personal characteristics and/or qualifications

n telling about life experiences

n introducing the idea of contributions

1 The instructor’s feedback

2 The experience with the teacher trainees, including the recordedsamples and transcripts and our in-class discussions about it

3 Our sample recording with transcript

Tape yourself again

Record yourself talking about a change in a workplace Explain what thechange is, why it is happening, and what the contributions are expectedfrom employees

How good is your test?

Funda Ku¨c xu¨k and JoDee Walters

This article reports on a study of the validity and reliability of tests administered in

anEFLuniversity setting The study addresses the question of how well face validityreflects more objective measures of the quality of a test, such as predictive validityand reliability According to some researchers, face validity, defined as the surfacecredibility or public acceptability of a test, has no theoretical basis since it is based

on the subjective perceptions of stakeholders such as teachers and students.However, due to lack of time or resources, or due to a perceived lack ofcompetence, practitioners tend to rely on the ‘appeal’ of language tests, ratherthan seek empirical evidence This article describes several ways of evaluatingachievement tests, comparing their results in order to shed light on what measurescan and should be taken to ensure that achievement tests accomplish theirpurposes

Background In large educational institutions, achievement tests are designed by testing

offices, rather than individual teachers, to ensure standardization

Unfortunately, instructors and test takers may not trust these tests or thetesters (Hughes 2003: 1) Even if teachers design their own achievementtests, such tests may not accurately reflect the students’ language knowledgeand skills For these reasons, the assessment of achievement tests is crucial.One way to assess such tests is to examine the qualities that determine theireffectiveness Bachman and Palmer (1996: 38) define these ‘good qualities’

as reliability, validity, authenticity, interactiveness, wash-back impact, andpracticality Bachman (1990: 289) identifies validity as crucially important

In general, validating a test involves gathering empirical data and otherrelevant information to show that the test is indeed measuring what itintends to measure There are several validity types, including predictivevalidity and face validity, each of which entails collecting data in differentways

To investigate predictive validity, which indicates that the test accuratelypredicts the possible future success or failure of the test takers (Hughes2003), test scores are correlated with scores on tests taken some time later.Face validity, another way of looking at the validity of a test, refers to thedegree to which the test seems valid, in the eyes of those involved in taking oradministering it, in terms of testing what it has to test (ibid.) Research intoface validity requires investigation of the subjective judgements andperceptions of the test’s stakeholders (preferably both instructors andstudents)

While validity is a fundamental quality of tests, reliability is a preconditionfor validity, because unreliable test scores cannot provide suitable

332 E LT Journal Volume 63/4 October 2009; doi:10.1093/elt/ccp001

ª

ª The Author 2009 Published by Oxford University Press; all rights reserved.

Advance Access publication February 23, 2009

grounds for valid interpretation and use (Bachman 1990) Two essentialconcepts are involved in reliability: ‘scorers’ reliability’ and ‘reliability interms of the test takers’ performance’ (Hughes op.cit.) Scorers’

reliability refers to the degree to which test scores are free frommeasurement error (Rudner 1994: 3) Reliability in terms of the testtakers’ performance refers to the extent to which test scores of agroup of test takers are consistent over repeated test applications(Hughes op.cit.)

Several researchers have conducted studies in an attempt to assess thereliability (Cardoso 1998; Nakamura 2006) or validity (O¨ sken 1999; Serpil2000; Yeg˘in 2003) of tests given in university English preparatoryprogrammes However, none of these studies has explicitly compared facevalidity with more objective measures of tests Therefore, this study aimed atboth measuring the validity and reliability of achievement tests

administered in a particularE F Lsetting and exploring how well face validityreflects relatively more objective measures of tests: reliability and predictivevalidity

The study This study addresses the following research questions:

1 To what extent do the achievement tests possess face validity in the eyes

of the instructors and students?

2 To what extent do the achievement tests possess reliability, in terms ofboth (scorers’ perceptions of) scorer reliability and the test takers’(perceptions of) performance?

3 To what extent do the achievement tests possess predictive validity?

4 How closely does the face validity of the achievement tests reflect thereliability and predictive validity of these tests?

The study was conducted at the Zonguldak Karaelmas University (Z KU)Foreign Languages Compulsory Preparatory School, in Zonguldak, Turkey.WhileZ KUis not an English medium university, the preparatory school(PS) aims to prepare students for those courses in their future

university departments that will include some teaching and materials inEnglish In the PS, students attend speaking, writing, and reading courses

in addition to a main course of integrated skills, which includesgrammar instruction There are also video and language laboratory courseswhich include listening skills Two midterm achievement tests are given ineach term, and a final exam is administered at the end of the course Onthese tests, while grammar, writing, vocabulary, and reading are

represented, listening is not tested at all, and speaking is represented only

on the final exam, with its weight relatively insignificant Neither thevideo nor the laboratory course is separately represented on the tests.After PS, the students move on to a General English (GE) course,addressing the four language skills, taken alongside their regular universitycourses

This study, conducted in 2007, was focused on the tests administered by the

PS during the 2005–2006 academic year Two different groups ofparticipants were included in this study Fifty-two students (four from eachuniversity department) who had been enrolled during the 2005–2006academic year formed the first group, and 29 instructors who had been

teaching at the school in the same year formed the second group AtZ K U,instructors rotate through the testing office, and all instructors participate inscoring the tests Therefore, the instructors had all been involved in thepreparation and scoring of tests.

Two questionnaires were employed in this study, one for instructors andone for students Common sources of invalidity and unreliability suggested

by the literature (for example Brown 1996, 2004; Genesee and Upshur1996; Hughes op.cit.) provided a basis for the Likert scale items The firstsections of both questionnaires contained the same questions, concerningparticipants’ perceptions of the face validity of the achievement tests Thesecond section of the instructors’ questionnaire investigated scorerreliability, and the second section of the students’ questionnaire concernedissues related to test takers’ performance It should be noted that thesesections, while purportedly measuring the reliability of the tests, are relying

on the participants’ perceptions of reliability However, many of the questionsaddress very specific and observable aspects of the testing experience and soare thought to be sufficiently objective The students’ questionnaire wasadministered in Turkish, while the instructors’ questionnaire was given inEnglish

In addition to the questionnaires, the 2005–2006 midterm and finalexam scores for 365 students were collected, as well as scores for the2006–2007 GE classes These scores were used to investigate predictivevalidity

Data analysis and

results

The scales in Table 1 were used in interpreting the means of the Likert scaleitems

Mean Positively oriented questions Negatively oriented questions

4.5–5 Very high Strongly agree Very low Strongly disagree

1.0–1.4 Very low Strongly disagree Very high Strongly agree

table1

Interpreting Likert scale

responses

Face validity Face validity was determined by asking both instructors and students

about how well they felt the contents of the courses were represented

on the achievement tests (as in O¨ sken (op.cit.) and Serpil (op.cit.)).The means of both groups’ responses to these questions can be seen

in Table 2 The means for Q12 indicate that both the instructors andthe students agree that the content of the courses is adequatelyrepresented on the tests, and, looking at the individual questions,

we can see that, for many of the questions, both teachers andstudents respond positively However, the groups appear to disagree

on Q5 and Q6, an expected result, since, as noted previously,speaking and video courses are essentially unrepresented on theexams Listening and laboratory courses are also not represented onthe tests, and for the instructors, the means for the relevant questions,Q9 and Q10, fall into the ‘undecided’ range It is unclear why the

334 Funda Ku¨cxu¨k and JoDee Walters

mean for Q9 is not lower, as listening is not represented on the test

at all, but it may be that some instructors believe that testing listening

is not necessary in this setting The students, on the other hand, doappear to believe that listening should be tested, given theirnegative response to Q9 Like the teachers, they are undecidedabout whether the content of the laboratory courses is sufficientlyrepresented Both teachers and students are also undecided aboutQ4, which relates to the content of the reading courses Overall,even though it appears that there are some areas of the course thatare under-represented on the tests, it can be concluded that theinstructors and students see the tests as possessing a high degree offace validity

mean

StudentsmeanQ1 The content of the main course book ‘Quartet’

was represented in the exams sufficiently

Q2 The content of the grammar book ‘Milestones’

was represented in the exams sufficiently

Reliability Previous studies have looked at scoring reliability issues either by

observing scoring practices (Brown 2003) or by examining the scoresthemselves (Manalo and Wolfe 2000), and no studies have examinedreliability from the test takers’ perspective Reliability in this study wasmeasured by asking specific questions of the instructors and studentsregarding the administration and scoring of the tests The means of theinstructors’ responses to the questions about scorers’ reliability arepresented in Table 3 The mean for Q13, the instructors’ overallperception of scorers’ reliability, falls into the range of ‘agree’ Thisindicates that, in the eyes of the instructors, the degree of scorer’s reliability

is relatively high However, despite this overall impression, the means

for several questions indicate some potential problems in scorers’

reliability The instructors do not agree that students are identified bynumber rather than name for subjective scoring, and they are undecided

as to whether more than one scorer was used for such scoring, whetherall scorers are trained, and whether their colleagues scored the examsreliably

Q1 The questions included in the exams permitted objectivescoring

3.724Q2 Testing office provided a detailed answer key 4.138Q3 The scorers who marked the exam papers were trained 3.344Q4 Students were identified by number, not name when scoring

was subjective (e.g in writing sections) to provide objectivity

2.448Q5+Only one instructor scored each exam paper when scoring was

subjective

3.448+Q6 The rating scales included in the key helped me while I was

scoring the exam papers

3.828Q7 We had meetings to agree with acceptable answers after the

exams

4.448Q8 The class which I instructed as the main course teacher and

the class which I invigilated during the exams were two differentclasses

4.670

Q9 The class which I instructed as the main course teacher and theclass whose papers I scored were two different classes

4.670Q10+The deadline for scoring and returning the exam papers

to the main course instructors affected my scoring practicesnegatively

table3

Scorers’ perceptions of

reliability

+ Indicates negatively oriented items.

In Table 4, the means of the students’ responses to questions about teststructure are given The mean for Q21, which asks whether, overall, thetest structure had a negative impact on performance, is 3.596,

indicating disagreement (on the negative-orientation rating scale) Inother words, in the eyes of the students, it appears that the degree ofreliability in terms of the test structure is relatively high The students’responses to individual questions give greater insight into specificaspects of test structure Of the four negatively oriented questions(Q1, Q2, Q3, and Q8), the means of Q3 and Q8 indicate disagreement,pointing towards test structure reliability in terms of the number

of questions and difficulty level of the questions The means of Q1and Q2, however, indicate that the students are undecided as to theindependent nature of the test questions, as well as whether there aretoo many questions The means of the remaining, positively orientedquestions all indicate agreement Thus, it can be concluded

that, according to the students, the test structure contributes toreliability

336 Funda Ku¨cxu¨k and JoDee Walters

Questions MeanQ1+Sometimes, two (or more) questions in the test seemed to be

closely related, so that if I could not answer one question, I could notanswer the other question either

2.846+

Q3+The exams included an insufficient number of questions 3.904+Q4 The instructions explaining what to do in each section in the exams

were explicit and clear

3.769Q5 The points allotted for each section of the exam were always stated

in the exam papers

4.692Q6 Time given to the students to complete the exam was always stated

in the exam papers

4.556Q8+All the questions in the exams had the same difficulty level 3.654+

Q12 The tables which were employed in the exams were clear and easy

to interpret

4.231Q21+In general, the structure of the tests hindered my ability to

display my best performance in the exams

3.596+

table4

Students’ perceptions of

test structure

+ Indicates negatively oriented items.

Table 5 presents the means of the students’ responses to questions abouttesting conditions The mean for Q22, which asks whether, overall, thetesting conditions had a negative effect on the students’ performance,indicates disagreement (on the negative-orientation rating scale) Questions

14 and 15 asked about the amount of time given for the test; the students donot agree that the time given is too short, but they are undecided aboutwhether too much time is given They also do not agree that the light,temperature, or ventilation in the testing environment hinders theirperformance, but they are undecided about the effect of noise The meansfor the remaining questions indicate agreement Overall, it appears that, inthe eyes of the students, the degree of reliability in terms of the testingconditions is relatively high

Q7 Information about how much the given tests would affect the final grade was always announced.

3.519 Q13 The instructors helped us to get used to the format of the exams 4.039 Q14+The time given to complete the exams was too short 3.865+Q15+The time given to complete the exams was too long 3.308+Q16 Equal timing was given to all classes which took the same test 4.558 Q17+Distracting sounds and noises lowered my performance in the exams 3.077+Q18+The little amount of light in the classrooms lowered my

performance in the exams.

4.096+Q19+The degree of the temperature in the classrooms lowered my

performance in the exams.

3.750+Q20+The little amount of air in the classrooms lowered my

performance in the exams.

3.635+Q22+In general, the bad environmental conditions hindered my

ability to display my best performance in the exams.