Elicited Imitation Toward Valid Procedures to Measure Implicit Second Language Grammatical Knowledge

DUFF University of British Columbia Elicited Imitation: Toward Valid Procedures to Measure Implicit Second Language Grammatical Knowledge YASUYO TOMITA University of Toronto Toront

RESEARCH ISSUES

TESOL Quarterly publishes brief commentaries on aspects of qualitative and

quantita-tive research This issue features a discussion of research involving elicited imitation.

Edited by PATRICIA A DUFF

University of British Columbia

Elicited Imitation: Toward Valid Procedures to Measure Implicit Second Language Grammatical Knowledge

YASUYO TOMITA

University of Toronto

Toronto, Ontario, Canada

WATARU SUZUKI

Miyagi University of Education

Sendai, Miyagi, Japan

LORENA JESSOP

University of Toronto

Toronto, Ontario, Canada

 In second language acquisition (SLA) research, two types of second language (L2) knowledge, 1 explicit and implicit, have been discussed for almost three decades 2 Although many SLA researchers agree that L2 instruction should give priority to implicit knowledge, researchers have not agreed on what type of test (e.g., oral narrative test and timed grammaticality judgment test) actually assesses these constructs of L2 implicit knowledge According to Ellis (2005), some important constructs

of implicit knowledge include (a) response according to feel rather than declarative rules, (b) spontaneity rather than planned behavior, and (c) a primary focus on meaning before forms Ellis and his colleagues have

1 L2 knowledge consists of phonological, lexical, grammatical, pragmatic, and sociolinguistic features However, this article will focus exclusively on L2 grammatical knowledge

2 Explicit knowledge refers to “knowledge of language about which users are consciously aware” (Ellis, 2004, p 229), whereas implicit knowledge is used “without attention to the role or even an ability to state it” (Bialystok, 1979, p 82)

recently attempted to develop elicited imitation (EI) as a measure of

L2 implicit knowledge (e.g., Ellis, 2005, 2006; Ellis, Loewen, & Erlam,

2006; Erlam, 2006) 3 In this article, we provide an instrument that can be

used by SLA researchers who wish to use EI to measure the implicit

gram-matical knowledge of adult L2 learners The primary purpose of this

instrument (Appendix) is to allow researchers to systematically refl ect on

their proposed EI research First, we will defi ne EI, and then we will

pro-vide suggestions to SLA researchers who wish to use our instrument 4

VALIDATION PROCEDURES

EI, which is commonly performed in a laboratory setting, requires

par-ticipants to hear and then repeat a sentence that usually includes a target

grammatical structure EI is assumed to be reconstructive in nature; that

is, during EI, participants reconstruct a stimulus sentence with their own

interlanguage grammar We would like researchers using EI to use our

seven criteria in considering the validity of their EI task as a measure of

implicit knowledge

1 It is important to ensure that the performance of EI is not greatly

infl uenced by participants’ rote repetition abilities Therefore,

stimu-lus sentences must be presented under conditions in which rote

rep-etition is not in play

2 Similar to the fi rst criterion, participants’ capacity to store or hold

information (i.e., short-term memory abilities) must not greatly infl

u-ence EI performance Thus, stimulus sentu-ences should exceed

partici-pants’ short-term memory capacity Keeping in mind that regardless

of a stimulus sentence length, the fi rst and last items are recalled

bet-ter than the middle items (i.e., serial order effect ), the target structures

should be embedded in the middle of the stimulus sentences

3 In order for EI tasks to measure implicit knowledge, participants

must attend to meaning rather than form

4 The performance of EI must not be greatly infl uenced by linguistic

complexity or diffi culty; some grammatical structures are easy to

repeat, whereas others may be diffi cult

5 It is important to ensure that the performance of EI is not greatly

infl uenced by the task’s ease or diffi culty (i.e., fl oor and ceiling effects )

Therefore, stimuli should be neither too easy nor too diffi cult

6 The performance of EI should not be greatly infl uenced by

instruc-tions about how to do EI tasks Therefore, clear instrucinstruc-tions for

3 For a review, see Jessop, Suzuki & Tomita (2007)

4 Note that this instrument should not be seen as prioritized or exhaustive

the participants must be provided For example, “Repeat the sen-tences exactly as you hear them” and “repeat the sensen-tences in correct English” may provide different results when the stimuli include ungrammatical structures

7 Obtaining comparable results from multiple measures ensures that

EI actually taps into implicit knowledge When EI measures implicit knowledge, a positive correlation between the participants’ perfor-mance in the EI task and their perforperfor-mance on other L2 implicit measures (e.g., oral narrative tasks, timed-grammaticality judgment tests) should be found

ETHICS AND FAIRNESS

Ethics and fairness include consideration of the testing environment, test materials, and participants’ background knowledge In order to main-tain fairness between test takers, the testing environment should be com-parable for all test takers, and the materials in the tasks should be fair and ethical to participants in terms of gender, religion, language, race, and eth-nicity (see International Language Testing Association, 2001) Participants’ differences in background knowledge, including linguistic and cultural knowledge, must not give some participants an unfair advantage over oth-ers Learners must also be appropriately informed and trained

IMPACT

Researchers in SLA must consider EI’s impact on L2 learning, teach-ing, and research The EI task should be conducted such that it will impact participants’ L2 learning as positively as possible 5 EI should also have a positive impact on L2 teaching, by raising teachers’ and materials developers’ awareness toward learners’ implicit knowledge L2 research should consider factors that have an impact on EI, such as implicit knowl-edge and mediating factors that affect EI performance

PRACTICALITY

Before initiating a study using EI, the issue of practicality must also be addressed If appropriate and suffi cient resources are not available, the study cannot be successful Resources range from suffi cient time, partici-pants, data, and expertise to economic resources during the designing, implementation, and analysis stages of the project

5 Ideally, the task should positively infl uence the participants in some way Thus, researchers should provide meaningful feedback that will promote participants’ L2 learning

SCORING

Because scoring strongly infl uences analyses and results, the scoring

must accurately refl ect the participants’ implicit knowledge about the

target Detailed scoring procedures must be provided, including the

types of mistakes that will or will not be scored, along with the rationale

Construct validity must also be present in scoring, bearing in mind that

grammar is being assessed, not pronunciation or knowledge about the

topics used in the EI task

Scoring interrater and intrarater reliability should also be achieved

Two or more trained researchers should score at the data with high

inter-rater reliability (e.g., at least 80%; see Mackey & Gass, 2005) If only one

researcher can score, the scoring should be done at least twice, with a

relatively long time interval between, ensuring scoring agreement with

high intrarater reliability

REPORTING IN ACADEMIA

Researchers should be careful when reporting the results of EI to the

public both orally (e.g., conference paper presentations) and in writing

(e.g., journals, technical reports) They must ensure that participants’

confi dentiality and anonymity are maintained They must also present

suffi cient information to assist L2 researchers who want to replicate the

reported studies and to enable them to accurately interpret the reports

Such information includes lengths of sentences, number of repetitions of

presented stimuli, timing, rationale for selecting stimuli sentences, and

instructions to participants

ACKNOWLEDGMENTS

The authors express their gratitude to Alister Cumming for helpful comments in the

preparation of this manuscript This article is based on a paper presented at the

annual meeting of the Japan Society of English Language Education in Oita, Japan

(August, 2007)

THE AUTHORS

Yasuyo Tomita is a doctoral candidate at the Ontario Institute for Studies in Education

at the University of Toronto, Canada Her interests include second language

acquisi-tion, form-focused instrucacquisi-tion, classroom research, and learner investment.

Wataru Suzuki is a lecturer in the English Language Department at Miyagi University

of Education, Miyagi, Japan His research interests include second language

acquisi-tion, psycholinguistics, and applied linguistics His recent research is about

languag-ing and communicative language learnlanguag-ing

Lorena Jessop teaches sociolinguistics at Queen’s University in Kingston, Ontario, Canada She is also a doctoral candidate at Ontario Institute for Studies in Education

at the University of Toronto Her interests lie in sociolinguistics, acquisition of L2 phonetics and phonology, and pronunciation instruction

REFERENCES

Bialystok, E (1979) Explicit and implicit judgments of L2 grammaticality Language Learning, 29, 32–68

Bley-Vroman, R., & Chaudron, C (1994) Elicited imitation as a measure of

second-language competence In E Tarone, S Gass., & A Cohen (Eds.), Research methodol-ogy in second-language acquisition (pp 245–261) Mahwah, NJ: Erlbaum

Ellis, R (2004) The defi nition and measurement of L2 explicit knowledge Language Learning, 54, 227–275

Ellis, R (2005) Measuring implicit and explicit knowledge of a second language:

A psychometric study Studies in Second Language Acquisition, 27, 141–172

Ellis, R (2006) Modeling learning diffi culty and second language profi ciency The

differential contribution of implicit and explicit knowledge Applied Linguistics, 27,

431–463

Ellis, R., Loewen, S., & Erlam, R (2006) Implicit and explicit corrective feedback and

the acquisition of L2 grammar Studies in Second Language Acquisition, 28, 339–369

Erlam, R (2006) Elicited imitation as a measure of L2 implicit knowledge: An

empir-ical validation study Applied Linguistics, 27, 464–491

International Language Testing Association (2001) Code of ethics for ILTA

Birmingham, AL: Author Retrieved September 28, 2007, from the ILTA homepage http://www.iltaonline.com/code.pdf

Jessop, L., Suzuki, W., & Tomita, Y (2007) Elicited imitation in second language

acquisition research Canadian Modern Language Review, 64 , 215–220

Mackey, A., & Gass, S M (2005) Second language research: Methodology and design

Mahwah, NJ: Erlbaum

Munnich, E., Flynn, S., & Martohardjono, G (1994) Elicited imitation and gram-maticality judgment tasks: What they measure and how they relate to each other

In E Tarone, S Gass, & A Cohen (Eds.), Research methodology in second language acquisition (pp 227–245) Mahwah, NJ: Erlbaum

Vinther, T (2002) Elicited imitation: A brief overview International Journal of Applied Linguistics, 12, 54–73

APPENDIX

Instrument for Developing Elicited Imitation Tasks as Measures of Adult

Learners’ Implicit Knowledge of Second Language Grammar

Validation: 1 EI performance is not greatly infl uenced by participants’

rote repetition abilities.

2 Participants’ short-term memory capacity does not greatly

infl uence EI’s performance.

3 Participants attend to meaning, not form, during EI tasks.

4 EI performance is generally unaffected by a) linguistic

complexity and/or diffi culty; b) instructions about how

to do EI tasks; c) task diffi culty.

5 Multiple L2 implicit knowledge measures are used to

ensure that EI actually taps into implicit knowledge.

Ethics and

Fairness:

1 The testing environment is fair and comparable for all

test takers.

2 EI materials are equally appropriate for all participants

(i.e., sensitive to issues of gender, religion, language,

race and ethnicity).

3 EI materials are fair to all participants regardless of their

language experiences and background knowledge.

4 Participants will be appropriately trained and informed

for the test.

Impact: 1 The feedback/results will be meaningful for participants,

promoting their L2 learning.

2 The EI task draws teachers’ and material developers’

attention to promoting participants’ implicit knowledge.

3 Results will provide information about implicit knowledge.

4 Results may possibly provide information about factors that

cause oral production problems (e.g., pronunciation).

Practicality: 1 Suffi cient and appropriate resources are available for the

designing, implementation and analysis stages of the project

(i.e., time, participants, expertise, equipment, and money).

Scoring: 1 Detailed scoring procedures, including types of mistakes

that will/will not be scored and reasons for the procedures.

2 Construct validity (e.g., pronunciation versus

grammar) is refl ected in the scoring.

3 Implicit knowledge is refl ected in scoring.

4 Inter and intrarater reliability will be present.

Reporting in

Academia:

Please check Yes, Partially, or No Yes Partially No

1 Participants’ confi dentiality and anonymity

are maintained in the reporting.

2 Lengths of sentences or number of

syllables are reported.

3 Number of repetitions is reported.

4 Instructions to participants are reported

(e.g., “Repeat in correct English”).

5 Timed or untimed performance is

reported If timed, the time between

stimulus sentences is reported.

6 The rationale for selecting sentences is

reported (e.g., topic familiarity, target

grammar).