Computer Assisted Language Learning, Grammar Instruction, Focus on Form, Structural Syllabus, English as a Second Language (ESL), ComputerBased Instruction

Joyce Nutta University of South Florida ABSTRACT The study described here compared postsecondary English as a Second Language ESL students’ acquisition of selected English structures bas

Is Computer-Based Grammar

Instruction as Effective as Teacher-Directed Grammar Instruction

for Teaching L2 Structures?

Joyce Nutta

University of South Florida

ABSTRACT

The study described here compared postsecondary English as a Second Language (ESL) students’ acquisition of selected English structures based

on the method of instruction—computer-based instruction versus teacher-directed instruction.1 The results showed that for all levels of English pro-ficiency, the computer-based students scored significantly higher on open-ended tests covering the structures in question than the teacher-directed students No significant differences were found between the computer-based and teacher-directed students’ scores on multiple choice or fill-in-the-blank tests The results indicate that computer-based instruction can

be an effective method of teaching L2 grammar

KEYWORDS

Computer Assisted Language Learning (CALL), Grammar Instruction, Focus on Form, Structural Syllabus, English as a Second Language (ESL), Computer-Based Instruction

INTRODUCTION

As the use of computers in language teaching increases, it is essential to establish research-based indications of the appropriate roles of Computer Assisted Language Learning (CALL) in curriculum and instruction One

of the most important questions focuses on whether technology should attempt to emulate the characteristics of a communicative classroom, en-gaging students in real and meaningful communication, or provide the types of tutorials and drills that tend to be de-emphasized in current teach-ing practice Numerous Second Language Acquisition researchers have

asserted that the computer should be used to replicate what they believe ought to occur in the classroom Many proponents of CALL have advo-cated the development of communicative computer programs that pro-vide opportunities for meaningful communication (Garrett, 1991; Underwood, 1993; Quinn, 1990; Lavine, 1992) Although some educa-tors have decried the use of computers as electronic workbooks for drill-and-practice exercises (Chun & Brandl, 1992; Underwood, 1993), others have advocated their use for tutorials and drills to free up more classroom time for real communication (Gilby, 1996; Hoffman, 1996)

The teaching context often determines the role of CALL In an English

as a Second Language (ESL) environment, communicative CALL pro-grams often supplement and augment classroom activities by providing games for practice or word processing applications for compositions In some English as a Foreign Language (EFL) programs, computer programs may complement or completely supplant classroom instruction by provid-ing instruction in a subject or skill not taught in the classroom (Soo & Ngeow, 1996) In the latter case, computers and other technologies are relied upon to provide a model of native speech that the instructors, many

of whom are nonnative speakers of English, cannot offer

During the past decade, numerous studies have examined different ap-proaches to teaching grammar (Doughty, 1991; Ellis,1985, 1993; Fotos, 1993; Green & Hecht, 1992; Tomasello & Herron, 1988) These studies have shown that many effective means of teaching L2 grammar are avail-able—from teacher-directed Constructivist tutorials (Adair-Hauck & Donato, 1994), to cooperative group work (Fotos, 1994) and individual study with textbooks (Scott & Randall, 1992) Because these studies indi-cate that various effective ways to learn grammar exist, teachers have an ever increasing array of options with which to meet the needs of students The use of computers to teach grammar has not received the same amount

of attention as communicative CALL, but computer-based grammar in-struction offers many potential benefits Although it is currently impos-sible for the computer to engage learners in authentic two-way communi-cation, it is, in fact, possible for CALL to provide rich input in the form of integrated multimedia programs and to provide explicit grammar expla-nations that can be viewed and reviewed at the learner’s own pace In a metaanalysis of research on the use of multimedia to teach a variety of subjects, Ragan, Boyce, Redwine, Savenye, and McMichael (1993) found that, in general, multimedia instruction reduces learning time by 30% compared to traditional instruction They further demonstrated that fea-tures of multimedia instruction such as learner interactivity and learner control over programs produce improved outcomes in achievement Results of studies on computer assisted instruction have been generally positive For over two decades, researchers have examined the effective-ness of computer assisted instruction to teach many different subjects In

1987, Kulik and Kulik published a ground-breaking metaanalysis of com-parative studies on computer assisted and classroom instruction in which they calculated an overall effect size of 31 Other studies by Niemiec and Walberg (1987) and McNeil and Nelson (1991) uncovered similar results

In spite of the abundance of comparative research on computer assisted instruction in other academic fields such as reading (Rachal, 1995) and the growing body of research on methods of teaching grammar, only re-cently have researchers investigated the use of computer-based L2 gram-mar instruction (McEnery, Baker & Wilson, 1995; Nagata, 1996) The results of these studies seem to indicate that computer-based grammar instruction can be as effective or more effective than more traditional in-struction (e.g., workbooks and lectures)

If subsequent research bears out this initial indication of the benefits of computer-based grammar instruction, this kind of instruction could be-come an important complement to communicatively oriented language classes By using the computer for the presentation, explanation, and ap-plication of grammatical structures, more classroom time could be dedi-cated to real communication that focuses on expressing meaning and us-ing appropriate grammatical structures to express that meanus-ing Shiftus-ing the systematic study of grammar points from the realm of the classroom

to the domain of the computer laboratory would enable instructors to take advantage of classroom interpersonal dynamics and allow them to take into account differences in background knowledge and learning styles The use of computer-based grammar instruction would also support more individualized instruction in programs that have open-entry/open-exit enrollment such as adult and vocational education courses Students would not enter the class in the middle of a linear instructional sequence based on a grammatical syllabus Instead, the syllabus could be based on themes of interest and relevance to students, and individual students could follow a grammatical syllabus in computer-based instruction outside the classroom

PURPOSE OF THIS STUDY

This study examined whether computer-based grammar instruction is

as effective as teacher-directed grammar instruction for postsecondary stu-dents at multiple levels of proficiency in an intensive ESL program Two experiments were conducted at different levels of English language profi-ciency, one at the first and second level of instruction and the other at the third and fourth level The primary research question posed in the study was whether there was a difference in the acquisition of a specific gram-mar point for students taught in a teacher-directed class versus those taught

in computer-based instruction

The first experiment compared the performance of one and level-two students in a computer-based group versus those in a teacher-directed group The second experiment compared the performance of level-three and level-four students in a computer-based group versus those in a teacher-directed group The purpose of conducting the two experiments was to examine the acquisition of discrete structures at different levels of profi-ciency in order to increase the generalizability of the results

Sample

The population of the study consisted of 53 students (24 females and 29 males) enrolled in an intensive academic ESL institute at a major univer-sity in Florida The ESL institute offers four levels of instruction and uses the Comprehensive English Language Test (CELT) to place in-coming stu-dents Ten students were enrolled in level one, 9 in level two, 20 in level three (in two sections), and 14 in level four Students were matched for native language (Japanese, Thai, Korean, Arabic, Spanish, Italian, French, and Portuguese) and pretest scores on the structures in question and then randomly assigned to computer-based or teacher-directed sections Al-though the groups were not matched for gender, the breakdown was fairly even, each group differing in gender makeup by no more than two stu-dents

Independent Variable

The independent variable was the method of grammar instruction, ei-ther teacher-directed or computer-based Because the participants in each experiment consisted of two levels of English proficiency (first and sec-ond levels combined, third and fourth levels combined), grammar points were introduced at the lower level of the combined groups and then re-viewed and expanded at the higher level This cycle held true for the teacher-directed as well as computer-based groups The treatment consisted of one hour of instruction per day for seven days Because studies have indi-cated that certain grammar points are more easily acquired than others (Green & Hecht, 1992; Scott & Randall, 1992; Krashen, 1981), this study examined the acquisition of verb tenses, which included elements of as-pect, one of the more difficult forms for ESL students to master (Green & Hecht, 1992)

Teacher-Directed Group

Students in the teacher-directed groups were taught by five different teachers of varying degrees of experience Classes were held in rooms with-out computer equipment or other instructional technology Class size ranged

from four to seven students Students at all levels used the Focus on

Gram-mar (1994) textbook series and engaged in a variety of types of activities

emphasizing interactive, meaningful, and creative expression

Computer-Based Groups

Students in the computer-based groups received computer-based instruc-tion outside the classroom The students who participated in Experiment One used ELLIS Middle Mastery (1996), and the students who partici-pated in Experiment Two used ELLIS Senior Mastery (1996) ELLIS was selected because of its multimedia delivery (audio and video, recording capability, etc.), its modeling of natural and contextualized language, its interactivity, and its clear grammar explanations and practice activities Although ELLIS lacks a mechanism for sophisticated learner feedback (see Nagata & Swisher, 1995), its exercises do provide simple corrections Because the ELLIS program is organized around communicative needs rather than grammatical structures, students followed a sequential check-list developed by the researcher to direct their navigation through the pro-gram (See the sample navigational guide in Appendix A.)

Dependent Variables

In Experiment One, the grammar structure of interest was the past tense, and in Experiment Two, the conditional tense The dependent variables were students’ achievement scores on three separate criterion-referenced tests over the selected structures The tests on the past tense included items covering the simple past in regular and irregular forms, the past continuous, and the present perfect (regular form) The tests on the con-ditional tense included items covering the factual, unreal, and unlikely conditional

The three tests consisted of (1) a discrete-point multiple-choice test, (2)

a fill-in-the-blank test, and (3) an open-ended test The researcher, to-gether with the ESL institute’s curriculum specialist and the grammar teach-ers, decided which grammar points were appropriate to teach and assess The researcher then developed test instruments based on the content of the instruction in the teacher-directed and computer-based groups The students’ scores on the three tests were retained as separate

mea-surements, not combined into a single dependent variable, since there is a theoretical basis for considering them as measures of different constructs For example, multiple choice tests do not measure linguistic production (Hughes, 1989) and therefore can be appropriately used to assess stu-dents’ knowledge of grammar within the framework of grammatical con-sciousness-raising (Sharwood-Smith, 1981) Since open-ended tests mea-sure students’ linguistic production, they represent more demanding tasks

of application of knowledge than do multiple choice tests The same bat-tery of tests were used for pretests, immediate posttests, and delayed posttests

A pool of test items was field tested at two adult education schools From the initial item pool, ten items were selected for the fill-in-the-blank tests (scale of zero to five), ten items for the multiple choice tests, and five items for the open-ended tests (scale of zero to one) Teachers at both adult education schools, as well as the grammar teachers at the institute, examined the tests for content validity and made suggestions for changes that were eventually incorporated into the final version of the tests

In addition to analyzing students’ performance as represented by their values for the three dependent variables, the researcher conducted native language interviews with students in the computer-based group of Experi-ment One and administered questionnaires to students in the computer-based group of Experiment Two The purpose of these interviews and questionnaires was to solicit students’ views of the computer program and computer-based learning

Data Collection and Analysis

The pretests were administered three days prior to the beginning of the treatment, the immediate posttests were administered on the last day of the treatment, and the delayed posttests were administered two weeks after the posttests After having been trained, the classroom teachers ad-ministered the pretests and delayed posttests to all students and the im-mediate posttests to the students in the teacher-directed groups The re-searcher administered the immediate posttests to the students in the com-puter-based group The researcher scored all the tests blindly with no knowledge of students’ identity or group status After the researcher scored the tests, a certified ESL teacher reviewed students’ answers For all po-tentially problematic answers and scores, the researcher and the teacher reached consensus on the basis of the scoring sheet (See sample ques-tions and scoring criteria in Appendix B.)

For the pretests, immediate posttests, and delayed posttests in Experi-ment One, the calculation of internal consistency was 90 for the fill-in-the-blank test, 58 for the multiple choice test, and 63 for the open-ended

test For the pretests, immediate posttests, and delayed posttests in Ex-periment Two, the calculation of internal consistency was 67 for the fill-in-the-blank test, 40 for the multiple choice test, and 43 for the open-ended test Analysis of Covariance (ANCOVA) was used to analyze the data Because of the small sample size, the alpha level was set at 10 for tests of significance

Results

As in most comparative CALL studies, it was difficult to distinguish which features of the computer-based instruction led to the outcomes found

in the study and whether these factors were inherent to computer-based instruction or simply an instructional strategy that could have been em-ployed with a variety of media under different circumstances (see Will-iams & Brown, 1991) Nonetheless, the results showed that computer-based grammar instruction is at least as effective as, and in some cases more effective than, teacher-directed grammar instruction

Experiment One

Table 1 summarizes the results of the descriptive analysis of students’ test scores in Experiment One

Table 1

Mean Test Scores by Group in Experiment One

Type Time Computer-Based Group Teacher-Directed Group

Posttest

Posttest

Posttest

Posttest

Posttest

Posttest

Analysis revealed a significant difference between the test scores of the students in the computer-based and those in the teacher-directed groups

on the open-ended immediate posttest (p < 10) The computer-based group’s mean score on this test was 3.10, and the teacher-directed group’s, 1.20 Using adjusted means, the effect size for the difference was 1.56 Analysis revealed a similar significant difference between the computer-based and the teacher-directed groups on the open-ended delayed posttest (p < 10) The computer-based group’s mean score was 3.44, and the teacher-directed group’s, 1.80 Again using adjusted means, the effect size for this difference was 1.73 Finally, analysis also showed a significant difference between the computer-based and teacher-directed groups’ mean scores on the fill-in-the-blank immediate posttest, 31.11 versus 14.90 (p

< 05) This difference disappeared, however, by the time of the delayed posttest No significant differences were found between the groups for the multiple choice immediate posttest or delayed posttest

Experiment Two

Table 2 summarizes the results of the descriptive analysis of students’ test scores in Experiment Two

Table 2

Mean Test Scores by Group in Experiment Two

Type Time Computer-Based Group Teacher-Directed Group

Posttest

Posttest

Posttest

Posttest

Posttest

Posttest

Analysis showed a significant difference between the computer-based and the teacher-directed groups’ mean scores on the open-ended delayed posttest The computer-based group’s mean score was 4.79, and the teacher-directed group’s, 4.00 The effect size, once more using adjusted means, was 1.19 No significant differences were found on any of the other posttests

Interviews and Questionnaires

Insights into students’ experience with computer-based instruction emerged from the student interviews and questionnaires By and large, students were satisfied with the computer-based instruction and expressed

a desire to spend more time per day using it Students indicated that the features of computer-based instruction that were most useful were the computer’s capacity that allowed them to review the tutorial as many times

as they wished, to proceed at their own learning pace, to record their voices and compare them against the model, and to get immediate feed-back on the exercises The Asian students especially appreciated not being

“singled out” to speak in class, while some of the Latin students indicated that they would have preferred more human interaction

Discussion

Although the sample size of this study was too small to draw definitive conclusions, the study does present evidence of meaningful differences in the computer-based and teacher-directed groups’ achievement scores on the open-ended tests If open-ended tests measure students’ ability to use grammatical structures creatively, it would seem that some elements of computer-based study support the development of this skill more effec-tively than traditional classroom instruction

Surprisingly, the scores of the students in the computer-based groups rose on the open-ended delayed posttests This result contrasts with those

of other studies of classroom grammar instruction in which students’ pro-duction of grammatical structures was accurate on immediate posttests but fell on the delayed posttests Perhaps the interim period between the immediate posttest and the delayed posttest described here allowed the students in the computer-based group time to apply the structure in ques-tion in real communicative situaques-tions, providing them with an addiques-tional opportunity to negotiate meaning and monitor their own output This find-ing may indicate, as Ellis (1993) has suggested, that the use of computer-based grammar instruction can complement individualized structural syl-labi in communicative classrooms and more effectively enable students to

use the newly acquired structures to negotiate meaning.

The potential of the computer laboratory to go beyond providing simple practice and reinforcement of grammar points taught in the classroom is only beginning to be explored Research should be conducted with differ-ent populations (e.g., elemdiffer-entary and secondary studdiffer-ents) and differdiffer-ent types of courses (e.g., Vocational ESL and English for Specific Purposes) Moreover, additional research is needed to ascertain which features of computer-based grammar programs promote the acquisition of L2 struc-tures (e.g., degree of learner control, feedback strategies, etc.) As the body of knowledge on the application of computer-based grammar in-struction increases, so will the flexibility and number of options available

to teachers and students