Testing the Improvement of English as a Foreign Language Instruction Among Chinese College Students Through Computerized Graphic Visuals

The mixed measures design consisted of test item graphic-related and graphic-unrelated and test time immediate and delayed posttests as the within-subjects variables and the five treatme

BRIEF REPORTS AND SUMMARIES

TESOL Quarterly invites readers to submit short reports and updates on their work These summaries may address any areas of interest to Quarterly readers.

Edited by ALI SHEHADEH

United Arab Emirates University

ANNE BURNS

Macquarie University

Testing the Improvement of English as a Foreign Language Instruction Among Chinese College

Students Through Computerized Graphic Visuals

QUAN YANG

Troy University

Dothan, Alabama, United States

MICHAEL MILLER

University of Arkansas

Fayetteville, Arkansas, United States

KANG BAI

Troy University

Dothan, Alabama, United States

doi: 10.5054/tq.2011.244021

&Advances in technology have impacted virtually every aspect of society around the world Technological access has produced affordable, real-time communications in nearly every country, and technology has become a major element in all facets of communication The use of technology to foster easier and faster communications has also become a conduit for material to be shared among teachers and faculty members

in different societies, and at least some element of educational technology has become standard in nearly all learning environments For nearly 40 years there has been agreement that computer-generated graphics can impact learning (Paivio, 1971) by allowing the dual coding of material by the learner to strengthen and deepen memory, recall, and cognition The process of verbal recall has been noted to be enhanced by pictorial exposure, allowing the connection

and interplay between different parts of the brain, thus creating better memory and learning (Paivio & Csapo, 1973) This theory is further supported by the findings of Canning-Wilson (2001), who examined the influence of pictures on learning and testing practices Canning-Wilson found that a learner’s relation to language and ability to create connections between words could be enhanced by using graphic visuals Kim and Gilman (2008) conducted a similar study and concluded that

an effective way to improve learning of English vocabulary was to use graphics to illustrate what vocabulary words mean

Although the use of visuals to reinforce verbal learning has been a consistent finding (Molitor, Ballstaedt, & Mandl, 1989), only with the emergence of computer technology has the visual element become more sophisticated, three-dimensional, and often, fluid and on-demand from the learner Lin and Chen (2007) studied the effects of different types of computer-generated visuals on second language acquisition and found that learners who had exposure to animation outperformed those with no visuals In particular, the instruction with embedded animation had a marginal effect among groups in facilitating the acquisition of the second language reading comprehension Matsuda and Shindo (2006) examined the education system using interactive 3D computer graphic animation and concluded that on teaching the theory, principle, or mechanism of science, the most effective teaching material may be an actual video Poohkay and Szabo (1995) explored student learning in math when instruction included animation and still graphics, and found that animated learning environments produced significantly higher test scores in math as compared to those lessons taught using text or still graphics This concept has been validated with different age groups and

in different disciplines (Rakes, 1996; Chen, 1997; Weiss, 1999b) Graphic visual augmentation of instruction has typically fallen into four categories: static visuals, animated graphics, concrete visuals, and abstract visuals (Weiss, 1999b) The use of any of the categories of instruction has typically been found to produce better learning than text alone, but they have also been found to support each other in creating combinations of visual supports for learning Smith and Smith (1991) compared the effects

of concrete and abstract visuals on students’ acquisition of abstract concepts, and found abstract visuals were more effective in student learning but that when used alone, either abstract or concrete visual illustrations might not be as powerful Graphics in instruction are typically used in elementary and secondary environments, and have only recently begun to grow more rapidly in their use on college campuses Similarly, little research has been conducted from a comparative standpoint

In Asian cultures where learning environments tend to be formal and instructor centered, the use of visual graphics as an instructional supplement has grown, but is largely undocumented and unexplored

Therefore, the purpose for conducting this study was to determine if animated illustrations would increase the recall and comprehension of a subject matter, English as a foreign language (EFL), among Chinese college students The study was specifically designed to identify the influence of graphics on Chinese EFL students’ learning by comparing abstract versus concrete graphics and static versus animated graphics on immediate and long-term retention of a lesson through computer-based instruction The study is significant because the use of visual graphic–supplemented instruction may be demonstrated to positively impact language instruction, particularly for nonnative-English-language students and the result in the development of innovative strategies for improved EFL/ESL curriculum

METHODOLOGY

The sample for the study included second-year college students in the departments of Chinese language and literature, political sciences, and history at a 4-year comprehensive university in China One hundred and seventy (170) students were randomly selected to participate in the study The students selected had all passed the Level 3 National English Proficiency Test and taken the course College Computer Basic Skills, required of all second-year students at the university The study subsequently accepted the assumption that participants had about the same level in both English and computer basic skills

The study was a 2 6 2 factorial design with an outside control group

to yield five main treatment conditions as the independent variables for the study Two types of graphics (abstract vs concrete) were crossed with two levels of animation (animated vs static), producing four treatment groups of graphic animation, namely, abstract–static, concrete–static, abstract–animated, and concrete–animated The outside control group received a treatment of text-only with no graphics or animation In addition, in order to further identify the effects of different types of graphics, students in each treatment group were again divided into two subgroups One subgroup took the Same Image test, a test with graphics directly from the text, and the other subgroup took the Different Image test, which used graphics different from those that appeared in the text The 170 students selected for the study were randomly assigned to five, 34-student treatment groups Each group was presented with one of the five versions of the language lesson, and 154 students completed both the instruction and posttests (see Table 1) Those who did not complete the treatments were eliminated from the study

The study used immediate and delayed posttests to measure how well students learned a lesson and how long they could remember the content A computer-based instruction (CBI) unit, ‘‘Basic Principles of

Laws of Motion’’ designed by Weiss (1999a), was used for the study, which explained principles of motion and Isaac Newton’s three laws of motion Immediately after the students watched the CBI lesson, they were given the first posttest, and a week later, they were administered the second, or delayed, posttest With the same questions presented in different order, the immediate and delayed posttests each contained 24 items, including 10 graphic-unrelated items and 14 graphic-related items Graphic-related questions used graphics to illustrate Newton’s laws of motion, while graphic-unrelated items were verbal descriptions of Newton’s laws of motion All the questions were designed to measure the students’ understanding of the concepts related to the lesson they learned The test scores of the students were used as the dependent variable for the study

In the data analysis, a three-way between-groups ANOVA was conducted on test scores The one-sample Kolmogorov-Smirnov test and Levene’s test of equality of error variance were also performed to verify the normality and equality of variance of the score data for the analysis of variance

Additionally, a three-way mixed ANOVA was conducted to explore the performance differences between unrelated items and graphic-related items The mixed measures design consisted of test item (graphic-related and graphic-unrelated) and test time (immediate and delayed posttests) as the within-subjects variables and the five treatment groups as the between-subjects variable Follow-up analyses of significant effects were conducted to assess differences between means

FINDINGS

As stated previously, a total of 154 EFL students were included in the data analysis These students represented five randomly composed groups and took both the immediate and delayed posttests The overall scores for the posttests (range 5 0 to 24) were calculated by summing up the correct answers from the multiple-choice questions The means and standard deviations of the test scores of the five treatment groups in immediate and delayed posttests are displayed in Table 2

TABLE 1

Summary of Treatments

Level of animation

Type of graphic Abstract Concrete Static Abstract–static (n 5 30) Concrete–static (n 5 31) Animated Abstract–animated (n 5 32) Concrete–animated (n 5 30) Text-only n531

As shown in Table 2, the overall means for the five treatments ranged from 11.23 to 14.30 on the immediate posttest and from 10.26 to 14.03

on the delayed posttest On both posttests, with either same image or different image test items, students in the concrete–animated group scored the highest, and those in the text-only group scored the second highest of all the five treatment groups Conversely, the results of the descriptive analysis with the other three graphic-related groups were mixed, with the concrete–static group apparently performing the lowest

on both posttests In most cases, the mean scores were slightly higher on the immediate posttest than on the delayed posttest, except for the text-only and abstract–animated groups Additionally, little difference existed between same image items and different image items, indicating that the relation to the graphics in the text did not seem to have much impact on the performance of the students in the tests

Results of Three-Way Between-Groups ANOVA on the Overall Scores

For the three-way ANOVA, the one-sample Kolmogorov-Smirnov test did not detect any violation of normality with the test scores, and the Levene’s test of equality of error variances failed to reject the null hypothesis that the error variance of the dependent variable was equal across groups (F 5 0.981, p 5 0.484) Table 3 displays the results of the

TABLE 2

Means and Standard Deviations of Test Scores

Treatment

Immediate posttest Delayed posttest Same

image

Different image Overall

Same image

Different image Overall

n M (SD) n M (SD) n M (SD) M (SD) M (SD) M (SD) Text-only 16 13.00

(3.16)

15 12.00 (3.68)

31 12.52 (3.40)

12.50 (3.20)

12.53 (4.78)

12.52 (3.97) Concrete–

static

15 11.73

(3.41)

16 10.75 (2.93)

31 11.23 (3.16)

9.13 (2.95)

11.31 (3.52)

10.26 (3.39) Concrete–

animated

15 14.26

(2.68)

15 14.33 (2.74)

30 14.30 (2.67)

14.13 (4.34)

13.93 (2.60)

14.03 (3.52) Abstract–

static

15 11.73

(2.46)

15 11.06 (2.71)

30 11.40 (2.57)

11.47 (2.45)

10.80 (3.32)

11.13 (2.89) Abstract–

animated

15 10.67

(3.11)

17 11.76 (3.47)

32 11.25 (3.30)

11.20 (3.26)

11.65 (3.33)

11.44 (3.25) Overall 76 12.29

(3.16)

78 11.96 (3.30)

154 12.12 (3.23)

11.70 (3.60)

12.03 (3.65)

11.86 (3.62) Note: Overall scores were measured in a range from 0 to 24 Same image test 5 test items directly related to the graphics in the text; Different image test 5 test items indirectly related to graphics in the text.

three-way ANOVA on the overall scores for the treatments, test times, and test types

As shown in Table 3, there was a significant main effect for the five treatment groups (F 5 10.8, p , 0.0001) and no other main effects or interactions were found statistically different This information demon-strates that the students of the five treatment groups performed differently on the test, but there was no difference based on test time, test type, or other interaction terms A further examination using the post hoc test for least significant difference (LSD) was performed, and the results (Table 4) indicated that the mean scores of the concrete– animated group were significantly higher than those of all other four treatment groups, and the text-only group, which performed the second best, was significantly superior to the three remaining treatments, which performed about the same on the tests

As can be seen in Table 4, even though the test results supported the position that the concrete–animated condition would outperform all other groups, as reported in Weiss’ (1999b) study, the finding was

TABLE 3

Three-Way Between-Groups ANOVA

Source

Sum of squares

Degrees of freedom

Mean square F

Signif-icance Treatment 458.92 4 114.73 10.80 0.000

Treatment by test time 13.59 4 3.40 0.32 0.865 Treatment by test type 25.27 4 6.32 0.60 0.667 Test Time by test type 8.26 1 8.26 0.78 0.379 Treatment by test time by

test type

36.48 4 9.12 0.86 0.489

TABLE 4

Multiple Comparisons Between the Treatment Groups

Treatment Mean difference Standard error Significance Concrete–animated vs text-only 1.65 0.59 0.006 Concrete–animated vs concrete–static 3.42 0.59 0.000 Concrete–animated vs abstract–static 2.90 0.59 0.000 Concrete–animated vs abstract–animated 2.82 0.59 0.000 Text-only vs concrete–static 1.77 0.59 0.003 Text-only vs abstract–static 1.25 0.59 0.035 Text-only vs abstract–animated 1.17 0.59 0.044 Concrete–static vs abstract–static 20.52 0.59 0.375 Concrete–static vs abstract–animated 20.60 0.58 0.301 Abstract–static vs abstract–animated 20.07 0.59 0.895 Note: The mean difference is significant at the 0.05 level.

unexpected that the text-only group performed significantly better than three other graphic treatment conditions This finding was inconsistent with other research that has demonstrated opposing findings

Results of ANOVA on Scores of Related and Graphic-Unrelated Items

In this study, a three-way mixed ANOVA design was used to identify any possible significant differences among the five treatment groups between graphic-related and graphic-unrelated items on immediate and delayed posttests In the design, test time (immediate and delayed posttests) and test item (graphic-related and graphic-unrelated items) were the within-subjects variables, and the five treatment groups were the between-subjects variable

Table 5 illustrates that, with the graphic-related items, students in the concrete–animated and text-only groups performed better than the other three groups on both the immediate and delayed posttests With graphic-unrelated items, however, the concrete–animated groups performed better on both posttests, while the other four groups were similar to one another, and the abstracted–animated and text-only scores were only slightly higher on immediate posttest and the delayed posttest, respectively In addition, the mean scores in the graphic-related items of all five groups were apparently higher than those in the graphic-unrelated items

The results of the three-way mixed ANOVA presented in Table 6 indicate that there was a significant interaction between test type and treatment (F 5 2.551, p 5 0.046), indicating that treatment effects on

TABLE 5

Mean Test Scores of the Five Treatment Groups

Treatment n

Immediate posttest Delayed posttest

Graphic-unrelated

Graphic-related

Graphic-unrelated

Graphic-related

Text-only 31 38.7 9.9 61.6 20.0 43.9 14.1 58.1 22.0 Concrete–static 31 41.0 14.5 51.6 17.3 33.2 11.7 49.4 19.3 Concrete–

animated

30 45.3 12.2 70.0 18.6 45.7 13.8 67.0 21.0 Abstract–static 30 37.3 9.8 54.7 15.7 37.0 14.7 52.0 17.9 Abstract–

animated

32 43.8 14.1 50.3 17.7 37.8 14.8 55.0 20.6 Overall 154 41.2 12.5 57.5 19.1 39.5 14.4 56.2 20.9 Note: The measures of graphic-unrelated and graphic-related items were based on the percentage of the correct answers to the total The possible range was 0–100.

related items differed significantly from those on the graphic-unrelated items

Further analysis with the post hoc LSD test was performed on the treatment effect on item type (graphic-related and graphic-unrelated items) and found that, with graphic-related items, the group with concrete–animated treatment outperformed all other groups and the text-only group scored the second highest on the test (see Table 7) Conversely, with graphic-unrelated items, although the concrete– animated group still performed significantly better than the other three graphic-related groups, this group scored no better than the text-only group (see Table 8) An important finding is that when the test items were not related to or supported by graphics or when only verbal descriptions of the test items were provided, the graphic-related groups,

TABLE 6

Three-Way Mixed ANOVA

Source

Sum of squares

Degrees of freedom

Mean square F

Signif-icance Within

Test time by treatment 5.70 4 1.43 0.94 0.444 Error (test time) 226.47 149 1.52

Item type 423.88 1 423.88 181.33 0.000 Item type by treatment 23.86 4 5.96 2.55 0.046 Error (item type) 348.31 149 2.34

Test time by item type 0.05 1 0.05 0.02 0.877 Test time by item type by

treatment

18.40 4 4.60 2.40 0.053 Error (test time by item

type)

286.27 149 1.92 Between

Treatment 136.14 4 34.03 6.87 0.000

TABLE 7

Multiple Comparisons Between Treatment Groups in Graphic-Related Items

Treatment Mean difference Standard error Significance Concrete–animated vs text-only 8.7 3.45 0.012 Concrete–animated vs concrete–static 18.0 3.45 0.000 Concrete–animated vs abstract–static 15.2 3.47 0.000 Concrete–animated vs abstract–animated 15.8 3.42 0.000 Text-only vs concrete–static 9.4 3.42 0.007 Text-only vs abstract–static 6.5 3.45 0.060 Text-only vs abstract–animated 7.2 3.39 0.035 Concrete–static vs abstract–static 22.8 3.45 0.409 Concrete–static vs abstract–animated 22.2 3.39 0.522 Abstract–static vs abstract–animated 0.7 3.42 0.843 Note: The mean difference is significant at the 0.05 level.

regardless of the concrete-animated group, performed as well in the test

as the text-only group

DISCUSSION

With the subjects being EFL students learning the content area in a non-English-speaking environment, the study reported significant findings, showing both consistencies and inconsistencies with the findings of previous research in which the subjects were native English speakers The results of the three-way between-groups ANOVA revealed significant differences in performance between the five Chinese student treatment groups, indicating that concrete animation had the biggest effects on the successful performance of the students However, when the CBI lesson was supported by graphics other than concrete animation, the text-only group performed significantly better on both the immediate and delayed posttests The significant interaction between treatment and test item identified by the three-way mixed ANOVA further indicated that treatment conditions had significant effects on the performance of the students when test items or questions were related to or supported by graphics However, when only verbal descriptions of the test items or questions were provided without graphics, the concrete–static, abstract–static, and abstract–animated groups performed the same as the text-only group

The achievement of Chinese students receiving concrete–animated graphics could be explained by the following three factors First, a concrete and animated picture, with its motion and realism, could draw attention more easily than abstract–animated or static or concrete–static graphics Because concrete–animated pictures can present different states of a subject matter, they can provide more information to a learner and require more mental processing than a static picture, which will

TABLE 8

Multiple Comparisons Between Treatment Groups in Graphic-Unrelated Items

Treatment Mean difference Standard error Significance Concrete–animated vs text-only 4.2 2.4 0.077 Concrete–animated vs concrete–static 8.4 2.4 0.000 Concrete–animated vs abstract–static 8.3 2.4 0.001 Concrete–animated vs abstract–animated 4.7 2.4 0.046 Text-only vs concrete–static 4.2 2.4 0.075 Text-only vs abstract–static 4.1 2.4 0.083 Text-only vs abstract–animated 0.5 2.3 0.827 Concrete–static vs abstract–static 20.1 2.4 0.976 Concrete–static vs abstract–animated 23.7 2.3 0.115 Abstract–static vs abstract–animated 23.6 2.4 0.125 Note: The mean difference is significant at the 05 level.

make the students think actively and help them internalize what they have learned In addition, concrete–animated pictures can reduce the cognitive load of knowledge acquisition, as they provide external support for mental simulations (Schnotz & Grzondziel, 1996) These features of concrete–animated graphics are helpful to EFL students who are learning a content area in English and in an environment where English is not spoken In such a learning environment, concrete– animated pictures can arouse the interest of the students in the subject matter and draw their attention to what they are required to learn This attention-gaining quality is significant and will favor the process of selective perception, which chooses only a small part of incoming stimuli

to be given consideration and retained in memory (Anderson, 1980) Second, the way in which information received through instruction is encoded may also contribute to the achievement by the concrete-animated group of the Chinese students According to Paivio and Csapo (1973), dual coding—the use of both a verbal and a nonverbal memory code—may be more likely to occur when the context of the instruction lends itself to imaging Concrete–animated pictures, due to their vividness and realism, could be effective in helping the Chinese students develop such dual coding of the information they received through the computer-based instruction Consequently, it could be anticipated that concrete–animated graphics would be retained better than the other means of graphical representation Finally, animation may have helped

to reduce the level of abstraction in the theoretical concepts of Newton’s laws of motion (Weiss, 1999b) In a subject involving motion, animation relieves the short-term memory of visualizing text or projecting static arrows into motion and, consequently, results in the memory being more focused on the concept being taught without the distraction of creating self-made mental images (Schnotz & Grzondziel, 1996) This reduction on the demands of the short-term memory may produce more accurate and successful encoding in the long-term memory

An unexpected finding was that the text-only group scored the second best of all treatment groups on both the immediate and delayed posttests as well as on the graphic-related test The finding may well be attributable to the learned cultural differences in Chinese society, where rote learning is a traditional model of instruction, suggesting that students have created a habit of learning, or rather, a style of learning in which they have become accustomed, and that the text-only learning process was demonstrated to be reinforced This suggests a significant cultural transition that may be moving into the Chinese culture, where media influence and access has grown dramatically As media, including graphic representation and technology access, have become more common teaching tools, the Chinese educational system may begin to forecast challenges to students who have been acculturated to one