Comment on Comment: The Effectiveness of Student Annotations and Teacher Feedback in L2 Business Writing

This paper examines the effects of improvement in writing quality on two significant areas in L2 business writing: (1) the short term writing goals and the long term writing goals and (2) the degree of involvement in the student annotations and the teacher feedback. Student annotation training and three businessletter writingtasks were completed over an intensive 5week period. All the participants were college undergraduates enrolled in a prerequisite course for business writing at the College of Business in Korea. The degree of involvement in the Student Annotations, Teacher Feedback, Comment on Comment, and Reflection was found pivotal in the short term writing goals versus the long term writing goals: (1) the lowerlevel proficiency writers benefit from the greater degree of student involvement and higher improvement in the writing quality, (2) the degree of student involvement elicits higher frequency in teacher feedback, (3) greater degree of student involvement elicits higher frequency in the student annotations, and (4) the greater the involvement, perceptions varied between the types of student annotations and the types of teacher feedback. The results of this study indicate salience in L2 business writing research in ESP.

Student Annotations and Teacher Feedback

in L2 Business Writing

Jiyon Lee (Ewha Womans University)

Lee, Jiyon 2011 Comment on Comment: The Effectiveness of Student Annotations and Teacher Feedback in L2 Business

Writing Korean Journal of English Language and Linguistics, 11-3,

547-575 This paper examines the effects of improvement in writing quality on two significant areas in L2 business writing: (1) the short term writing goals and the long term writing goals and (2) the degree of involvement in the student annotations and the teacher feedback Student annotation training and three business-letter writing-tasks were completed over an intensive 5-week period All the participants were college undergraduates enrolled in a prerequisite course for business writing at the College of Business

in Korea The degree of involvement in the Student Annotations, Teacher Feedback, Comment on Comment, and Reflection was found pivotal in the short term writing goals versus the long term writing goals: (1) the lower-level proficiency writers benefit from the greater degree of student involvement and higher improvement

in the writing quality, (2) the degree of student involvement elicits higher frequency in teacher feedback, (3) greater degree of student involvement elicits higher frequency in the student annotations, and (4) the greater the involvement, perceptions varied between the types of student annotations and the types of teacher feedback The results of this study indicate salience in L2 business writing research in ESP

Key Words: student annotations, teacher feedback, degree of

involvement, comment on comment, L2 business writing

"independent-writers" (Brown, 2001; Ellis & Johnson, 1994; Harmer, 2004; Hyland, 2002, 2003; Xiang, 2004) To instill the awareness in writers, numerous cases in L2 writing have discussed the variables describing the relationship among learner types, feedback types, and sociocultural aspects According to Casanave (2003), the researcher described the "sociopolitical" phenomenon of a L2 writing class from the author’s professional experience

Because the concept of language employment is commonly referred to as a "social activity" (Hyland, 2003; McNamara, 2001), writing not only requires schema to produce lines of text, but also the product of writing (Ellis & Johnson, 1994; Frendo, 2005; Hyland, 2003; Swales, 1990) needs to match the purpose in the particular social activity, which is known as genre-based writing (Swales, 1990; Tribble, 1996; Hyland, 2002, 2003) Commonly, writing a business letter consumes a large part of the working-day for business professionals, and writing a business letter engages the writer to accomplish effective written communication in content, organization, and accuracy (Frendo, 2005) that meets the demands of the "situation and context" (Dudley-Evans & St John, 1998)

This study, therefore, attempts to underscore the effects of student annotations and teacher feedback in L2 business writing quality Also, in this study, a quantitative analysis aims to correlate the degree of involvement (Donna, 2000; Ellis & Johnson, 1994; You, 2004) to the student annotation types and the teacher feedback types The student writers were required to

write persuasive business letters with additional involvement: varying forms of student annotations Any form that defines the degree of involvement―Student Annotations, Teacher Feedback, Comment on Comment, and Reflection―is investigated in this research Therefore, the purpose of this study is to bring extended insight into the effectiveness of student annotations and teacher feedback in L2 business writing

This study includes the following research questions:

1 What are the effects of the short term and the long term writing goals on the improvement of L2 business writing quality?

2 What are the effects of the degree of involvement in the student annotations and the teacher feedback on the improvement of L2 business writing quality?

2 Literature Review

The significant shift in the lingua franca, from French to English, has altered attitudes in teaching and learning English Notably, the English language has emerged undisputedly as the global communication medium By all means, English as the lingua franca represents communication that pervades "both within countries and internationally" (Kirkpatrick, 2007) The world language, therefore, has become the medium among people from different nations and across different disciplines The relationship can be seen further between the non-native speakers of English (NNS) and the international context The relationship of the non-native speaker in an international context therefore specifies the communicative purpose, which can be seen in English for Specific Purposes (Dudley-Evans & St John, 1998; Harding; 2007) Notably, identifying the root of the

communicative purpose draws attention to the teaching needs It leads to specially designed teaching―for instance, in L2 business writing (Ellis & Johnson, 1994; Frendo, 2005; Hyland, 2003)―to fit the needs of the learner in specific genres (Ellis & Johnson, 1994; Frendo, 2005; Hyland, 2003; Swales, 1990) Within the discourse communities, Swales explains that the preferred way of communication can differ; for example, writing is considered the predominant medium in the information exchange, especially for those office workers (Dudley-Evans & St John, 1998) Therefore, written business English and business communication has become unavoidable for L2 writers

In L2 writing, however, Connor and Kaplan explained how writing for the native speaker can possess attributes different from the non-native speaker (Connor & Kaplan, 1987) It is only natural for the non-native speaker to write with restrictions; differently however, flexibility is freely-present for the native-speaker to write, since the native-speaker has a larger knowledge-capacity of the English language Additionally, Grabe and Kaplan have actively shown that in contrastive rhetoric, writing practices of certain languages and cultures can produce writing that would be considered inappropriate for native-speaker situations; such as, patterns in organization, lexis, and linguistic features (as cited in Hyland, 2002) It is necessary

to recognize that certain features requisite in English writing may not exist in the L2 writer’s native language

To make sure L2 writing succeeds, the next juncture in hand is the significant role of the L2 writing instructor Kroll (2003) investigated the "issues in the acquisition of English-language writing skills." Tools to enhance writing improvement for teachers and students indicate new advances to bridge both writing skills and successful results in writing A possible explanation is given by Warschauer in that, for example, computer media may allow the learners to be in control over

their learning—instead of the learning controlling the learner (as cited in Kroll, 2003) Because all writing production begins and ends with the writer alone for both L1 and L2 writers, it is important that writing instructors attempt to develop autonomous writers; in essence, self-discovery writers (Brown, 2001; Ellis & Johnson, 1994; Harmer, 2004; Hyland, 2002, 2003; Xiang, 2004) The autonomous writers, therefore, would be able to make educated decisions about their own writing (i.e patterns in organization, lexis, and linguistic features).

To address the issue of teacher feedback (Bitchener et al., 2005; Bitchener & Knoch, 2009) and student revisions in L2 writing (Choi, 2010; Goldstein, 2004; Sach & Polio, 2007), effective and useful teacher feedback is considered as the primary scaffolding strategy (Frankenberg-Garcia, 1999; Hyland, 2003; Kabilan, 2007; Shin, 2003; Vygotsky, 1986) Scaffolding can bring profound effect in learning when a proficient individual assists a lower level learner to eventually reach independent improvement (Guerrero & Villamil, 2000; Vygotsky, 1986) To most students, in a setting similar to the Korean context, the teacher’s role becomes imperative in scaffolding, which later the learner-writer is able to better the particular learning or strategy

to its own independent ability In contrast, Truscott (1996) suggests the concern of undependable feedback Truscott addresses that the quality and the type of feedback may not be

in congruence with what the student writer needs; for example, feedback given "not on the accuracy of the language they use to convey their ideas" (as cited in Kroll, 2003) Despite the controversy over effective or ineffective feedback in grammar, Ferris (1999), however, gives her effort to support teacher feedback (as cited in Kroll, 2003), which brings positive effects in the student revisions, which is consistent with the studies in self-monitoring and student annotations (Brown, 2001; Ellis & Johnson, 1994; Harmer, 2004; Hyland, 2002, 2003; Suh, 2005; Xiang, 2004)

In addition, Schneider and Andre (2005) open the study with

a striking question all teachers and student inquirers would like to know: "How well do students think university prepares them for the writing they will do in their future work lives?" They also stress "the transfer of writing skills from university to the workplace." Similar to the genre-pedagogy (Hyland, 2007), local skills in writing do not merely reflect what is needed to complete a writing task at the workplace The quality of teacher feedback, the types of teacher feedback, and the tools to enhance improvement in L2 writing—such as, student annotations and self-monitoring assessments (Frendo, 2005; Suh, 2005; Todd et al., 2001; Xiang, 2004)

—may contribute to the confidence-building and appropriate L2 writing classroom (McNamara, 2001) Therefore, reacquainting the purposes of writing research and instruction—with the true human-writer involvement and the true environmental push—can reveal what writing instruction and writers require in the L2 writing classroom (Casanave, 2003; You, 2004)

Written Examination (TWE) on a scale from 0 (lowest) to 5 (highest).

3.2 Data Collection Procedure

A different writing prompt was given for each writing task: the pretest, the in-class writing task (one prompt for all three drafts), and the post-test The student annotation training and the overall three writing tasks were completed over an intensive 5-week period Each writing prompt asked the student writers to write a persuasive business letter adapted from a business English course book (Clark et al., 1994) The pretest and the post-test did not require any student annotations nor any teacher feedback For the pretest and the post-test, a limit of 30 minutes was given to complete the task The in-class writing task, however, was designed to measure the degree of involvement;

it included the Student Annotations (SA) and the Teacher Feedback (TF) in all the three drafts (see Figure 1) To speed the process of writing that includes the student annotations and to narrow the miscommunication in the student annotations due to any language barriers, all the students were trained and therefore required to compile a list of useful annotations from the adapted subcategories (Storch & Tapper, 1997) and from the adapted categories of response intentions (Johnson, 2000; Todd, et al., 2001,) The compiled list of student-derived student annotations was made into a packet that was ready for use during the in-class writing task

Importantly, to measure further the student-involvement, the participants in the experimental group received additional instruction to complete the Reflection (R1 and R2) and the Comment on Comment (CC1 and CC2) before draft two (D2) and draft three (D3) To measure further the teacher-involvement

as well, one additional teacher feedback (TF2*) in response to the

Experimental Control

Draft 1

D1

Draft 1 D1

Comment on Comment (CC1) was required before the teacher feedback (TF2) in draft two The Reflection (R1 and R2) was not statistically observed due to the small sample size The purpose

of the Reflection (R1 and R2), which was adapted from a feedback assessment form (Frendo, 2005), was to encourage student awareness (McNamara, 2001) in response to the teacher feedback (TF1 and TF2) from the previous draft (D1 and D2) The choices (☺, , and ) circled in the Reflection by the student writers gave a success indication of either a positive, neutral, or negative response to the teacher feedback (TF1 and TF2) that they received in the previous draft (D1 and D2) To sustain the awareness level and to encourage effective self-monitoring in the revision process, the student writers were then required to respond to the teacher feedback (TF1 and TF2) from the previous draft (D1 and D2) Considering the extra amount of work required in the in-class writing task that requires to write the student annotations, a limit of 60 minutes was given to complete the persuasive business letter (Caudery, 1990; Polio et al., 1998; Shin, 2011) Once the completed drafts were submitted, all the student annotations (SAs, Rs, and CCs) and all the teacher feedbacks were categorized and quantified according to the adapted subcategories and the adapted categories of response intentions

3.3 Data Analysis

The writing quality for all the participants (pretest, post-test, draft 1, draft 2, and draft 3) from both groups was holistically scored implementing the adapted Test of Written Examination (TWE); it was measured on a scale from 0 (lowest)

to 5 (highest) To measure the writing improvement, the means and standard deviations for D3-D1 (it does not include draft 2) and for Post-Pre (post-test and pretest) were calculated to determine the differences in the writing proficiency levels and

the two groups Also, a two-way ANOVA analysis was performed in the following three areas: (1) to observe the improvement in writing quality in the short term effect (D3-D1) and in the long term effect (Post-Pre) in comparison with the Level and the Group, (2) to investigate whether the interaction effect is present between the students' level and the group, and (3) to determine the differences in the degree of involvement for all the student annotations (SA), the student annotations and comments on comment (SACC), and the teacher feedbacks (TF) Next, the Pearson's chi-square test was used to investigate the differences in the frequency distribution of the Student Annotations and the Teacher Feedback between the two groups The t-test also was used to observe the statistical differences between the mean of the Student Annotation frequency and the Teacher Feedback frequency in each of the sub-categories used in the two groups The adapted sub-categories were used by both the teacher and the student writer in the student annotations, comments on comment, and the teacher feedback; however, the adapted categories of response intentions were not calculated in the present study since it was used differently between the teacher and the student writer

To determine the consistency in the coding of the Student Annotations (SA), Comment on Comment (CC), and Teacher Feedback (TF), the interrater reliability ranged from 86 to 94 All the types of Student Annotations, Comment on Comment, and Teacher Feedback were categorized into the adapted sub-categories and into the adapted categories of response intentions and were coded and quantified by the teacher of the course It was later cross-examined with another rater to reach

an agreement in the holistic scoring, in the coding, and in the counting of frequencies of the student annotation types and the teacher feedback types The rater holds a doctorate in English Education and is currently a college professor in English

Education In this study, the significant level was set at 05.

4 Results and Discussion

4.1 Writing Improvement: Short Term and Long Term

A six-point holistic scale was used to verify the level of writing competency in short term effects (D3-D1) and in long term effects (Post-Pre) in the two groups The ANOVA results,

as shown in Table 1, show a significant difference in each of the Level and the Group in the short term scores (D3-D1) The results imply that the dialogue using the additional student annotations and the effective teacher feedback may have been highly beneficial in the lower-level writing proficiency student

writers in the experimental group (Experimental-Low, M = 3.11; Control-Low, M = 91) As seen in Table 2, the Level and the

Group have a significant effect on the score D3-D1 below the 01

level (Level, p = 00; Group, p = 00) Also, there is a significant interaction effect between the Level and the Group on D3-D1 (p

= 00).

Table 1 Descriptive Statistics of Short Term Score Improvement (D3-D1)

by Level and Group

Note: D3-D1 does not include draft 2.

Source SS df MS F p

Table 2 Two-Way ANOVA Results of Short Term Score Improvement

(D3-D1) by Level and Group

Note: D3-D1 does not include draft 2.

Figure 2 Mean Differences of Short Term Score Improvement (D3-D1)

by Level and Group

Table 4 displays the ANOVA results and shows a significant difference between the Level in the long term scores (Post-Pre); however, no significant difference was noted between the Group in the long term scores (Post-Pre) The Level has a significant effect on

the long term scores (Post-Pre) below the 01 level (p = 00) The Group has no significant effect (p = 21) on the long term scores (Post-Pre) No interaction effect (p = 78) between the Level and the

Group was present in the long term scores (Post-Pre) The results are consistent with the above Table 1 It implies, again, that the lower

level writers (Experimental-Low, M = 2.31; Control-Low, M = 1.95)

may gain beneficial effects for long term writing goals, utilizing the extended practice of writing annotations and receiving useful teacher feedback

Source SS df MS F p

Table 3 Descriptive Statistics of Long Term Score Improvement

(Post-Pre) by Level and Group

Table 4 Two-Way ANOVA Results of Long Term Score Improvement

(Post-Pre) by Level and Group

Figure 3 Mean Differences of Long Term Score Improvement (Post-Pre)

by Level and Group

4.2 Degree of Involvement: Annotations and Feedback

To determine the differences in the degree of involvement for all the student annotations (SA), the student annotations and

comments on comment (SACC), and the teacher feedbacks (TF),

Table 5 Descriptive Statistics of Total Frequency of SACC

by Level and Group

Figure 4 Mean Differences of Total Frequency of SACC

by Level and Group

Table 6 Two-Way ANOVA Results of Total Frequency of SACC

by Level and Group