Using analytic rubric checklists to enhance third year students essay writing skills at university of technical education m a 60 14 10

Abstract This study was conducted to evaluate the effectiveness of using analytic rubric checklists for writing and peer correction activities to enhance writing skills for third-year in

VIETNAM NATIONAL UNIVERSITY – HCM CITY

UNIVERSITY OF SOCIAL SCIENCES & HUMANITIES

DEPARTMENT OF ENGLISH LINGUISTICS AND LITERATURE

Submitted in partial fulfillment of the requirements for the degree of

Master of Arts (T.E.S.O.L.)

Candidate: VÕ DUY MINH – TESOL 2005 Supervisor: CAO THỊ QUỲNH LOAN, M.A

- 2010 -

CERTIFICATE OF ORIGINALITY

I hereby certify my authorship of the thesis submitted today entitled:

USING ANALYTIC RUBRIC CHECKLISTS TO

ENHANCE THIRD-YEAR STUDENTS’

ESSAY WRITING SKILLS

AT TECHNICAL EDUCATION UNIVERSITY

In terms of statement of Requirements for Theses in Master’s Program

issued by Higher Degree Committee

Ho Chi Minh City, October 2010

VÕ DUY MINH

RETENTION AND USE OF THE THESIS

I, VÕ DUY MINH, hereby state that I being the candidate for the degree of Master in T.E.S.O.L accept the requirements of the University relating to the retention and use of Master’s theses deposited in the Library

In terms of these conditions, I agree that the original of my thesis deposited in the Library should be accessible for purposes of study and research, in

accordance with the normal conditions established by the Library for the care, loan or reproduction of the theses

Ho Chi Minh City, October 2010

VÕ DUY MINH

ACKNOWLEDGEMENT

First I would like to express my gratitude to my thesis supervisor, Ms Cao Thị Quỳnh Loan, who gave me special instructions, contributions and provided valuable supports in the preparation and completion of my thesis Also, I would like to thank all of my teachers from my university to master course for giving me my knowledge

Next I would like to send my special thanks to my wife, Ms Tăng Thị Yến Nga, for all that she has to stand to let me have enough time to focus on my learning For me to be able to go from my in-service university to post-graduate diploma and master course, she has to sacrifice a lot of things

I also would like to thank all of my friends as well as all of my students, who have come to me with questions and questions, with this discussion following another discussion I don’t know whether my answers help them or not but their questions contribute a great deal to my knowledge

I also would like to sincerely thank all of staff working for the University

of Social Sciences and Humanities in Ho Chi Minh city, Ms V Thị Hồng Mai,

Ms V Thị Nữ Anh, Ms Phan Thị Mai Trang, Ms Mai Thị Thy Vn in the Department of English Linguistics and Literature; Ms Thủy, Ms Điệp, Ms Thu

in Post Graduate Department, as well as Ms Tm, Ms Lin, Ms Hồng in Service Team, Mr Tm, Mr Minh, Mr Đẹp, Mr Phương in Maintenance Team and so

on …

This thesis is my success, however, these people, in any position, already contributed an important role to my success

I owe those

LIST OF CO-OPERATIVE TEACHER & SCORERS

I – CO-OPERATIVE TEACHER:

LÊ PHƯƠNG ANH : A candidate of T.E.S.O.L Master Degree at the

University of Social Sciences and Humanities of Ho Chi Minh city;

Teachers at the University of Technical Education

II – SCORERS:

1 PHAN THỊ LAN ANH : Master in T.E.S.O.L from the University of Social Sciences and Humanities of Ho Chi Minh city; Teachers at Nông Lâm University

2 NGUYỄN ÁI HOÀNG CHÂU : Master in T.E.S.O.L from the University

of Social Sciences and Humanities of Ho Chi Minh city; Teachers at Bạc Liêu University

3 LÊ THỊ THANH HÀ : Master in T.E.S.O.L from the University of Social Sciences and Humanities of Ho Chi Minh city; Teachers at the University

of Technical Education, Hồ Chí Minh city

4 BÙI THỊ CAO NGUYÊN : Master in T.E.S.O.L from the University of Cần Thơ; Teacher at Cao Lãnh city Highschool

5 TÔ THỊ THANH THẢO : Master in T.E.S.O.L from the University of Victoria; Teacher at Ho Chi Minh city Open University

Table of Content

Certificate of Originality

Retention and Use of the Thesis

Acknowledgement

List of Co-operative Teacher and Scorers i

Table of Content ii

List of Figures, Tables, and Graphs viii

Abstract x

Chapter 1 – INTRODUCTION 1

1.1 - Background of the Problems 1

1.2 - Statement of the Problems 7

1.3 - Purposes of the Study 7

1.4 - Research Questions 9

1.5 - Research Hypothesis 9

1.6 - Scope of the Study 9

1.7 - Definition of Terms 10

1.8 - Limitations of the Study 11

Chapter 2 – LITERATURE REVIEW 13

2.1 - Impacts of Differences in Cultures 13

2.1.1 – On Learning Second Language 13

2.1.2 – On Writing Second Language 14

2.2 - Holistic Scoring and Analytic Scoring 19

2.2.1 – Holistic Scoring Rubrics 20

2.2.1.1 – Holistic Scoring 20

2.2.1.2 – Holistic Scoring of some Tests 23

2.2.1.2.1 – Test of English as Foreign Language (TOEFL) 23

2.2.1.2.2 – Cambridge First Certificate in English (FCE) 23

2.2.1.2.3 – International English Language Testing System (IELTS) 24

2.2.1.3 – Disadvantages of Holistic Scoring 24

2.2.2 – Analytic Scoring Rubrics 28

2.2.2.1 – Analytic Scoring 28

2.2.2.2 – Disadvantages of Analytic Scoring 29

Chapter 3 – METHODS & PROCEDURES 33

3.1 - Research Design of the Study 33

3.2 - Participants in the Study 35

3.2.1 – Teachers & Scorers 35

3.2.1.1 – Teachers 35

3.2.2.2 – Scorers 35

3.2.2 – Students 36

3.2.2.1 – The control group 36

3.2.2.2 – The experimental group 36

3.3 - Variables in the Study 36

3.3.1 – Independent Variable 36

3.3.2 – Dependent Variables 37

3.3.3 – Controlled Variables 37

3.4 - Research Procedures and Pilot Testing 37

3.4.1 – Research Procedure 37

3.4.2 – Pilot Testing 39

3.5 - Instruments and Materials 40

3.5.1 – Instruments 40

3.5.1.1 – Questionnaire 40

3.5.1.2 – Students’ Feedback 41

3.5.1.3 – Scoring the Final Papers 42

3.5.1.4 – Critical  value and Tests 43

3.5.1.4.1 – Critical  value 43

3.5.1.4.2 – T-test 43

3.5.1.4.3 – Shapiro-Wilk test 43

3.5.1.4.4 – Percentage 44

3.5.1.4.5 – Correlation 44

3.5.1.4.6 – Quantile - Quantile Plots 44

3.5.1.4.7 – Box plots 44

3.5.2 – Materials 45

3.5.2.1 – Checklists and Instructions 45

3.5.2.1.1 – Checklists 45

3.5.2.1.2 – Instructions 45

3.5.2.2 – Course book used in the Study 46

3.5.2.3 – Writing papers done in the final exam 46

3.6 - Assumptions in the Study 46

Chapter 4 – DATA ANALYSIS & DISCUSSIONS 48

4.1 - DATA ANALYSIS 48

4.1.1 – Questionnaire 48

4.1.1.1 – Students’ information 48

4.1.1.1.1 – The most pleased skill 48

4.1.1.1.2 – The most wanted-to-be used-fluently skill 50

4.1.1.1.3 – Students’ learning purposes 51

4.1.1.2 – Students’ attitudes towards essay writing skills 53

4.1.1.2.1 – Question 1 53

4.1.1.2.2 – Question 2 54

4.1.1.2.3 – Question 3 55

4.1.1.2.4 – Question 4 56

4.1.1.2.5 – Question 5 58

4.1.1.2.6 – Question 6 59

4.1.2 – Students’ feedback 61

4.1.3 – Results from Scoring 62

4.1.3.1 – Holistic Scoring for Final Scores 63

4.1.3.2 – Analytic Scoring for Final Scores 64

4.2 – DISCUSSION 66

4.2.1 – About the controlled variables at the beginning of the study 67

4.2.2 – About the scores for the final papers 67

4.2.2.1 – Reliability of the scores given 67

4.2.2.2 – Comparing the score given to the two groups 68

4.2.3 – About four aspects of essays 74

4.2.3.1 – Responses to questions 1, 2, 3, 4 and 5 in the questionnaire 74

4.2.3.2 – Students’ feedback 76

4.2.3.3 – Results from scoring the four aspects 76

4.2.4 – Findings 77

4.2.5 – Answering the research questions 78

4.2.5.1 – Research question 1 78

4.2.5.2 – Research question 2 79

Chapter 5 – CONCLUSION & IMPLICATIONS and RECOMMENDATIONS 82

5.1 – Conclusion 82

5.2 – Implications 83

5.2.1 – Pay more attention to the feature of cultural differences 83

5.2.2 – Pay attention to the issue of writer-responsible language versus reader-responsible language 84

5.2.3 – Use analytic checklists in the process of teaching writing essays 84

5.3 – Recommendations 85

Bibliography 86

English Books & Documents 86

Vietnamese Books & Documents 91

Online documents (with links) 92

APPENDICES Appendix A: James Dale Williams’s holistic scoring scale 96

Appendix B: TOEFL holistic scoring scale 98

Appendix C: FCE holistic scoring scale 100

Appendix D: IELTS holistic scoring scale 102

Appendix E: Pilot testing Questionnaire 104

Appendix F: Questionnaire 109

Appendix G: Rubric checklist for Introduction 113

Appendix H: Rubric checklist for Paragraph in Body 115

Appendix I: Rubric checklist for Conclusion 117

Appendix J: Analytic rubric scoring scale (for scorers) 119

Appendix K: Syllabus 121

LIST OF TABLES, GRAPHS & FIGURES

I CHAPTER ONE: INTRODUCTION

Figure 1: Kaplan’s rhetorical patterns of written discourse in English (a) and

Vietnamese (b) 02

Figure 2: English and Vietnamese news item patterns 03

II CHAPTER TWO: LITERATURE REVIEW

Table 1: Features of the holistic rating scale used in Nakamura study (2004) 27 Table 2: Features of the analytic rating scale used in Nakamura study (2004) 27 Table 3: A comparison of holistic and analytic scales on six qualities of test

usefulness 30

III CHAPTER THREE: METHODS AND PROCEDURES

(no Table and Figure)

IV CHAPTER FOUR: DATA ANALYSIS & DISCUSSIONS

Figure 3: The difference between the control group and the experimental

group in the most pleased skill at the beginning of the course (above) and at the end of the course (below) 49

Figure 4: The difference between the control group and the experimental

group in the most wanted-to-be-used-fluently skill at the beginning (above) and at the end of the course (below) 51

Figure 5: The distributions of students’ learning purposes (data collected at

the beginning of the course) 52

Figure 6: Question 1 for students’ points of view towards the role of writing

skill in a successful career at the beginning (above) and the end (below) of the course 54

Figure 7: Question 2 for students’ points of view towards the organization of

the essay at the beginning (above) and the end (below) of the course 55

Figure 8: Question 3 for students’ points of view towards depending

Vietnamese style to develop an English essay at the beginning (above) and the end (below) of the course 56

Figure 9: Question 4 for students’ points of view towards the most important

factor to write an English essay at the beginning (above) and the end (below) of the course 57

Figure 10: Question 5 for students’ points of view towards the most difficult

aspect to write an English essay at the beginning (above) and the end (below) of the course 59

Figure 11: Subjects' awareness towards essay writing skills at the beginning of

the course (above) and at the end of the course (below) 60

Table 4: Changes of awareness towards the four aspects of students in the two

groups 61

Figure 12: The distributions of students’ feedback towards four aspects of

essay writing skills 62

Figure 13: The mean scores given by five scorers (in light colors) and the total

mean scores (in dark colors) for the final scores 63

Figure 14: The mean scores given by five scorers (in light colors) and the total

mean scores (in dark colors) for the aspect of Organization 64

Figure 15: The mean scores given by five scorers (in light colors) and the total

mean scores (in dark colors) for the aspect of Vocabulary 65

Figure 16: The mean scores given by five scorers (in light colors) and the total

mean scores (in dark colors) for the aspect of Grammar 66

Figure 17: The mean scores given by five scorers (in light colors) and the total

mean scores (in dark colors) for the aspect of Content 66

Table 5: The correlation coefficients among the scorers 68 Table 6: Comparing the statistical values analyzed from the scores of the final

papers between the Experimental group and Control group 69

Graph 1: Box-plots for the two groups scores 70

Table 7: Comparison between experimental group and control group in data

set for the box-plots 71

Graph 2: Q-Q plot for the score distribution of the Experimental group 72 Graph 3: Q-Q plot for the score distribution of the Control group 72 Table 8: The results of Shapiro-Wilk test for the normality of scores’

distributions of the two groups 73

Table 9: The t test result for the final scores of the control group and the

experimental group 74

Table 10: Comparing the control group’s responses to the first and the second

versions of the questionnaire 74 Table 11: Comparing the experimental group’s responses to the first and the

second versions of the questionnaire 75

Figure 18: Comparison the average scores of the four aspects between the two

Abstract

This study was conducted to evaluate the effectiveness of using analytic rubric checklists for writing and peer correction activities to enhance writing skills for third-year in-service university students at Technical Education University This technique is aimed at helping students write papers which are direct and focused and use “writer-responsible” language by continuously reminding them of the organization of essays

Data was collected and analyzed by means of a combination of qualitative and quantitative methods The responses to two versions of a questionnaire, the co-operative teacher’s observations and the results of the final papers were analyzed The aims of these analyses were primarily to determine whether students’ essay writing skills would be improved when these students based their writing and peer correction activities on analytic rubric checklists, which focused much on the organization of English essays, and to discover how students’ attitudes towards aspects of English essays changed after they were taught by this teaching technique

The results displayed that when students were equipped with the analytic rubric checklists to do their writing and peer correction activities, their essay writing skills would be improved remarkably as well as they would pay more attention to the organization of English essays

CHAPTER ONE

INTRODUCTION

This thesis reports the results of experimental research which was done on 99 third-year university students The aim of this research is to reduce the negative effects of first language transfer when learners write essays in English by providing them with analytic rubric checklists for them to use during writing and peer correction activities

This Chapter presents (1.1) the background of the problems, in which the main problems for the essay writing skills are introduced, (1.2) the statement of the problems, (1.3) the purposes of the study, in which the aims and the objectives of the thesis are referred to, (1.4) the research questions, which the thesis was done to answer, (1.5) the hypothesis, (1.6) the scope of the study, (1.7) the definition of terms, (1.8) the limitations of the study and the chapter summary

1.1 – BACKGROUND OF THE PROBLEMS:

It is undeniable that writing in a second language will always be more constrained, more difficult, and less effective than writing in first language (Silva, 1993: 668) Such constraints, difficulties and ineffectiveness clearly come from the limited second-language knowledge of writers Weigle (2000) stated that “because

of the constraints of limited second-language knowledge, writing in a second language may be hampered because of the need to focus on language rather than the content” (Weigle, 2000: 35) Besides the above limited linguistic disadvantage, second-language writers may also be disadvantaged by cultural factors, which they may fail to perceive These factors very often lead second language writers to approach the topic and develop their ideas in a second language piece of writing in the same way that they always do in their mother tongue Based on the thesis

Figure 1: Kaplan’s rhetorical patterns

of written discourse in English (a) and Vietnamese (b) (source: Kaplan, in

Brown 1994 : 322 , re-drawn)

writing academic papers in English are (1) the ways they approach the topic and (2) the style they develop their ideas in English

For the first obstacle, Kaplan (1966 in Brown 1994: 322) described patterns

of written discourse in languages as in Figure 1 English discourse, according to Kaplan (1966), proceeds in a straight line,

while discourse in Asian pattern proceeds

in a spiral line As Vietnam is an Asian

country which has been heavily influenced

by Chinese culture, Kaplan’s remark is

probably applicable to the case (Lê

Nguyễn Minh Thọ, 2000: 28) Kaplan’s

remarks can be used to explain the reason

why Vietnamese writers usually delay the

introduction of topics For English people,

such a way is considered to be “beating

around the bush” when pieces of redundant information are introduce before the topic is approached McKay (1992) characterized Pacific Rim writers – including Vietnamese writers – as not initially stating the statement of purpose in the discourse and she claimed these writers think that “getting to the point too soon does not stimulate the readers’ curiosity nor does it create suspense” (McKay, 1992: 5) Therefore, it is common to find an elaborate maze of wordiness before arriving at the topic sentence which is normally placed at the end of the passage” (McKay, 1992: 6)

Thus, Vietnamese and English differ in the placement of topic in news items (Lê Nguyễn Minh Thọ, 2000: 29) Lê Nguyễn Minh Thọ (2000), quoting Đinh Văn Đức and Nguyễn Hòa (1999), identified two different most-commonly-used patterns

of political and social new writing in English and Vietnamese as shown in Figure 2

English pattern Asian pattern

As Figure 2 shows, the main topic of papers in English writing style is presented right from the beginning while it is usually not given until the background information has been given in Vietnamese style The Vietnamese way to approach main topic in Figure 2 more or less displays the way Vietnamese people communicate Moreover, a piece of Vietnamese writing is considered as a good one when it is slowly approaching the main point (Nguyen [1991] in Ellis [1996: 64]),

“Vietnamese rarely use a direct approach in their communicative, as it is regarded as blunt and rude,” the author added Vietnamese people think that they will be impolite if they directly address the main topic without having some words to ask after the listener and her/his relatives (Hữu Đạt, 2000: 65) This opinion is completely similar to Ellis’s (1996), “The directness of Western discourse styles might be viewed by the Vietnamese as rude and uncouth” (Ellis, 1996: 64), he observed Hinds (1990) called this way to approach the topic as “delayed introduction of purpose” and affirmed that this way has the undesirable effect of making an essay appear incoherent to the English-speaking reader, although the style does not have that effect on the native readers (Hinds; 1990: 99)

Figure 2: English and Vietnamese news item patterns

(Source: Lê Nguyễn Minh Thọ, 2000: 29)

The second obstacle for Vietnamese writers, when doing their work in English, is that they always find it hard to make explicit the connections between propositions and ideas in texts Vietnamese people not only prefer to “beat around the bush” in the way of approaching the topic but also preferring to “beat around the bush” in the way of expressing their ideas While English is considered a a “writer-responsible” language (Hinds (1987), in Weigle, 2002: 21-22; Paltridge and Startfield, 2007: 12-51; Limon, 2006: 2), that is, the native English tend to think that the responsibility for effective communication lies primarily with the writer, while Vietnamese writers typically use a “reader-responsible” language as is common in writers of other Asian languages (Paltridge and Startfield, 2007: 49) That is to say,

in English writing, if communication fails we do not blame the reader for not making enough effort, but assume that what was said was insufficiently clear or well-organized, in Vietnamese writing, a “reader-responsible” language, it is seen as the reader’s responsibility to understand what the writer intended to say and writers may prefer to offer hints and nuances rather than make direct statements After years

of reading, writing, thinking, asserting, arguing, and defending their ideas in such a

“reader-responsible” way, Vietnamese writers find it difficult to express their idea in texts in a way that readers do not need to infer the connections on their own

Consequently, when writing English, a language which focuses narrowly on the main ideas of an essay, Vietnamese students usually have problems when they have to abandon the habits of their familiar way to express ideas in their second language and to organize the writing hierarchically

Thus, Vietnamese writers’ problems in writing texts in English are therefore deep-rooted in differences between the Vietnamese culture and the Western one Differences in cultures, inevitably, entail linguistic differences, causing obstacles to L2 writers Brock and Walters (1992) considered papers written by Asian writers

“weak” from the points of view of the Western style when support, unity and coherence of the writings are taken into consideration; that is to say, the style to

develop the writing (Brock and Walters; 1992: v-vii) Grabe and Kaplan came to same the conclusion, stating, “Vietnamese students focused less effort on the plot development and more effort on the additional features of the character” (Grabe and Kaplan, 1996: 191) Those authors found the following:

“ the Vietnamese stories appear to be less goal-oriented and hence less focused on the plot that the typical English story The Vietnamese students in particular also drew more heavily on dialogue

in their stories and the dialogue information did not include information that, in general, forwarded the action of the story but was reflective or attributive in nature.”

(Grabe and Kaplan, 1996: 191-192) Also, the way to write an appropriate conclusion is completely different between the two cultures In English writing style, writers are advised not to introduce any new idea in the conclusion (Pyle and Page, 1944:171; Smalley and Ruetten, 1982: 151) In Vietnamese culture, people have the tendency to think they will make others uncomfortable when offering a closed ending So, Vietnamese students habitually leave their writing open-ended, leaving something more to be discussed

Mohan and Lo (1985 in Mc Kay, 1992) believed one factor which may be contributing to L2 writers’ unfamiliarity with English rhetorical patterns is their previous education which may have emphasized accuracy on sentence level rather than rhetorical development However, “whatever the reason – transference, development or education – the fact is that if L2 writers are not able to meet their readers' expectations on the rhetorical level they are likely to be judged as less effective writers by their Western audience” (McKay, 1992: 8) This author concluded “in order to be effective writers in English, Pacific Rim students will need to be able to employ the organizational conventions that their Western readers expect” (McKay, 1992: 8) To a large extent, effective writers are successful

because they meet the expectations of their audience They know what to write and what not to write so as to establish a strong reader-writer relationship (McKay,

Unfortunately, that method is not an easy one for all students Teaching Vietnamese students to write English essays, then, is not as simple as to teach them the sentence structures, the grammar rules, vocabularies, etc but needs to include in the syllabus some ways that can help students overcome their instinct which leads them to express their ideas in their mother tongue As writing is both a social and a cultural activity and second-language writers may be disadvantaged by social and cultural factors (Weigle, 2000: 22-38), teachers of writing English essays for Vietnamese students must find a way to minimize the effects of differences in

cultures rising up in the process of doing the work, in other words, to get rid of students’ favorite habit of expressing their ideas

1.2 – STATEMENT OF THE PROBLEMS:

For university students in the third year, writing English is not only for the purpose of expressing their ideas on paper but also for expressing their ideas in a way that can convince readers Years of using their mother tongue very often makes Vietnamese students express their ideas on paper in a way which is similar to the way that they express their ideas in their mother tongue “Beating around the bush” before approaching the topic in introduction and digressing from the main points when developing pieces of writing are the two biggest problems that the Vietnamese students have to overcome if they want to improve their writing skill

1.3 – PURPOSES OF THE STUDY:

This study aims to improve the third-year in-service university students’ essay writing skills by introducing analytic rubric checklists, which orientate students towards English writing style Students were asked to use the analytic rubric checklist to check their writing by themselves and to do the peer correction activity in order to keep those analytic rubrics always in their mind and This study also studies the change of students’ points of view towards the organization of essays after the teaching technique is applied

This teaching technique focuses on the organization of the essays rather than other aspects as grammar, vocabulary and content To a certain extent, the organization of a piece of writing can be seen as the “face validity” of the writing

To be persuaded, a reader must be convinced at the first view, which is the organization of the writing, before he/she approaches other kinds of the “validities”

of the papers

In 2001, Rosenfeld, Leung and Oltman did research on 155 undergraduate and 215 graduate faculty members in 21 United States universities to specifically identify the essential students’ L2 writing skills and found that the most important characteristic of academic writing was the organization of the writing (Hinkel, 2004: 18) With the mean importance rating on a scale of 0 to 5, they found the following results:

Task Statement

Mean Importance Rating

Organize writing to convey major and supporting ideas 4.19 4.46 Use relevant reasons and examples to support a position 4.09 4.34 Demonstrate a command of standard written English,

including grammar, phrasing, effective sentence, structure,

spelling, and punctuation

(Source: Hinkel, Eli: Teaching Academic ESL Writing, 2004: 18-19)

(*) responses of undergraduate faculties

(**) responses of graduate faculties

Rosenfeld, Leung and Oltman found that the responses of both communities

of undergraduate and graduate faculties clearly indicate that organizing writing to convey major supporting ideas and using relevant examples to support them occupy top priority in the quality of academic discourse (Hinkel, 2004: 18-21) Moreover, responses of graduate faculties, who surely thought much about the conveyance of ideas, proved the importance of the organization writing when it came close to the maximum end on the scale (5.0)

To a certain extent, the above result leads to the purpose of this thesis

1.4 – RESEARCH QUESTIONS:

The study has been done to answer the two following research questions:

1 Is the performance of students in the experimental group better than that of students in the control group after the teaching technique of using analytic rubric checklists has been applied?

2 How would the students in the experimental group change their attitude towards the aspects in writing English essays after being taught with the teaching technique?

1.6 – SCOPE OF THE STUDY:

This study is designed to work with and for third-year in-service university students majoring in English These students learn English not only to communicate with foreigners or to express their ideas but to be teachers of English in the future For these students, who already have a good command of English grammar and vocabulary and have had practice in writing a range of everyday text types, a good piece of writing is not simply a writing paper perfect in grammar, spelling, punctuation, etc but a writing paper which is developed in as close to native-like style as possible Also, the study can be applied to students in foreign language centers who are trying to perfect their academic essay writing skills for some specific tests such as TOFEL, IELTS

1.7 – DEFINITION OF TERMS:

In this thesis, the term “analytic rubric checklists” is used to refer to

checklists in which questions are put forward for students to follow to make sure they are developing the essay as required The checklists focus on each part of an essay, that is the introduction, paragraphs in the body and the conclusion The checklists will be discussed in details in Chapter Three, entry 3.5.2.1.1 Actually, the checklists include analytic and holistic rubrics, of which analytic rubrics focus on the organization of the essays and holistic rubrics focus on the content and ways of expressing ideas in English However, if a student fails to meet the demands of the analytic rubrics of the checklist, he/she will get no mark at all, even though his/her ideas may be convincing The term of analytic rubrics, therefore, is inserted to the checklist to underline the style of developing an English essay in its organization

Also, the term “essay writing skills” or sometimes “writing English essay

skills” is used to refer to the ability that a student has to write an approximately

300-word essay in English The essay can be of different kinds, such as argumentative, example, comparative or contrast, classification, process analysis or cause and effect However, under the time constraints of the study, only the kinds of argumentative and example essay were analyzed

The term “peer correction activity” used in the study is a little bit different

when it was applied to the two groups in the study For experimental group, peer correction activities were done based on the checklists of analytic rubrics whereas students in the control group did the activities based on the knowledge that they had collected from the lesson and on their own feelings

The term “English writing style” used in this thesis should be restrictedly

understood as the organization of an English writing paper as introduced in the

materials such as Writing Academic English, written by Alice Oshima and Ann Hogue, Academic Writing Course, written by R R Jordan, Refining Composition

Skills, written by Regina L Smalley and Mary K Ruetten, Writing for Advanced

Learners of English, written by Francoise Grellet or Discoveries in Academic Writing, written by Barbara Harris Leonhard The differences in writing styles

between Vietnamese and English are partly defined previously in this Chapter

The term the most pleased skill is used to mean the skill out of the four skills

in using English that students in the two groups feel confident with The term the

most wanted-to-be-used-fluently skill is used to mean the skill that students in the

two groups want to be able to use fluently most These terms are used often in Chapter 3, Chapter 4 and in the questionnaire

1.8 – LIMITATIONS OF THE STUDY:

The research was conducted using a Quasi-Experimental Two-group Post-test only Design which resulted in the biggest limitation to the research This kind of research introduces a limitation to the validity and reliability of the research as the classes used for the research were assumed to be more or less equal to each other

Also, the research was done with a population of only 99 students in the two groups, 47 in the experimental group and 52 in the control group A larger population would make the results of the research more valid and more reliable

This study was also designed to supplement the syllabus of teaching writing English essay skills for third-year in-service university students in the department of English at Technical Education University The technique is directly related to the score that students will get at the end of the course; therefore, those students paid more attention to what their teachers taught Due to such motivation, the results obtained from those students were rather interesting The study should, therefore, be done with students who have low motivation in writing to check its validity and reliability

The purpose of the study was to evaluate students’ improvement in the global strategies (the content and the form of the writings) rather than to focus on surface

mistakes (grammar, spelling, etc …); therefore it had certain limitations with students who still have problems with surface mistakes

Last but not least, the study had to be based on the assumption that students

in the control group and experimental group were at the same level of English and other assumptions relating to controlled variables

To sum up, Chapter One already went through some basic aspects of my research At the beginning, this Chapter discussed features that affect essay writing skills Many researchers have explored the differences in cultures Western countries and Asian countries, such as Vietname, where second language learners perform their productive skills in a way that is familiar in their native culture Of the two productive skills, speaking and writing, the latter must be influenced more when, as

it is proved, second language users think longer about how to use his/her second language, his/her instinct of using the second language in his/her native culture will

be stronger (Davis and Pearce, 2000: 96; Brown, 1994: 23; Lê Nguyễn Minh Thọ, 2000: 39) The study aims to improve English learners’ essay writing skills by providing them with a tool - the checklists of analytic rubrics that they could use to check themselves when doing the activity of writing as well as doing the activity of peer correction It was hypothesized that the teaching technique applied would change students’ awareness toward the organization of the writing paper and would enhance students’ ability of writing This Chapter also refers to some limitations that the study got, which impacted on the results of the study

In the next Chapter, some literature relating to the impacts of differences in cultures on learning and writing second language, to analytic and holistic rubrics will be reviewed

CHAPTER TWO

LITERATURE REVIEW

This Chapter reviews the issues relevant to the aspects that the study focuses

on to improve the essay writing skills for third-year in-service university students Thus, the following discussions provide views on the literature about (a) impacts of differences in cultures, and (b) aspects relevant to rubrics and using rubrics in supporting teaching style

2.1 – IMPACTS OF DIFFERENCES IN CULTURES:

2.1.1 – ON LEARNING SECOND LANGUAGE:

In 1999, Robert L Politzer, when reporting at the Fifth Annual Round Table

Meeting on Linguistics and Language Teaching Developing Cultural Understanding through Foreign Language Study at Georgetown University (United State of

America), affirmed:

“As language teachers we must be interested in the study of

culture not because we necessarily want to teach the culture of the other country but because we have to teach it If we teach language

without teaching at the same time the culture in which it operates, we are teaching meaningless symbols or symbols to which the students attach the wrong meaning …”

(Politzer, 1999: 100)

Supporting Politzer’s idea, Kuo and Lai (2008), based on the studies of Samovar, Porter, and Jain (1981), explained why culture and communication are inseparable That is, according to those authors, because culture not only dictates who talks to whom, about what, and how the communication proceeds, but also helps to determine how people encode messages, the meanings they have for

messages, and the conditions and circumstances under which various messages may or may not be sent, notices, or interpreted (Samovar, Porter, and Jain [1981]

in Kuo and Lai, 2008: 2) Kuo and Lai (2008) pinpointed that to be able to bridge the culture gap, “second language teachers, therefore, should pay more attention

to the diversities of cultures, identify key cultural items in every aspect when they design a language curriculum, and apply appropriate teaching strategies to learning activities” (Kuo and Lai, 2008: 1) The two authors, then, came to the conclusion that culture is the foundation of communication Actually, many authors, such as Atkinson and Ramanathan (1995), Brock (1992), Carrell (1984), Ellis (1994, 1996), Grabowski (1996), Lado (1957), Long and Richards (1999), McKay (1992), Davies (2003a; 2003b), Cumming (2001), Alasdair (2008), Myles (2002), Kuo and Lai (2008), Limon (2006) Ghosn (1996), and others have shared the same ideas Culture, thus, must be incorporated as an essential component of second language learning and teaching Only after cultural issues become an inherent part of the language curriculum and instruction can students be successful in their target language learning

2.1.2 – ON WRITING SECOND LANGUAGE:

In recent years the effects of differences in cultures between L1 and L2 have been the focus of a lot of researches, and it has become clear to researchers that many aspects of writing are influenced by cultures (Weigle, 2000: 20) Atkinson and Ranamathan (1995) affirmed “these differences are viewed largely from a cultural perspective, that is, as part and parcel of the divergent social practices, or discourses,

of the L1 composition - versus ESL - teaching communities” (Atkinson and Ranamathan, 1995: 539-540) They found the following:

“Although these two differing world views may be masked by allegiance to superficially similar paradigms of writing and writing instruction, they are in fact the products of two distinct cultures – with

their own oft-contrasting norms of what academic writing is, what constitutes good academic writing, and how the latter can best be communicated.”

(Atkinson and Ranamathan, 1995: 540)

With regard to the writing skill, Weir (2005), in his Language Testing and

Validation, thought “writing is a social and a cultural act and is not limited to the

individual space of the writer” (Weir, 2005: 108) Weigle (2002) had the same idea

when stating “writing is both a social and a cultural activity, in that acts of writing cannot be looked at in isolation but must be seen in their social and cultural contexts” (Weigle, 2002: 22) Actually, the above authors were not the first to come

to such a conclusion, Leki (1992) and Grabe and Kaplan (1989, 1996) provided useful introductions to some of the cultural influences on writing Those authors pointed out that variations in writing in different cultures does not reflect inherent differences in thought patterns but rather “cultural preferences which make greater use of certain options among the linguistic possibilities” (Grabe and Kaplan, 1996: 184) Kaplan argued “although various rhetorical modes are possible in any language, each language has certain clear preferences” (Kaplan, [1966] in Brown, 1994: 322-323) Writers of English, generally, prefer subordination and hierarchical organization, focus narrowly on the main ideas of an essay; writers of Arabic are said to use more co-ordination and parallelism; writers of Spanish prefer lengthy introductions, almost opposite to those of English (Collado [1981] in Leki, 1992); Chinese writers tend to provide a series of examples without stating the main point

of the example or tying them together through a generalization (Matalene (1985), in Leki, 1992); Japanese prefer to offer hints and nuances (Hinds [1987] in Limon, 2006: 2); McKay (1992), based on the study done by Eggington (1987) and Hinds (1987), found a tendency in developing pieces of writing among the community of Pacific Rim writers She claimed that the tendency has been influenced by a pattern

of rhetorical development which originated in classical Chinese poetry According

to this Ki – Shoo – Ten – Ketsu pattern1, the third section – the ten –introduces a new

topic may lead Western readers to see this section as an irrelevant aside while Eastern writers are likely to view this section as introducing an important tangential topic that the reader must relate to the main topic (McKay, 1992: 5) Weigle (2002) affirmed “investigation into contrastive rhetoric has demonstrated that cultural expectations can have a consequence for the coherence of texts – that is the organization of a text into a meaningful whole” (Weigle, 2002: 21) Upon this socio-cultural aspect, second language writers may wrongly assume that the social conventions that exist in their native language are also operative in their second language Therefore, many second language writers usually think that structuring an argument or supporting a main point is the same in both cultures “Making these assumptions can result in second language writers’ being misunderstood and less effective writers,” McKay pinpointed when she studied the socio-cultural factors in teaching composition to Pacific Rim writers (McKay, 1992: 1)

As theories of second language education have evolved from grammar – translation to the audio – lingual method, then to the more communicative approaches that are commonly used today, the role of writing in teaching second language has also changed When grammar – translation method was the most favorite method in teaching second language, writing has commonly been viewed as

a support skill, used to reinforce the acquisition of grammar With the audio – lingual method, writing was seen as a way to support the memorization of language structures Until recently, the communicative approaches – with their emphasis on oral proficiency – have tended to de-emphasize writing skills Fortunately, ideas from writing-to-learn, writing across the curriculum, and writing for academic purposes movements in composition and English as a second Language have all had

1

Ki – Shoo – Ten – Ketsu is a rhetorical movement (consisting of four parts) in Japanese discourse, including but not limited to narratives and essays Ki offers topic, Shoo develops topic, Ten adds a surprise turn, and finally ketsu offers conclusion This rhetorical movement, originated from Ba – Gu – Wen in Chinese poetry,

an impact on thinking about the place of writing in second language education (Homstad and Thorson, 1994: 1)

These days, when “the ability to write effectively is becoming increasingly important in our global community and instruction in writing is thus assuming an increasing role in … second – and foreign – language education” (Weigle, 2002: 1), moreover “writing will ensure people of a successful career when it is used in most

of formal documents, letters of application, reports, minutes, etc.” (Freeman and Richard, 1988: 97), writing skills, especially essay writing skills, becomes more important when it orientates learners towards communicating – in written language – more formally and more effectively

Writing is particularly important because of its empowering role It enables

us to launch our ideas into the world, to win membership of discourse communities and to influence others We owe it to our students to provide them with the basic knowledge and skills they need to be able to communicate effectively in writing in English in any academic, professional or other social contexts in which they may become involved in the future Limon (2006) maintained that the starting point is a detailed awareness of the way different textual genres work in different cultures and

a strong sense of reader expectations (Limon, 2006: 7)

Additionally, writing skill is a little bit different from the other productive skill – speaking, where the writers’ products are not only in black and white, which are unable to correct, but also must be expressed in such a way that the receiver can not misunderstand Grabowski (1996) claimed:

“Writing, as compared to speaking, can be seen as a more standardized system which must be acquired through special instruction Mastery of this standard system is an important prerequisite of cultural and educational participation and the maintenance of one’s rights and duties … The fact that writing is more

standardized than speaking allows for a higher degree of sanctions when people deviate from that standard.”

(Grabrowski, 1996: 75) Unfortunately, while writing is a difficult skill that needs practicing very often, the second language users seem to have less opportunity to practice it It is inevitable when in our real world writing probably is the linguistic skill that is least

used by most people in their native language Davies and Pearce (2000) affirm “even

in the most “advanced” societies a significant percentage of the adult population writes with difficulty” (Davies and Pearce, 2000: 96) Moreover, writing is an activity of the minority, mostly done as part of professional obligation except for a few who occasionally produce personal writing; therefore, "the motives for writing are more abstract, more intellectualized, further removed from immediate needs"(Vygotsky [1962] in Pilus, 1993: 2) Those aspects turn teaching writing skills into one of the biggest challenges in teaching second language

Teaching learners’ academic writing skills in second language has been of concern for a long time in tertiary education (Belcher and Braine, 1995; Jordan,

1997 in Bacha, 2002: 161) Students studying in institutions of higher learning in the medium of English, which may not be their native language, have been found to face problems mainly in writing, making them unable to cope with the institution’s literacy expectations However, these “disadvantaged” students may be able to develop writing skills significantly with positive instructional attitudes towards the errors they make and with the awareness on the teachers’ part of learner problems (Grabe and Kaplan, 1996; Shaughnessy, 1977; Zamel, 1983 in Bacha, 2002: 161)

As a rule of thumb, when two languages are in contact with each other, the native language always has certain effects on the second language, especially on the writing skill (Lado (1957), in Vietnamese version, 2003: 9-14; Bùi Khánh Thế, 2005: 11-14) About the effects that the native culture has on the process of learning second language, Brown (1994) wrote:

“No one can deny the effect of one’s native culture, or one’s predisposition that is the product of perhaps years of schooling, reading, writing, thinking, asserting, arguing, and defending In our current paradigm of attending carefully to schemata and scripts, native language patterns of thinking and writing simply cannot be ruled out.”

(Brown, 1994: 323) The writers, when doing the job in their second language, usually let their instinct control the skill by doing it as the way they use their native language in thinking, or in communicating (Davis and Pearce, 2000: 96; Brown, 1994: 23) The most obvious reason is that when a person writes, he/she has more time to think about what to write than the speaker who may have to react to the other almost immediately (Lê Nguyễn Minh Thọ, 2000: 39) Writers generally has more time to plan, review and revise their words before the words are finalized, while speakers must plan, formulate and deliver their utterance within a few moments if they want

to maintain a conversation (Weigle, 2002: 15-16) Lê Nguyễn Minh Thọ stated in his thesis, “The more he/she spends time thinking, the more likely he/she is to be affected by the native culture,” (Lê Nguyễn Minh Thọ, 2000: 39) Therefore, Davis

and Pearce (2000) suggested students should not let their instinct control this skill

when they are doing the activity of writing (Davis and Pearce, 2000: 96) Unfortunately, it is not an easy thing to do

2.2 - HOLISTIC SCORING AND ANALYTIC SCORING:

The best way to improve our learners’ writing skills is undeniably to let them write and write more After letting our learners write, we – teachers of writing – collect the papers, read them all, and give feedback for our students And then, our students will receive feedback from us, revise based on the feedback, and then repeat the whole process as often as possible Unfortunately, this puts an enormous load on us – classroom teachers, who have been faced with reading and providing

feedback for perhaps 50 essays or more every time a topic is assigned To reduce the work load for classroom teachers, a measure to evaluate pieces of writing which has not only validity and reliability but also make the evaluation meet the purpose of asking students to demonstrate their membership in the community of fluent writers

of English should be developed

The past 20 years have seen two major changes in the teaching and rating of L2 writing In teaching, emphasis is shifting from product to process; in rating, indirect, “objective” tests have given way to direct assessment of samples of student writing Writing assessment research, “like oral proficiency assessment research, has focused on improving the reliability of rating scales and procedures, potentially at the cost of the ratings’ validity” (Connor-Linton, 1995: 762) Such researches have introduced holistic scoring rubrics and analytic scoring rubrics

2.2.1 - HOLISTIC SCORING RUBRICS:

2.2.1.1 – Holistic Scoring:

Until the 1960s, the assessment of large numbers of students’ ability of writing chiefly used multiple-choice exams Such measures of writing evaluation are called indirect assessments of writing skills, typically asking questions about spelling, punctuation and editing, but do not require learners to produce written text Some teachers and researchers, however, were concerned that such exams were not very effective, lacking both reliability and validity Lack of reliability was deemed the biggest problem with classroom assessment, and lack of validity was deemed the biggest problem with large-group assessment (Williams, 2003: 318) It is argued that the only way to measure writing was to ask students to write and that multiple choice tests were invalid

Largely in response to this criticism, Educational Testing Service (ETS), the group that sponsors the National Teachers Examination and the Advanced Placement Program tests, decided to explore the possibility of developing a valid

and reliable way to evaluate writing After several years of effort, Educational Testing Service came up with the method known as holistic scoring (White, 1986) The means of scoring is defined as “… a procedure for assessing writing in which a reader judges a writing sample for its overall effectiveness” (Hobson and Steele in McLean, 1992: 4) The method “involves having … readers evaluate a writing passage for its overall effectiveness, as a whole, rather than by considering its individual features such as word use, grammar, punctuation, organization, and style

in isolation” (Glowacki, in McLean, 1992: 14) In holistic scoring, essays are not given low scores just because they contain many mechanical errors, nor are they given high scores just because they are well organized The reader considers the overall impression created by the student's writing paper and assigns a score consistent with that overall impression

The method quickly became popular as an effective means of testing large numbers of students, especially at the university level, because it is valid, highly reliable, and doesn’t take much time (Williams, 2003: 318) Since holistic evaluation can be as reliable as multiple-choice testing and since it is always more valid, it should have the first claim on our attention when we need scores to rank-

order a group of students (Cooper and Odell, 1977: 4), in the early 1980s, individual

teachers started using holistic scoring in their own classrooms as a means of reducing their paper load while simultaneously increasing the reliability of assessment

As mentioned previously, holistic scoring involves looking at the whole essay, not just parts of it The procedure is based on the notion that evaluating writing skill does not consist of measuring a set of sub-skills, such as knowledge of

punctuation conventions, but rather of measuring what White (1994) called “a unit

of expression” (White, 1994: 18) Clearly, some things are more important than

others when it comes to a successful response A writing paper may gain its effectiveness when it, in general, can persuade readers with its content even though

the language, grammar, or punctuation used in the paper causes some problems The purpose, therefore, is to make an overall assessment of the quality of the writing as a whole (Williams, 2003: 318) Its purpose is not to correct, edit, or diagnose the weaknesses of a writing paper, but to form a general impression of the writing (Charney [1984] in McLean, 1992: 11)

This method of scoring asks raters to read quickly and make a judgment

about the total effectiveness of the writing sample with factors such as organizations, spelling, and grammar considered to be of equal importance

(Hobson and Steele, in McLean, 1992:04), Hobson and Steele, moreover, advised

raters should not reread the paper to justify the score in terms of specific errors (Hobson and Steele, in McLean, 1992: 6) It is thought that when readers read a

paper very quickly he/she can make his/her assessment more reliably Skilled holistic readers, therefore, are encouraged to take only about a minute or two to go through a two-page paper because the more time readers take to get through a paper, the more inclined they are to begin mentally editing, focusing on the surface errors (Williams, 2003: 319)

Holistic scoring solves the problem that a piece of writing may receive different assessments from different people because each evaluator is likely to look for different qualities in the piece of writing The problem will be solved through a process of “socialization,” during which evaluators agree to reach a consensus on a

specific set of criteria, called a rubric

Depending on the purposes of the curricula, the rubrics for holistic scoring

will differ from one another Williams (2003), in his Preparing to Teach Writing,

suggested rubrics for older students usually use a 6-point descending scale to gauge the quality of each response A paper that scores a 6 is very good; one that scores a 2

is not good at all In addition, a rubric is divided into lower half and upper half to specify the general quality of a response Upper-half papers (6, 5, 4) are well written, whereas lower half papers (3, 2, 1) are not No direct correspondence exists

between numeric scores and letter grades Translating scores into grades is a separate procedure and should not even be discussed as part of a round of holistic scoring (Williams, 2003: 319) Williams’s holistic rubric scale can be found in the Appendix A (page 96), in which, besides the rubrics applied for Middle and Secondary Students that have been mentioned above, includes rubrics for Writing skill in Elementary level with the scoring scale of three

2.2.1.2 – Holistic Scoring of some Tests:

2.2.1.2.1 – Test of English as a Foreign Language (TOEFL):

The purpose of the Test of English as a Foreign Language (TOEFL) is to evaluate the English proficiency of people whose native language is not English and the scores are used primarily in decisions about admission to colleges and universities in United States of America and Canada (Weigle, 2002: 141) More

specifically, the test aims to define the ability of the attendees to generate and

organize ideas, to support those ideas with examples or evidence, and to compose in standard written English, according the TOEFL Bulletin (ETS 2000)

The writing is scored on a six-point holistic scale, with the score of 6 is the best and the score of 0 is given when a paper contains no response, merely copies the topic, is off-topic, is written in a foreign language, or consists of only keystroke characters The TOEFL holistic scale is on the Appendix B (page 98)

2.2.1.2.2 – Cambridge First Certificate in English (FCE):

The First Certificate in English examination is part of a suite of English language examinations at five levels of proficiency administered by the University

of Cambridge Local Examinations Syndicate These examinations are used to certify English language proficiency for a variety of purposes, for example, the examinees that pass the third level of proficiency in the system are presumed to have sufficient language proficiency for office work or to pursue a training course in English

The First Certificate in English holistic scale is on the Appendix C (page 100)

2.2.1.2.3 – International English Language Testing System (IELTS):

The purpose of International English Language Testing System (IELTS) is to assess the language ability of candidates who need to study at the post-secondary or university level or work in a professional capacity where English is used as the language of communication Thus, the level of English targeted by IELTS is more similar to the TOEFL than to FCE, which is targeted at a somewhat lower level of English proficiency (Weigle, 2002: 155)

Scores on the IELTS are reported as band scores between 1 (non user) and 9 (expert user) Separated band scores are reported for each skill section as well as an overall band score The overall band descriptors are found in Appendix D (page 102)

2.2.1.3 – Disadvantages of Holistic Scoring:

The holistic scoring method has been used because it is relatively economical; raters can assign a global rating quickly (Hoffman and Holden, 1997: 2) However, the method has also been criticized by other researchers

Weigle (2002) maintained “one drawback to holistic scoring is that a single

score does not provide useful diagnosis information about a person’s writing ability”

(Quellmalz [1986] in Hoffman and Holden, 1997: 3; Weigle [2002] in Weir, 2005: 188) Hamp-Lyons (1995) had the same ideas when indicating “the writing of second language English users is particularly likely to show varied performance on different traits, and if we do not score for these traits and report the scores, much

information is lost” (Hamp-Lyons, 1995: 760) She affirmed “in order to reach a

reasonable balance among all the essential elements of good writing, readers need to

pay conscious attention to all those elements” (Hamp-Lyons [1991] in Weir, 2005:

189) Weir (2005) also agreed to those ideas when he thought if the overall score has been affected by just one or two aspects of the work, it is very dangerous to evaluate the effort of a writer (Weir, 2005: 188) This is really the major disadvantage of holistic scoring when it can give us useful ranking information but no details, providing no useful diagnostic information about the writers’ writing ability With holistic scoring, the result of a writing paper makes result-readers fail to distinguish between various aspects of writing papers such as control of syntax, depth of vocabularies, organization and others Hamp-Lyons (1995) observed that scores generated holistically cannot be explained to other readers in the same assessment community; diagnostic feedback is out of the question, she then symbolized a holistic scoring system as a closed system, offering no windows through which teachers can look in and no access points through which researchers can enter (Hamp-Lyons, 1995: 760-761) Different writing papers can have the same scores but actually those same scores do not need to arrive at the same criteria, therefore, it

is very difficult interpret holistic scores (Weigle [2002] in Weir, 2005: 188) Consequently, “writers who have pieces of writing scored under holistic scoring system cannot be protected against the influence on raters’ scores of features of writers’ text where the scoring system obscures the basis for scores” (Hamp-Lyons, 1995: 761)

This is especially problematic for L2 writers since different aspects of writing ability may develop at different rates for different L2 learners (Weigle [2002] in Weir, 2005: 188; Park, 2008a:1) Because the objective of the writing assessment program is to improve the student’s writing skills, which are constituted by different elements rather than one or two prominent ones, all elements must be taken into consideration when evaluated

Weigle (2002), additionally, introduced another disadvantage of holistic scoring system That is the superficial characteristics such as length and hand-writing may influence the anchor score of the writing He studied the scores rated

under both holistic and analytic scoring systems and found out that scores rated under holistic scoring system have been shown to correlate with those superficial characteristics (Weigle [2002] in Weir, 2005: 188) That is to say, very long writing papers and bad script writing papers often get low scores Moreover, Swartz, Patience and Whitney (1985, in McLean, 1992), stated that raters’ reading ability may affect the reliability of a large-scale writing assessment (Glowacki, in McLean, 1992: 14)

Weigle (2002) also noted that holistic scoring system would lack validity He stated “holistic scoring has also come under criticism in recent years for its focus on

achieving high inter-rater reliability at the expense of validity” (Weigle [2002] in

Weir, 2005: 188) This idea, however, fails to reach a consensus in the community

of researchers, especially when there is a lack of agreement about ways of measuring reliability of writing papers and there are many different ways available

to compute writing reliability Nakamura (2004) conducted a research with ninety L2 English writers on both holistic and analytic scoring systems and found that analytic scoring system was more reliable About the validity in Nakamura’s research, the correlation coefficient of the students' scores in two scales was 95 (the variance was over 90%), which was extremely high Therefore, it might be that holistic scale test results could predict analytic scale test results with a high degree

of probability and vice versa (Nakamura, 2004: 48), maintaining that the validity was equal to the two scoring scales The holistic and analytic rating scales used in her study are attached on the next page Otherwise, a study conducted by Bauer (1981) showed that the analytic scoring system had the higher reliability than holistic system, specifically 954 and 928 respectively (Glowacki, in McLean, 1992: 17) while the inter-rater correlation in the study conducted by White and Polin (1986) and Swartz, Patience and Whitney (1985) showed that holistic scale was higher (Glowacki, in McLean, 1992: 17)