The effect of portfolio writing process on students’ writing ability: A case study in the context of the university of languages and international studies, Vietnam National

This paper will purport to identify the effects of portfolio writing process from students’ self-reflection and to shed light on the actual influence through their writing pieces.

ẢNH HƯỞNG CỦA QUÁ TRÌNH LÀM TẬP BÀI VIẾT ĐỐI VỚI KĨ NĂNG VIẾT CỦA SINH VIÊN:

NGHIÊN CỨU ĐIỂN MẪU TRONG BỐI CẢNH TRƯỜNG ĐẠI HỌC NGOẠI NGỮ, ĐẠI HỌC QUỐC GIA HÀ NỘI

Nguyn Th Nhung Trường Đại học Ngoại Ngữ, ĐHQG Hà Nội

viết dưới góc ñộ một công cụ ñánh giá thay thế hoặc

nhận xét theo cặp, nhưng chưa có nhiều công trình

chuyên sâu về ảnh hưởng của quá trình làm tập bài

viết ñến khả năng viết của sinh viên

Do ñó, nghiên cứu này nhằm (1) tìm hiểu tác

ñộng của quá trình làm tập bài viết thông qua ñánh

giá của chính sinh viên và (2) chỉ ra ảnh hưởng thực

sự của quá trình này qua việc phân tích bài viết của

sinh viên Đây là một nghiên cứu, sử dụng hai công

cụ nghiên cứu: phng v n (interview) và phân tích

tài liu (document analysis)

Nghiên cứu cho thấy sự tiến bộ rõ rệt trong các

mảng coverage of main points và vocabulary

trong khả năng viết của sinh viên Việc phân tích tài

liệu ñã chỉ ra những thay ñổi tích cực trong các

phiên bản viết của sinh viên nhờ vào nhận xét của

giáo viên, bạn cùng lớp và việc tự sửa lỗi của chính

bản thân người viết

Đáng chú ý, có sự không thống nhất trong kết

quả của hai câu hỏi nghiên cứu Một số khía cạnh

trên thực tế chỉ ra từ bài viết lại có những thay ñổi

tích cực hơn rất nhiều so với nhận ñịnh ban ñầu của

sinh viên và ngược lại

T khóa: portfolio, portfolio writing process,

writing abilities, CAE marking scheme

Abstract: A vast body of research has studied

writing portfolio as a form of assessment tool or in

terms of peer written feedback However, the influence of portfolio writing process on students’ writing ability has still been awaiting for thorough investigation.To fill in the gaps, this paper will purport to (1) identify the effects of portfolio writing process from students’ self-reflection and (2) to shed light on the actual influence through their writing

pieces Multiple-case-study design was employed;

in which interview and document analysis served

as two data collection instruments Together with face-to-face-semi-formal interviews, three portfolios

of three participants were chosen for analysis For each portfolio, the researcher randomly chose two writing pieces of different genres to analyze Findings from two research questions revealed a similarity of the most tremendous progress in

coverage of main points and vocabulary

Document analysis proved that positive changes were made from version to version thanks to the teacher’s and peers’ comments and the writer’s self-correction Remarkably, there also existed noticeable discrepancy between the findings of the two research questions Some writing abilities reported as moderately improved turned out to be more positively progressed than the students had thought and vice versa

Key terms: portfolio, portfolio writing process,

writing abilities, CAE marking scheme

THE EFFECT OF PORTFOLIO WRITING PROCESS

ON STUDENTS’ WRITING ABILITY:

A CASE STUDY IN THE CONTEXT OF THE UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES,

VIETNAM NATIONAL UNIVERSITY

1 Statement of research problem and

rationale for the study

Recently, many researchers and educators have

mentioned the inclusion of portfolio in writing

assessment as one of the innovative and effective

methods for restructuring traditional composition

courses (Kieffer & Faust 1996) In the studies of

several researchers such as Herter (1991) and

Ballard (1992), they have successfully proved the

improvement of students’ writing skills with the

application of writing portfolio

Realizing the benefits which portfolio brings

about, the Fast Track Program at the Falculty of

English Language Teacher Education (FELTE),

University of Languages and International Studies

(ULIS), Vietnam National University (VNU)

started to include portfolio as an assessment tool

for third-year-fast-track students from the school

year of 2005 – 2006 Aiming at level C1 for those

targeted students with reference to the Common

European Framework of Reference for Languages

(CEFR), the writing program puts central focus on

the Certificate in Advanced English design (CAE)

Thus, portfolio as an assessment tool also involves

elements of CAE marking descriptors There have

been a certain number of studies related to the

topic of portfolio in the scope of ULIS, VNU

Several papers and articles have investigated

portfolio from the angle of an alternative

assessment tool for traditional assessment (Tran et

al 2008) Unfortunately, they only gave

theoretical backgrounds without studying on any

particular group of students Besides, many other

theses exploit portfolio in the aspect of peer

written feedback such as Pham (2007), Le (2009)

and Hoang (2012) As a matter of fact, students’

writing portfolios are to be assessed by specific

marking criteria However, there has not been any study which offers an insight into students’ self-assessment of their own writing ability as a result

of portfolio writing process, guided by those criteria Particularly, the specific impacts of portfolio writing process on students’ practical writing ability with all factors considered together still remains cryptic

All the aforementioned reasons motivated the researcher to conduct a research paper entitled

“ The effect of portfolio writing process on students’ writing ability: a case study in the context of the University of Languages and International Studies” to fill in the identified gaps in the local literature

2 Aims of the study

This paper is conducted to address the two following research questions:

Question 1: What is students’ self-assessment

of their writing ability as a result of portfolio writing process, as guided by CAE marking descriptors?

Question 2: To what extent does the

portfolio-writing process affect students’ portfolio-writing ability, as guided by CAE marking descriptors?

a) What changes are made from version 1 to version 2?

b) What changes are made from version 2 to version 3?

3 Scope of the study

Portfolio is used in various fields, but within the frame of this paper, the researcher only approaches it in educational field

The population of the study are second-year-fast-track students whose writing course includes the use of portfolio

4 Methods of the study

4.1 Research design: Multiple case study

design

In this study, the researcher applied the

multiple case study design to seek answers to the

two research questions because it is an approach,

which - according to Baxter and Jack (2008) - can

ensure that the issue is not explored through one

lens, but rather a variety of lens This ‘allows

multiple facets of the phenomenon to be revealed

and understood’ (Baxter & Jack 2008, p.544)

4.2 Sampling

Because of the researcher’s limited time and

human resources, the study applied purposive

critical-case sampling process, in which the

researcher chose a small number of important

cases who ‘displayed the issue or set of

characteristics in their entirety or in a way that was highly significant for their behavior’ (Cohen, Manion & Morrison cited in Vu 2012, p 16)

Motivation, attitudes and learning strategies

are three discernible determiners affecting learning progress (Gadner 1968; O’Malley & Chamot 1990) Second-year students are assumed

to have developed certain learning strategies in this skill after more than three terms of exposure Additionally, the three aforementioned indexes are

shown more significantly clearly among fast-track students than main-stream ones Therefore, for the convenience in case selection, second-year-fast-track students were targeted as the population,

which helped increase the validity and reliability

of the data amassed

The following diagram illustrates the participant selection process:

Records of students’ marks played a pivotal

role in singling out students who exhibited

clearest improvements in writing results of the

two successive semesters

Questionnaire for case selection involves (A)

Assessments of attitudinal and motivational

variables associated with writing learning and (B)

Writing strategies (adapted from Gardner, 1985)

The case choosing decision was reaffirmed based

on the results of questionnaire for Motivation &

Attitude and Learning Strategies

Diagram 2.3: Scores for motivation & attitude and learning strategies

The diagram showed one common

characteristics among three cases - strong

motivation and positive attitude towards

learning writing Nonetheless, S2 and S3 appeared

to have relatively good learning strategies whilst

the other gained very low score for this category

(99) Understandably, there is more likelihood for

students with both strong motivation and positive

attitudes together with good learning strategies to

make progress in the process of doing the

portfolio Additionally, with the target shot at

learners demonstrating not-so-good learning

strategies, the progress could be clearly

highlighted

4.3 Data collection instruments

4.3.1 Semi-structured interview (RQ 1)

Interviews were utilized in both the case

choosing stage and the data collection stage to

answer Research Question 1

Interview is rightfully seen as one of the most

widely used and most fundamental research

technique since it allows the researcher to

investigate the phenomena that are not directly

observable such as learners’ self-reported

perceptions or attitudes Furthermore, researchers

could elicit additional data if initial answers are

vague, incomplete, off-topic, or not specific

enough However, the researcher was also aware

of some caveats of interviews Therefore,

face-to-face retrospective semi-structured interview

was employed with concerted attempts being made to revive students’ remembrance of their previous semester’s portfolio by several techniques (letting them see the writing syllabus and their own portfolio again as well as familiarizing themselves with the criteria to assess writing ability) Clarifications were also given and questions could be asked right away if necessary The interview guidelines for the students followed strictly the skeleton of the framework for document analysis There were four main

sections: (1) Content; (2) Communicative

achievement; (3) Organization; and (4)

Language In each section, there are specific

discriptors which are synthesized from the CAE marking scale and the Glossary for this scale

4.3.2 Document analysis (RQ2)

Three portfolios of three student cases were chosen to investigate the effect of portfolio on students’ writing performance For each portfolio, the researcher randomly chose two writing pieces

of different genres to analyze

Document analysis framework: In order to

ensure the objectivity when analyzing writing pieces in students’ portfolio, there was also a bad

need to follow CAE marking rubric

4.4 Data analysis method and procedure

4.4.1 For interviews: Qualitative analysis strategies

The method and procedure of analyzing data

collected from interviews are shown in the

diagram below:

4.4.2 For document analysis: Qualitative

analysis strategies

Following is the diagram visualizing the qualitative analysis method and procedure of students’ writings

5 Findings

As the ultimate goal of case study is to

understand the complexity of a case in the most

complete way possible, each case was analyzed

and discussed respectively instead of in cluster,

guided by the CAE marking descriptors This

helps increase the possibility of attaining the

richest understanding of a case

Moreover, it should first be noticed that several

aspects in the CAE marking scheme for writing

were not reported to be improved much, as a result

of the portfolio writing process Therefore, the

following presentation of data analysis and discussion pays particular attention to the writing abilities which have gone through the most considerable progress

5.1 Research question 1

The answer to this question was received from

interview data

5.1.1 Case 1

In case 1’s perception, she made almost no or just marginal improvements in such writing

abilities as target reader, communicative

achievement, organizational pattern and

grammar She explained:

Before assigning any task, theory on specific

genres such as brochure or leaflet was delivered

by our teacher and samples are analyzed

thoroughly Moreover, when I come home,

before writing any task, I always revise the theory

Hence, there can’t be any misleading in the

content or mistake in the conventions of the

communicative task from the beginning (S1.21)

Moreover, case 1 self-assessed herself as

competent in grammar, so rarely could

grammatical mistakes occur Hereinafter, the

researcher focuses on only the most radical

changes in terms of cohesive devices and

vocabulary

Cohesive devices

Positive changes could be seen through the

ability “to link two simple sentences to make a

complex or compound one” (S1) She added,

when writing frequently, she tended to use

cohesive devices more regularly and correctly

From mistakes in former versions, she could draw

valuable experience for her own

Vocabulary

This participant expressed a tremendous

satisfaction with her vocabulary derived from

conducting the portfolio There was a big

improvement in terms of vocabulary range,

appropriacy and accuracy S1 felt deeply

contented when talking about what she could

learn:

It [portfolio writing process] offers me a good

chance to exploit extra resources such as

dictionaries, thesauri, internet, etc and my friends’

comments In fact, I could learn and apply a wider

range of synonyms, antonyms, collocations and

low-frequency lexis also! [S1.28]

5.1.2 Case 2

Among the three cases, S2 was the one to

reflect the most obvious progress Except for some

aspects with no or little improvements, she

claimed that the rest were positively adjusted not only from version to version but also throughout

the portfolio writing process

Target reader and coverage of main points

Firstly, S2 reported “Portfolio writing process

sharpened my critical thinking in choosing the

target reader” She admitted that sometimes the

readers of her writing were not focused in first versions Nonetheless, with constructive comments and suggestions from the teacher and her classmates, she could shoot target at particular readers more precisely

Secondly, for coverage of main points, S2

reflected radical improvements in the quality of the main points Her first draft occasionally contained vague arguments Notwithstanding, when it came to latter versions, such weaknesses were overcome to make the content more lucid

As a result, the final versions were always the ones she was most contented with

Cohesive devices

Among all the aspects, the competence in using

cohesive devices was perceived to have the most

substantial enhancement in S2’s case

In the previous previous semesters, I had a lot

of difficulties and shortcoming in using linking words I did not often use connectors among sentences as well as paragraphs, so the text sounds desultory Through the portfolio writing process, I gradually pay more attention in using such devices

to enhance the coherence and cohesion [S2.37] Not only linking words, but also grammatical devices were reported to be paid greater attention

Positive changes could be seen through the ability

“to repeat key words a certain times to highlight the writing topic”

Vocabulary and grammar

Despite showing a keen appreciation of the teacher and peers’ feedback and how much she

learned from them, S2 claimed that “sometimes

comments from friends are vague, too general or

even wrong”, so she needed to consider these very

carefully before revising

5.1.3 Case 3

In general, S3’s self-assessment of her writing

ability as a result of the portfolio writing process

more or less shared the same features with S2’s A

lot of improvements were perceived to be present

in terms of coverage of main points, format and

register, cohesive devices, vocabulary and

grammar

5.2 Research question 2

5.2.1 Case 1

Throughout the content analysis of her

portfolio, the improvement in coverage of main

points, format and function, vocabulary and

grammar were reflected vividly, which will be

thoroughly discussed in the following part

Coverage of main points

Although the writer claimed that there were

only few adjustments in the writing content, the

changes in the three versions of leaflet, by

contrast, showed ample progress

Firstly, in latter drafts, the WHAT and WHERE parts were combined into one with the restructuring of content, which increased the logicality in comparison with version 1 Secondly,

in the first version, the HOW part seemed rather verbose with four complicated steps to access the learning resource centers Thanks to her self-correction, they were shortened into three more lucid ones Furthermore, contact information, which was absent from the first and second drafts were timely added in the last version From version

2 to version 3, there was almost no change

Vocabulary

Generally, S1’s language ability underwent significant betterments as she reported In former versions, the writer often made mistakes in word forms, word choices and expressions though such slips had very little impact on communication However, adjustments were made to correct the

errors and diversify the vocabulary, which can be

seen as follows:

expressing…

damage

(Note: sign “X” means the word/ phrase/ sentence does not appear in the version)

And here comes the changes in grammatical items

relationship going on

relationship going on

…

( Note: sign “X” means the word/ phrase/ sentence does not appear in the version)

From the above two tables, it is conspicuous

that all existent mistakes in version 1 were not

corrected after receiving peers’ comments In fact, some suggestions were worth considering and if

they had been fully taken into consideration,

correction could have been seen right in the

second version As stated earlier in answers for

RQ1, S1 often thought that she hardly ever made

such kinds of errors and sometimes peers gave

unconvincing comments This could partly

explained for the late correction until the last

version, after the teacher provided her feedback on

both her writings and her viewpoints on peers’

comments

5.2.2 Case 2

Among the three cases, the most outstanding

progress can be in S2’s writing pieces Except for

such aspects as genre, format, register and

cohesive devices, improvements were clearly

exhibited in the other ones namely coverage of

main points, organizational pattern, function,

vocabulary and grammar

Coverage of main points

In analyzing case 2’s writings, it can be seen

that positive changes were made in the content of

both the essay and critical review Interestingly,

all the changes found in the writings concured

with what the informant reported in the interview

Taking the essay about motivation for

example, obvious changes were witnessed in the

very first paragraph of the body part In the first

draft, to show her agreement with the importance

of university qualifications, she used a quotation

from Theore Rosevelt However, according to a

peer’s comment about the need to “clarify the

saying to make it relevant to what was being

mentioned”, the writer added more details to

support the main idea in the second draft:

According to Theodore Roosevelt, the great president of America, “A man who has never gone to school may steal a freight care, but if he has an university certification, he may steal the

whole railroad This means that when a

person is educated at university, they will have certification which convinces employers about his abilities

(S1’s writing 1 – version 2)

However, when it came to version 3, with her reconsideration, the writer used other reasonings

in placement for the above quotation, making her argument a lot stronger

…After spending four or five years finishing tertiary education, if they study and practice well, they will have both outstanding diploma and excellent commands of computer as well as foreign language As a result, plenty of good job opportunities will be opened to them, and getting a desirable position in career with high salary is only in their touch

(S1’s writing 1 – version 3)

Besides, in the first two drafts, some extreme

expressions such as “young people” and “most

students” were also replaced by “many young

people” and “most students” in the last version

Organizational pattern

For this criterion, changes from version 1 to version 2 were made based on peer’s comments Meanwhile, changes from version 2 and version 3 were made based greatly on the teacher’s feedback Specific examples are shown in the following table:

Writing 1 –

Motivation

When attending university, the youth can study a lot of academic

history course, philosophy course

youth can study a lot of academic courses like geography, history or philosophy courses

And, studying at university is a

university is a good way…

Unchanged

Writing 2 –

Critical

review

The first and the last text are

Unchanged The first and the last text are

Surprisingly, like vocabulary, such

grammatical mistakes as article, subject-verb

agreement were not corrected until the last

version Specific changes are also synthesized below:

Writing 1 –

Critical review

“Adulthood”

Unchanged Chapter 4 entitled “Adulthood”

The major weaknesses of this

assessment

Unchanged The major weaknesses of this

assessment

data

Unchanged There are no accurate and reliable

data Writing 2 –

Motivation

5.2.3 Case 3

Compared to S1 and S2, S3’s actual changes in

her writings were exactly the same as what she

perceived it

Coverage of main points and target reader

S3 showed adjustments in not only supporting

ideas but also in the coverage of main points At

first, although content points were included with

some development of the topic, and target reader

was only generally informed some irrelevances

and omissions were still presented The revision

saw noticeable improvements after peer’s

feedback was taken into consideration

To be more specific, with the essay about love,

in the first draft, the outline was not clear and

supportive to the topic Basically, the essay

discussed apology in three kinds of relationships:

love between friends, family members and lovers Nonetheless, as one student remarked:

“…your ideas are quite confusing and not persuasive enough All the three types of relationships mentioned are in common of reasoning why it’s hard for people to say sorry and why they should say sorry Furthermore, the reasons are too general and can be applied

to every situation, not just the specific case Therefore, you should not state your ideas as different types of different relations.”

Having pondered over this comment, in version

2, S3 substituted almost all the main points, each

of which was adequately supported with clear and well-elucidated sub-ideas The new outline can be viewed as follows:

For: Saying sorry is one of the best ways to protect your relationship from breaking

- It is hard to apologize because of self-esteem and ego

- However, once overcoming such obstacles, happiness will come and sadness will disappear

Against: However, apology can be meaningless if we keep making mistakes and then

just say sorry

- Admitting the fault and trying to fix it is good

- Nevertheless, without improvements, the apology becomes untrustworthy

- Hence, sooner or later, the loving relationship will come to an end

Obviously, the content was enormously enhanced to cement the main arguments Besides,

the last sentence in the introduction of version 1

which sounded confusing and unnecessary was

omitted after revision

Love is the most invaluable gift that god gives

us Its strength can help people overcome

difficulties However, it is so fragile that even a

small mistake can break it Therefore, apology

is very important to maintain love as someone

has said “Love, as in enabling a relationship to

survive, is all about being able to say sorry”.

Honeyed words are sometimes not as

powerful as apologies

(S3’s writing 1 – version 2)

6 Discussion

In answering research question 1 and 2, the

research yielded significant findings In general,

the research revealed several correlations between

the students’ self-assessment on their writing

ability and the actual progress revealed through

document analysis This is known as positive

changes in terms of content and vocabulary It is

true to the students’ self-report that changes were

made based on peers and teacher’s comments as

well as self-correction However, interestingly,

there also existed discrepancy in cases’ response

and their actual progress in writing One one hand,

although students reported no or little

improvements in such aspects as content,

grammar, format and function, there were in

fact more positive changes than students

themselves self-assessed This means sometimes

the underlying effect of portfolio writing process

was not recognized by the students On the other

hand, for cohesive devices, unlike what all

participants self-assessment, not many changes

could be identified when comparing the first

version to the second and third one The fact is

that all three cases did a good job in using

cohesive devices from the first version, so it is

understandable when there were little

modification in latter versions Accordingly, it can

be inferred that the portfolio writing process not

only positively affected the writing ability from

version to version but also through the process With all the factors considered together, the findings of the present research more or less shared the same features of findings from other studies in the local field The results supported the attempts of Coffin et al (2003) and Deane (2013) that students’ practical writing skills were enhanced considerably through portfolios, mainly based on teachers’ and peers’ comments However, what makes the extant research different from others is that the researcher could (1) point out the discrepancy between students’ self-assessment and the actual progress, (2) identify the progress in students’ writing ability thanks to their self-correction of mistakes also

7 Conlusion

In general, the study explored the impact of portfolio writing process on students’ practical writing abilities Through the in-depth analysis of the data collected from the interview and the document analysis, fruitful findings for two researcher questions are summarized as follows:

Research question 1

On one hand, from the participants’

self-assessment, such writing abilities as coverage of

main points, cohesive devices and vocabulary

were reported to witness the most discernible progress Generally, most of the writing ability enhancements were made thanks to the teacher’s and peers’ comments Nevertheless, some were self-correction after re-reading and reconsideration Notably, as being reported, peers’ feedback and suggestions did not concur with the writer’s view at times Thus, some changes were not made after comments On the other hand, the least changed aspects fell into less tangible aspects

like target reader, reader’s attention, genre,

register, function and organization pattern Research question 2

The answer to research question 2 helped to verify the results of research question 1 The analysis of three versions of the students’ writing