04051001848 a comparison between teacher’s assessment and students’ self assessment in an ielts writing course = tìm hiểu sự khác biệt giữa việc Đánh giá của giáo viên và sự tự Đánh giá của học sinh trong khóa học viết ielts
INTRODUCTION
Statement of research problem and rationale for the study
Assessment is essential in the educational process, as it provides valuable information that informs decisions regarding students, curricula, programs, and educational policies (Cheng, Rogers, & Wang, 2008; Nitko).
Self-assessment is a powerful tool that not only measures learner understanding but also fosters active engagement in the learning process It empowers students to take control of their education by providing insights into their abilities and progress, allowing them to compare their skills with the course objectives Research indicates that accurate self-assessment enhances learners' confidence and helps them adopt effective strategies to meet task requirements.
Self-assessment in language learning enables ESL and EFL learners to understand their achievements and proficiency levels (Oscarson, 1997) Effective learning is positively linked to students' active engagement, autonomy, and motivation, which can be fostered through self-assessment Although self-assessment is not traditionally included in formal courses, students often utilize it informally (Boud, 1995) Instead of relying solely on comprehensive diagnostic tests, teachers can directly inquire about the challenges students perceive in their learning (Harris & McCann, 1994) Therefore, incorporating self-assessment is recommended for monitoring progress and facilitating improvement.
The primary goal of assessment is to enhance student learning; however, engaging students in the assessment process remains challenging Despite the clear advantages of self-assessment, it is often underrepresented in assessment literature (Wragg, 2001) In the context of Vietnamese EFL classrooms, which predominantly focus on evaluating language competence through testing and summative assessments, there is limited opportunity for classroom-based assessment activities This restriction hinders the practice of self-assessment among students Consequently, this study aims to explore how English language learners engage with and perceive self-assessment related to specific skills, as well as to evaluate the accuracy of their self-assessments compared to teacher evaluations.
This research focuses on writing skills, which are believed to improve significantly through self-assessment, complemented by peer assessment and teacher feedback (Andrade et al., 2008) Writing tests are often the most challenging for test takers (Uysal, 2009), as evidenced by the low average band score of 5.7 achieved by Vietnamese students in the IELTS exam in 2019, the lowest among the four tested skills (IELTS, n.d.) This underperformance may stem from inadequate preparation, highlighting the importance of students' understanding of the writing test requirements outlined in the writing band descriptors Effective self-assessment practices that align closely with teacher evaluations can serve as reliable indicators of student progress and exam readiness Furthermore, it can be inferred that students with higher abilities tend to demonstrate greater accuracy in their self-assessment skills.
The research was conducted in a Hanoi university Within the framework of this study, it aims to answer the following questions:
1 To what extent is there a correlation between teacher’s assessment and students’ self-assessment in IELTS academic writing skill at a university in Hanoi?
2 How does the correlation between the teacher’ assessment and students’ self- assessment differ among the two groups of students (intermediate and advanced)?
3 What is the students’ perception of the effects of self-assessment skill in their English writing learning process?
Scope of the research
This study employs a mixed-methods design, incorporating both quantitative and qualitative approaches The participants include 30 second-year university students who engaged in a self-marking process following a writing test Additionally, semi-structured interviews were conducted with 10 students, representing over one-third of the population, which allows for generalization of the findings.
Significance of the study
This study aims to explore the self-assessment practices of writing students, focusing on self-grading, the accuracy of self-assessments compared to teacher evaluations, and students' attitudes towards the benefits of self-assessment Upon completion, the research will provide valuable insights into the current state of self-assessment in EFL classrooms, particularly highlighting Vietnamese language learners' perceptions The findings will contribute to classroom-based assessment, emphasizing the importance for English teachers to effectively utilize self-assessment for the benefit of language learners.
Method of the study
This study utilizes a mixed-methods approach, incorporating both quantitative and qualitative designs The quantitative analysis focuses on data derived from writing assessments conducted by students and teachers, while qualitative analysis is based on interviews Two primary data collection instruments are employed: a writing test and semi-structured interview protocols The writing test data addresses the first two research questions concerning the correlation between teacher assessments and students' self-assessments In contrast, the interviews provide insights into students' perceptions of the effectiveness of self-assessment in enhancing their English writing skills.
Organization of the paper
This research consists of five chapters presented as below:
Chapter 1 – Introduction – presents the rationale, aims, scope, significance, method and structure of the study
Chapter 2 – Literature review – provides the definition of key terms as well as the framework of the study and includes reviews on related studies to the research topic
Chapter 3 – Methodology – gives information about the setting of the research, sampling method, participants’ information, data collection instruments, and data analysis methods of the research
Chapter 4 – Findings and discussion – presents, analyzes and discusses the research findings
Chapter 5 – Conclusion – provides the summary of findings; implications, limitations of the study; and suggestions for further research.
LITERATURE REVIEW
Dynamics in the Field of English Language Testing and Assessment
Evaluation, assessment, and testing have been essential in English Language Teaching (ELT) for over 50 years, significantly influencing language policy, teaching methods, curriculum design, teacher development, and second language acquisition Conversely, ELT has undergone radical changes, challenging standardization through new genres, multimodalities, world Englishes, and English as a lingua franca Additionally, the growing impact of sociocultural theory and critical perspectives has reshaped language learning constructs This evolving landscape has sparked increased research interest in assessment practices, focusing on validity, reliability, authenticity, practicality, and the washback effect on educational stakeholders, all aimed at aligning with contemporary educational goals in ELT.
First and foremost, some paradigm shifts in second language teaching have caused fundamental changes in assessment field Table 1 below compares the old and new issues in ELT
Table 1 Old and new paradigm (Tosuncuoglu, 2018)
Old teaching foreign language paradigm New teaching foreign language paradigm Focus on language
Focus on communication Integrated skills
Learner-centered Emphasis on procedure
Tests only test Tests to teach
The current educational transformation emphasizes a shift from teacher-centered to learner-centered approaches, highlighting the importance of active student participation in the classroom This transition is supported by White's (1988) progressivism philosophy, which prioritizes analyzing students' needs and interests Additionally, there is a growing focus on the learning process rather than solely on the final product.
Understanding the learning process offers valuable feedback for both learners and instructors, facilitating ongoing improvement Additionally, the relationship between testing and teaching has evolved from being distinct tasks to a continuous cycle, with testing now often referred to as "assessment."
Assessment encompasses both teacher-led tasks and student-initiated activities, aimed at delivering valuable feedback that facilitates essential adjustments in teaching and learning processes (Black & William).
Assessment activities can occur between teachers and students, among peers, or even through self-assessment, as highlighted by Cheng and Fox (2017) This paper emphasizes the crucial role of learners in the assessment process.
The shift from traditional to alternative assessment
In the traditional exam-focused culture, testing has been the primary method for assessing English language proficiency However, contemporary assessment practices recognize testing as just one approach among various alternative assessments Consequently, assessment is now viewed as a broader concept encompassing multiple methods (Kulieke et al., 1990).
Alternative assessment is increasingly valued as forms of classroom-based assessment integrated into the teaching and learning process (Davison, C., &
Alternative assessment encompasses various approaches aimed at evaluating students' problem-solving abilities through authentic tasks relevant to real-world issues Simonson et al (cited by Semire Dikli, 2003) identified three key methods: authentic assessment, performance-based assessment, and constructivist assessment In contrast, Reeves (2000) introduced cognitive assessment, performance assessment, and portfolio assessment Despite their differing frameworks, all these methods emphasize the importance of measuring student growth and performance, allowing learners to showcase their knowledge and apply higher-order thinking skills effectively.
Below is the description of the shift from traditional to authentic assessment
Table 2 Traditional and Authentic testing (Tosuncuoglu, 2018)
Performing an assignment Real-life
Traditional assessment primarily involves testing methods such as multiple-choice, true/false, short answer, and essay formats, each with its own strengths and weaknesses regarding objectivity, practicality, and cognitive demand In contrast, alternative assessment encompasses a variety of procedures, including checklists, journals, logs, and multimedia recordings, along with self-evaluations and teacher observations (Huerta-Macías, 1995) Additionally, strategies for alternative assessment may involve portfolios, conferences, diaries, self-assessments, and peer assessments.
Traditional assessments are often viewed as indirect and inauthentic due to their standardized nature These assessments typically focus on a single performance instance, are speed-based, and rely on norm-referenced scoring (Bailey, 1998) Additionally, Law and Eckes highlight the limitations of such assessment methods.
Traditional assessments, as noted by (1995), are single-occasion tests that measure students' abilities at a specific moment, failing to provide insights into their progression or the challenges they faced during the test Bailey (1998) highlighted the lack of feedback for learners, emphasizing that the assessment process is often individualized and disconnected from real-world contexts Furthermore, Law and Eckes (1995) argued that most standardized tests primarily evaluate low-order thinking skills, a view supported by Smaldino et al (2000), who pointed out that these assessments overly focus on memorization and recall, reflecting a lower level of cognitive skill.
Alternative assessment fosters higher-order thinking skills by providing students with opportunities to perform tasks, receive feedback, and grow If a student struggles with a task at one moment, they have additional chances to showcase their abilities in different contexts Furthermore, this approach allows teachers to evaluate students' strengths and weaknesses across various areas and situations (Law and Eckes, 1995).
Elliott (1995) emphasized that enhancing the effectiveness of alternative assessment can be achieved by incorporating key guiding principles, with self-assessment being a crucial component.
1 Selecting assessment tasks that are clearly aligned or connected to what has been taught
2 Sharing the scoring criteria for the assessment task with students prior to working on the task
3 Providing students with clear statements of standards and/or several models of acceptable performances before they attempt a task
4 Encouraging students to complete self-assessments of their performances
5 Interpreting students’ performances by comparing them to standards that are developmentally appropriate, as well as to other students’ performances.
Types of Assessment
In assessment categorization, common factors include the purpose, measurement methods, formality, class-based assessment, implementation procedures, timing, and the role of assessors.
The description of each classification is illustrated in the following tables
Classifying purpose Summative Assessment Formative Assessment
Assessment tools such as tests, assignments, and projects are essential for evaluating the extent to which students have achieved learning objectives These tools help measure student learning, academic success, and skill development at the conclusion of an instructional period (Cheng, L., and Fox, J., 2017).
Formative assessment is typically utilized throughout the learning process to provide immediate feedback on students' progress and to highlight specific areas that require further improvement (Little and Perclová).
Norm-based Assessment Criterion-based Assessment
It reveals the level of achievements among students and makes comparison of students’ rank in a group
It compares achievement of a student with a set of curriculum requirements
Assessment is conducted in a structured and systematic manner, often involving written documents like tests, quizzes, or papers The primary goal is to provide a numerical score or grade that reflects students' performance.
It can be done by just asking students in a group or as individuals (Wragg, 2001)
It may occur during observation, inventories, participation, peer and self- assessment, and discussion performance
It provides evidence for use by students and their teachers to decide students’ current learning level, their next goals and best ways to reach the targets
It happens after learning has occurred to see whether learning has happened It reveals students’ learning status at a particular time
It focuses on metacognitive skills and provide students with assessment strategies such as self- and peer-feedback in order to assist them in becoming lifelong learners
The study emphasizes the significance of self-assessment as an informal evaluation method, integral to the course through training sessions This approach aligns with assessment as learning, as it familiarizes students with expected writing standards, guides them in setting and monitoring their learning goals, and encourages the development of diverse strategies to achieve these goals Consequently, this practice may enhance students' writing outcomes in examinations and positively influence their overall learning journey.
Self-assessment- a significant change in the role of assessor
2.4.1 Position in the field of assessment
Alternative assessment, as outlined in section 2.2, serves to thoroughly evaluate learners' learning processes, with essential data derived from their classroom performance Additionally, the fourth principle encourages students to take an active role in the assessment process, fostering self-regulation and participation in evaluating their own work This involvement enhances their self-awareness regarding the steps needed for successful target language acquisition.
Self-assessment is a key component of alternative assessment practices, emphasizing the importance of learner participation According to Black and William (1998), learners should actively engage in assessment processes, a view supported by Oscarson (2009) Birjandi and Hadidi Tamjid (2012) highlight that optimal learning conditions and student achievements are fostered through active involvement in all educational stages, aligning with cognitive and constructivist learning theories (Cobb 1994; von Glasersfeld 1995).
The growing interest in second-language self-assessment and peer assessment is largely driven by the shift towards learner-centered approaches in education, as highlighted by researchers such as Bachman (2000) and Hamayan.
Self-assessment in language learning is rooted in the constructivist perspective, which posits that knowledge is constructed by learners rather than simply acquired (Von Glasersfeld, 1989, cited in Kim, 2005) This approach emphasizes that each learner creates their own understanding (Parviz Birjandi & Nasrin Hadidi Tamjid, 2012) As a crucial element of the evolving assessment practice known as Assessment for Learning (AfL), self-assessment reflects the growing impact of socio-cultural theory in English Language Teaching (ELT) This theory suggests that learning is contextual and shaped by social interactions (Lave & Wenger, 1991; Vygotsky).
1978) The learning process is first mediated by others and then goes through the processes of individual’s active engagement in internalization (Rogoff, 2003)
From a sociocultural viewpoint, Assessment for Learning (AfL) highlights the significance of students' social interactions with teachers and peers, as well as their individual actions in the learning process In the classroom, AfL practices incorporate peer and self-assessment to enhance student engagement and understanding.
Self-assessment is a multifaceted concept that involves students actively engaging in the evaluation of their own work According to Boud (1991), it entails students identifying relevant standards and criteria to assess their performance Andrade and Valtcheva (2009) further emphasize that self-assessment serves as a formative assessment tool, fostering students' reflection on their work quality and enabling them to judge their accomplishments against established criteria, ultimately guiding them in making necessary revisions.
Georgiou and Pavlou (2003) highlight that self-assessment empowers students by enhancing their understanding of assessment criteria, allowing them to feel more involved in their evaluation process For self-assessment to be effective, students must clearly understand the performance expectations and the criteria by which they will be assessed, necessitating the integration of criterion-based procedures into the self-assessment process.
2.4.3 Self-assessment in comparison with Teacher and Peer Assessment
The assessment process has traditionally been viewed as the responsibility of instructors, who make decisions regarding student progress and achievement However, with the shift towards a learner-centered approach, there is an increased emphasis on student participation in learning activities, empowering them to take on significant roles as assessors This transition in the role of assessors is illustrated in Table 4 (Thawabieh, 2017).
Table 4 Comparison between Students’ Self-Assessment and Teacher-Assessment
Teacher tells and student listens
Teacher is uncertain of the student ability to assess his work
Teachers and students are co-learners Teacher and student together use formative assessment
Teacher believes that self-assessment is a learnable skill
The transition highlighted in the table redefines learners as active participants in the learning process rather than passive recipients This shift necessitates the implementation of ongoing classroom-based assessments, moving beyond traditional evaluations at the course's conclusion Consequently, teachers can maintain comprehensive records of students' progress while empowering learners to take charge of their educational journey through self-assessment and peer assessment.
Peer assessment, as defined by Falchikov (1995), involves groups evaluating and rating their peers, potentially without prior agreement on assessment criteria (F Dochy, M Segers & D Sluijsmans, 1999) This process may utilize pre-designed rating instruments or checklists to guide evaluations While it encompasses peer marking and feedback, it is crucial to view peer assessment as an integral part of the learning process that fosters skill development (Somervell, 1993).
In 1999, it was suggested that students, by observing their peers during the learning process, often gain a deeper understanding of their classmates' work compared to their teachers Consequently, peer assessment can play a significant role in enhancing the self-assessment process for students.
Engaging students in self-assessment is crucial for improving learning outcomes and fostering academic self-regulation, which involves the ability to monitor and manage one's own learning (Pintrich, 2000; Zimmerman & Schunk, 2004) Additionally, self-assessment enhances learners' abilities to evaluate their strengths and weaknesses, thereby promoting the development of self-regulation (Oscarson, 1997, cited in Brindley 2001).
Self-regulation enables students to establish personal goals, create adaptable plans for achieving them, and track their progress effectively By understanding their objectives, students can better manage their learning and evaluate their advancement toward these goals.
Research indicates that students who enhance their self-regulation skills tend to achieve higher academic performance According to Schunk (2003), these self-regulated learners not only learn more effectively but also outperform their peers who lack these skills.
2.4.4.1 Self-Assessment and Learners’ Autonomy
Learner autonomy, as defined by Henri Holec (1981), refers to the capacity to take control of one's own learning process This includes being responsible for making decisions about various aspects of learning, such as setting objectives, defining content and progression, choosing methods and techniques, and effectively monitoring the learning process in terms of rhythm, time, and place.
Applying Criteria-Referenced Self-Assessment in Writing
Criteria-referenced self-assessment allows students to evaluate their work against clearly defined standards, enabling them to make necessary revisions (Andrade & Valtcheva, 2009) However, students often neglect this process due to insufficient conditions Goodrich (1996) highlights the essential requirements for students to successfully engage in self-assessment.
• Awareness of the value of self-assessment,
• Access to clear criteria on which to base the assessment,
• A specific task or performance to access,
• Direct instruction in and assistance with self-assessment,
• Cues regrading when it is appropriate to self-assess, and
• Opportunities to revise and improve the task or performance
Most classroom conditions for effective assessment can be achieved by following a structured approach For example, providing students with a clear assessment rubric meets the requirement for transparent evaluation criteria A well-designed rubric aids teachers in grading and allows students to self-assess their work based on defined quality levels, from excellent to poor (Andrade, 2000; Arter & Chappuis, 2007; Goodrich, 1997) To create an effective rubric, assessors should identify common student mistakes and outline the criteria needed to achieve specific marks This clarity equips students with essential information about the task and sets clear learning objectives for producing high-quality work.
2.6.1 Students’ Attitude towards Using Rubrics-based Self-Assessment in
Writing attitude encompasses a writer's emotional response to their work, which can vary from positive to negative (Graham, Berninger & Fan, 2007) It reflects not only personal feelings about one's writing quality, such as believing "I think my writing is good," but also the perception of external feedback, like the belief that "I think my instructor reacts positively to my writing" (Hoda Sarkhoush, 2013).
Research by McKenna, Kear, & Ellsworth (1995) indicates that students who possess a positive attitude towards writing are more likely to actively participate in writing tasks than those with negative attitudes Additionally, Brown (as cited in Hosseini et al., 2013) suggests that learners with negative attitudes may experience a lack of motivation, hindering their performance on tests, while those with positive attitudes can enhance their learning progress.
A study by Hoda Sarkhoush (2013) revealed that Iranian EFL learners with positive attitudes towards writing demonstrated significantly better writing performance compared to those with negative attitudes This finding aligns with research by Hosseini et al (2013), which established a correlation between positive attitudes and higher IELTS test scores.
2.6.1.2 Attitude towards Self-assessment in Writing
Andrade and Du (2007) found that students perceive self-assessment in writing as valuable; however, they require support and practice to fully benefit from the process.
Du (2007) reported six important findings related to students’ attitudes towards self- assessment practice
Students who frequently engaged in self-assessment in writing developed more positive attitudes towards the process When informed about teacher expectations, they felt more capable and motivated to self-assess effectively It was only through careful and formal implementation of self-assessment that they recognized the significance of criteria-referenced self-assessment for proofreading, guiding revisions, and reflecting on their understanding Self-assessment was seen to offer numerous benefits, including enhanced focus on key task elements, improved material sourcing, better identification of strengths and weaknesses, and increased motivation, confidence, and mindfulness, potentially reducing writing apprehension However, many students acknowledged inconsistencies in their self-assessment practices, indicating insufficient engagement Additionally, Andrade and Du (2007) highlighted a disconnect between teacher expectations and the quality standards students set for themselves, with some students admitting they primarily self-assessed based on teacher criteria rather than their own.
Previous related Studies on Self-assessment in Writing
There were a couple of typical examples of studies on students’ self- assessment in EFL writing, especially in IELTS writing
In a study by Oscarson (2009) involving Swedish upper secondary students, participants self-assessed their written assignments and were later interviewed about their experiences The results indicated that students' self-assessments were generally accurate when compared to teachers' grades, although they were more critical of their spelling and grammar than other skills such as sentence structure and vocabulary Notably, students often underestimated their performance relative to the researchers' evaluations Both students and teachers expressed positive attitudes towards self-assessment activities in EFL writing, recognizing their effectiveness in enhancing language skills and competence.
A study by Blue (1993) revealed that achieving alignment between students' self-assessments and teachers' evaluations in IELTS writing is challenging The research found that higher-scoring IELTS students often underestimated their English proficiency, while lower-scoring students tended to overestimate it Notably, the highest self-assessment score of 15.5 came from a student who only achieved a score of 4.5 in IELTS, highlighting a significant discrepancy.
In Vietnam, the use of self-assessment in English as a Lingua Franca (ELF) classrooms is quite restricted, primarily due to the lack of research on self-assessment practices in English language learning, especially concerning ELF writing skills.
This chapter reviews the foundational literature relevant to the current study, beginning with an overview of the evolving landscape of English language assessment and its contemporary trends It then delves into self-assessment, exploring its definitions, purposes, implementation, advantages, and disadvantages, as well as the interplay between teacher assessments and student self-assessments The discussion continues with a focus on second language writing, particularly academic writing in the IELTS context Additionally, the chapter highlights the role of rubrics as effective tools for facilitating self-assessment in writing and examines students' attitudes towards their use Finally, it references several prior studies on self-assessment in writing.
METHODOLOGY
Restatement of research questions
This study explores the relationship between teacher assessments and university students' self-assessments in the context of an IELTS writing test, while also examining students' perceptions of self-assessment in the language learning process The researcher seeks to answer specific questions related to these themes.
1 To what extent is there a correlation between teacher’s assessment and students’ self-assessment in IELTS academic writing skill?
2 How does the correlation between teacher’ assessment and students’ self- assessment differ among the two groups (intermediate and advanced) of students?
3 What is the students’ perception of the effects of self-assessment skill in their English writing learning process?
The setting of the study
A study was conducted at a leading university in Hanoi, recognized for training foreign language teachers in Vietnam The institution offers specialized courses in two main areas: Language Teacher Education and Applied Languages The Language Teacher Education program provides pre-service training for teachers in various languages, while the Applied Languages program features six specializations, including Translation and Interpreting, Applied Linguistics, Management, Economics, Tourism, and International Studies.
Upon completing the course, students are anticipated to reach a language proficiency of level 5/6 in their first foreign language, corresponding to C1 on the Common European Framework of Reference, and level 3 in their second foreign language, equivalent to B1 on the same framework.
To fulfill language proficiency requirements, students must complete their initial foreign language practice courses within the first two academic years Following this, they will advance to the Teacher Language Education specialized program from their third year through to graduation.
The course materials for first foreign language practice are specifically designed, compiled and edited by university expert teams, which are categorized in two forms: Social and Academic English.
Participants
The study involved 30 second-year students majoring in English Language Teacher Education at a university in Hanoi, all aged 20 These students had completed a two-year English language practice course and taken the VSTEP examination, which is tailored for Vietnamese foreign language teachers They were expected to achieve a level of 5/6, equivalent to C1 on the Common European Framework of Reference (CEFR) However, their language competencies varied significantly; while some reached the required level 5/6, others only attained level 4/6, failing to meet the university's English competency standards According to the CEFR, those scoring level 4/6 are classified as intermediate learners, corresponding to B2 or a band score of 5.5-6.0 in IELTS.
Participants in the study were categorized into two groups: intermediate and advanced, based on their proficiency levels of 5/6 and above, which corresponds to C1+ or IELTS band scores of 6.5-7.5+ This classification facilitated the writing test and self-assessment practices, addressing the second research question effectively.
In the English language practice course, students learned various forms of English, including Social English for daily conversations and Academic English for formal education and higher studies English for Academic Purposes is particularly emphasized for IELTS preparation Although participants were familiarized with the IELTS writing format and received evaluations across different band scores, none had taken the official IELTS exam Throughout the course, students were encouraged to commit to their success and actively engage in completing their writing tasks and self-assessment.
Table 5 Information of the participants
Factors Number of the participants
Level of education 30 second-year students
Advanced level: 13 students Intermediate level: 17 students Total: 30 students
To accurately assess the correlation between teacher assessments and student self-assessments, the study selected a writing marker who is an IELTS expert with a high band score and direct experience in teaching academic writing An experienced teacher from the Faculty of English Language Teaching Education at a university in Hanoi, with a deep understanding of IELTS writing, was invited to participate With a rich background in IELTS instruction and an impressive score of 8.5 in IELTS writing, she is deemed highly qualified to evaluate student essays effectively.
Data collection instruments
This study utilizes a combination of quantitative and qualitative methods to explore students' self-assessment practices To address the first two research questions, the researcher compares teacher evaluations with students' writing scores from an IELTS test, using the IELTS writing band descriptors for analysis For the third question, semi-structured interviews are conducted to collect qualitative data, providing insights into students' perceptions of their self-assessment skills.
In this study, students’ writing performance was measured by an IELTS writing test Tests are considered a research instrument (Gass & Mackey, 2007)
An IELTS writing test includes two tasks: Task 1 and Task 2 In this study, the tasks in both IWT 1 and IWT 2 were selected by researcher in advance from Test
2 in the book Cambridge English IELTS Academic 13 by Cambridge University Press
In IWT 1, candidates must write a concise report of at least 150 words about a bar chart within a 20-minute timeframe They are evaluated on their skills in organizing, presenting, and comparing data effectively.
The chart below shows the percentage of households in owned and rented accommodation in England and Wales between 1918 and 2011
Summarise the information by selecting and reporting the main features, and make comparisons where relevant Write at least 150 words
Meanwhile, IWT 2 requires the test-takers to write an argumentative essay (in
Candidates are evaluated on their capacity to construct a clear and persuasive argument within a 40-minute timeframe, requiring a minimum of 250 words They must provide supporting evidence or examples that reinforce their main ideas while demonstrating flexibility and accuracy in their use of English The topic chosen relates to a common social phenomenon in daily life, making it easily understandable and relevant to Vietnamese students.
Write about the following topic:
Some people believe that nowadays we have too many choices
To what extent do you agree or disagree with this statement?
Give reasons for your answer and include any relevant examples from your own knowledge or experience
In language assessment research, scales are considered one of research instruments (Torres & Perera, 2010)
The criterion-related validity of the IELTS writing test is essential for assessing learners' target abilities, as highlighted by Hughes (2007) This validity is divided into concurrent and predictive types, with the IELTS writing test demonstrating strong concurrent validity by evaluating key aspects of writing skills (Uysal, 2010) These aspects include organizational ability, sentence construction, lexical range, grammatical accuracy, spelling, and punctuation, all of which are outlined in the IELTS writing band descriptors (Noor, 2020).
IELTS writing band descriptors assess four key criteria: task achievement (TA), coherence and cohesion (CC), lexical resource (LR), and grammatical range and accuracy (GRA), each rated on a 0–9 scale According to Shaw and Falvey, these criteria are essential for evaluating writing proficiency.
The creation of a scale and its corresponding descriptors is crucial for ensuring the validity of assessments (2008, p.12) The IELTS descriptors favor analytic marking over holistic approaches, allowing for more detailed observations, which reduces the risk of impressionistic bias from raters and avoids norm-referencing (Shaw and Falvey, 2008, p 37).
The descriptors were revised following several research studies, notably Kennedy and Thorp’s analysis of IELTS sample scripts (2007) and the Common Scale for Writing studies by Hawkey (2001) Two independent rater teams engaged in an iterative process of trialing and redrafting the descriptors Sample scripts were evaluated against the updated descriptors, supported by both quantitative and qualitative validation studies (Shaw and Falvey, 2004).
The IELTS official assessment criteria for Task 1 and Task 2 scripts are illustrated as following (UCLES, 2007)
Task Achievement in Task 1 assesses how well candidates meet the task requirements within a specified word limit, while Task Response in Task 2 evaluates their ability to present and develop an argument in response to the prompt, supported by knowledge, personal experiences, solid evidence, and concrete examples.
Coherence and cohesion are essential for ensuring clarity and fluency in writing Coherence involves establishing logical connections between sentences and paragraphs, while cohesion focuses on the effective use of cohesive devices, such as conjunctions, pronouns, and synonyms, to enhance the relationships between ideas.
(3) Lexical Resource: This criterion means the range of vocabulary the candidate uses and how he/she use them appropriately in response to the prompt
(4) Grammatical Range and Accuracy: This criterion is to assess the variety and accuracy of grammatical resources used at sentence-level
In this study, students are trained to effectively use a rubric for self-assessment by first being introduced to the IELTS assessing criteria in simplified language or Vietnamese They carefully read the band descriptors and identify differences among the four criteria across various band scores The teacher explains each criterion's requirements and illustrates them with writing samples, focusing on detailed descriptions for band scores 5.0 to 7.0 Students analyze provided essay examples using the marking rubrics to assign scores, calculate averages, and justify their evaluations with evidence from the writing Finally, the teacher reveals the actual scores, and students proofread and mark their own writing based on the four criteria.
Kvale (1996) emphasizes that interviews allow both participants and researchers to recall and reconstruct reported information, potentially leading to predictions based on responses This method provides the researcher with deeper insights into aspects that student writing test scores may not reveal Consequently, the researcher opted for face-to-face, in-depth interviews (Appendix 3) following students' self-assessments of their writing to explore their perceptions of the impact of self-assessment skills on their writing.
This study developed interview questions grounded in empirical research from the literature review, focusing on the positive effects and challenges of self-assessment, as well as students' attitudes towards self-assessment skills, particularly in writing.
To prevent misunderstandings and facilitate clear communication, the questions were presented in both English and Vietnamese The researcher recorded portions of the interviews, subsequently transcribing them for data analysis This qualitative data from the interviews provided valuable insights that clarified specific issues addressed in the questionnaires.
The researcher conducted semi-structured interviews with ten randomly selected students at varying writing levels to explore their perceptions of self-assessment skills in the learning process Utilizing a prepared set of two open-ended questions, the interviews aimed to investigate the impact of criteria-referenced self-assessment on students' writing and to determine the extent of its benefits.
Data collection procedures
Step 1: Prepare beforehand and ask for consent:
The researcher contacted 30 students and the teacher at FELTE to ask for their agreement to partake in the study and do the writing test as well as the interview
Step 2: Collect data from the writing test:
On a designated test date, 30 participants took the IELTS writing examination at ULIS, which lasted for 60 minutes After completing the writing task, they received 40 minutes of training on using IELTS Task 1 and 2 Writing band descriptors to self-evaluate their work Results were collected approximately 20 minutes later.
Step 3: Collect data from the interviews:
The researcher conducted interviews with ten randomly selected students who had previously self-assessed their writing skills Each student responded to questions using separate interview protocols, which were recorded for analysis The researcher then coded and analyzed the students' responses to create a quantitative profile reflecting their perceptions of the impact of self-assessment on their language learning.
Step 4: Collect writing results from the teacher
The teacher responsible for grading the students' writing papers announced the scores within two to three weeks Subsequently, the researcher gathered the marked writing papers, which included detailed comments and scores for each criterion and the total band score The overall writing scores were then classified into two groups based on the participants' levels: intermediate and advanced students.
Data analysis
The study employed a mixed-methods research design, incorporating both quantitative and qualitative approaches to analyze data through descriptive statistics and interpretative methods In phase 1, the analysis of data collected from students' writing test results involved three key steps.
Step 1: Examine the data: Once collecting the students’ writing test results, the researcher carefully examined whether there were enough 60 scores from 30 writing test papers marked by students themselves and the teacher The researcher started analyzing the data
Step 2: Analyze and report the data: Data from 60 writing scores were analyzed and synthesized Results from the students’ writing test papers will help answer the research questions 1 and 2 about the correlation between students’ self- assessment and teacher’s assessment Besides, the writing scores marked by teacher were also employed to make comparison with students’ self-graded scores from the questionnaires
The data will be processed in Excel which illustrates statistics in histogram and SPSS 26.0 software which shows the statistical relationship via Pearson correlation and T-test a Histogram
The study utilized Excel to process students' self-graded scores from two self-assessments alongside teacher scores, generating bar graph histograms to visualize the frequency distribution of these scores The histograms effectively displayed the occurrence of each score, highlighting the differences between students' self-assessments and teacher evaluations Additionally, the analysis revealed the mean and standard deviation discrepancies between the two sets of scores, providing insights into the correlation between them.
The data was analyzed using SPSS software version 26.0 to calculate Pearson Correlation statistics, which assess the relationship between two continuous variables This study specifically employed Pearson’s correlation coefficient to evaluate the correlation between students’ self-graded scores and teachers’ scores A high correlation indicates a strong positive relationship, suggesting that students possess a good ability for self-assessment The degree of correlation is presented in the accompanying table.
Table 6 Description of Pearson’s Correlation Coefficient (from Statistics
If the value is near ± 1, then it said to be a perfect correlation: as one variable increases, the other variable tends to also increase (if positive) or decrease (if negative)
High If the coefficient value lies between ± 0.50 and ± 1, then it is said to be a strong correlation
Moderate If the value lies between ± 0.30 and ± 0.49, then it is said to be a medium correlation
Low When the value lies below + 29, then it is said to be a small correlation
No correlation When the value is zero c T-Test: Paired-sample T-Test
A T-Test is a statistical test which is used to compare the means of two groups
This study examines the differences between students' scores and teachers' scores, utilizing the paired-samples T-Test due to both variables originating from the same population of students' writing papers The analysis aims to address the second research question by comparing these scores effectively.
Pair 1: Intermediate students’ self-graded scores and teacher’s scores
Pair 2: Advanced students’ self-graded scores and teacher’s scores
Three important sections which were described from the results of the paired T-Test are:
The t-value, denoted as "t," represents the ratio of the mean difference to the standard error of that difference This calculation is essential for comparing sample means against the null hypothesis As the differences increase, the absolute value of the t-value also rises, indicating a stronger deviation from the null hypothesis.
• “df’: the degrees of freedom for the test
In hypothesis testing, the null hypothesis posits that the true mean difference between two samples is zero, suggesting that any observed differences can be attributed to random variation The paired T-test employs the p-value to assess the statistical significance of the results, with a p-value less than 0.05 indicating that the null hypothesis can be rejected This rejection implies that the study's findings are statistically significant, meaning that the observed differences are unlikely to be due to chance Consequently, the alternative hypothesis, which asserts that the true mean difference is not zero, is accepted (Rouder et al., 2009).
The qualitative evidence gathered from interviews, alongside the quantitative data from the writing test, will provide insights into students' perceptions regarding the impact of self-assessment skills, addressing the third research question.
Step 1: Transcribe the recordings: After having listened to the recordings of ten students’ interview, the researcher transcribed them into written form for further analysis
Step 2: Choose the relevant data: In the transcripts of the recordings, only data which could help to answer the third research question were selected for in-depth analysis Next, the researcher identified patterns in students’ responses before illustrating the data in charts or table
Step 3: Conclude the findings: The findings were reported, and discussion was drawn correspondingly
The researcher detailed the methodology of the study, utilizing the IELTS writing test, IELTS writing band descriptors, and interviews as primary data collection instruments A total of 30 second-year students participated in the IELTS writing test, from which 10 students were selected for interviews Data from the writing tests was analyzed using Excel and SPSS software, while the interview data underwent qualitative analysis by the researcher.