04051001900 An investigation into the validity of an English achievement test for 12th graders in Vinh Phuc Province = Nghiên cứu tính giá trị của một bài kiểm môn Tiếng Anh cuối kỳ dành cho học sinh lớp 12 ở tỉnh Vĩnh Phúc
INTRODUCTION
Rationale for the research
English is currently the primary and mandatory subject in most high schools, and it is also a requirement for graduation This situation necessitates a transformation in the teaching and learning approaches of both educators and students, as their methods are interconnected.
Teaching and testing are closely interconnected as teaching enhances learners' knowledge through diverse methodologies, while testing evaluates students' competence and informs instructional improvements Effective testing measures cognitive abilities, enabling educators to tailor methods and materials to students' levels Additionally, academic records bridge the relationship between families and schools, allowing parents to monitor their children's progress and emphasizing the importance of reliable and valid assessments for better learning outcomes.
In Vietnam's English education, testing and assessments are crucial to the teaching process, as they validate the effectiveness of instruction The teaching and learning activities generate valuable language materials that can be utilized for testing purposes Consequently, testing remains a significant focus for most educators.
To ensure high-quality testing, it is essential for testers, raters, and test takers to evaluate the test based on specific criteria By assessing language competence, educators can reflect on their teaching methods and identify both the strengths and weaknesses of the test If significant shortcomings are found in the test preparation, it is crucial for testers to reevaluate the test accordingly.
2 that the test suits the content validity of the test However, if the test satisfies the requirement of testing, it suggests that the test is valid and reliable
Numerous studies have examined the validity and reliability of tests, including significant international research by Gavilánez and Calero.
Research on the reliability and validity of tests has been conducted by various scholars, including Ozera, Shawn, Sulbarana, and Garvey (2017), as well as Abella, Urrutia, and Shneyderman (2013), and Midgley, Kaplan, Middleton, and Maehr (2005) In Vietnam, notable studies have been carried out by Linh (2006), Phương (2008), Thủy (2008), and Nguyen Thi Hồng (2008), primarily focusing on university and college settings Additionally, in the context of high schools, researchers like Sáu (2009) have explored test validity and its impact.
During my time in Vinh Phuc, I encountered numerous complaints from teachers and students regarding the final tests' quality, which often failed to accurately reflect the teaching and learning process The assessments did not align with the course objectives or the expected skills and knowledge of the students Furthermore, language testing at SS High School lacked sufficient attention, as teachers struggled to find the time to thoughtfully plan their tests Consequently, some students received lower marks and were required to retake critical assessments.
Because of all above –mentioned reasons, I decided to carry out the study entitled:
This article investigates the validity of an English achievement test for 12th graders in Vinh Phuc Province, focusing on identifying its strengths and weaknesses regarding validity and reliability Additionally, it offers solutions for enhancing the test's effectiveness.
Scope of the study
Due to time constraints and research limitations, this study does not aim to address all aspects of effective achievement tests, such as reliability, validity, discrimination, authenticity, and backwash effects Instead, it focuses specifically on the content validity and reliability of the end-term achievement tests for English grade 12 at SS High School in Vinh Phuc.
In the academic year 2021-2022, this study focused on assessing the final achievement test's content validity and reliability The evaluation involved comparing the test content with the objectives, syllabus, and text allocation Additionally, the research examined how reliability evidence can enhance the validity of the test.
Aims and objectives
This study investigates the content validity of the final achievement test for high school students in Vinh Phuc Province during the 2021-2022 academic year It evaluates the relevance and coverage of the test content in relation to the test specifications and the actual performance of the examinees The analysis is conducted by three testing experts utilizing Bachman and Palmer's framework.
The 1996 framework and test score analysis reveal a strong alignment between the test content, construct, and design framework, as well as the performance of test takers This study aims to assess the content validity of end-term achievement tests, highlighting their strengths and weaknesses.
Research questions
Specifically, the research was carried out to answer the following questions:
Research question 1 To what extent is the content validity of the test against the test specifications?
Research question 2 To what extent is the content validity of the test judging from the test results?
Methods of the study
This study employs a mixed-methods approach, integrating both quantitative and qualitative techniques The quantitative aspect involves analyzing data from a final English test for grade 12 students at SS High School in Vinh Phuc Province, where the content validity of each language component was quantified and expressed as percentages Additionally, qualitative insights were gathered through interviews with teachers regarding the test and its specifications, further reflecting the test's validity.
4 point is that qualitative method was used to interpret the data into the meaning of test samples and their components in terms of content validity.
Organization of the study
The thesis is organized into four major chapters:
Chapter 1 is the introduction that presents such initial information as the rationale, aims, methods, research questions and the organization of the study
Chapter 2 reviews all related literature that provides the theoretical basis for language testing and language evaluation First, the relationships of language testing with teaching and learning and objective testing are presented Then, the achievement tests; test specification; multiple choice questions and testing language components are discussed in a careful way Next, the most important theoretical part, validity in terms of content validity and reliability are deeply taken into consideration Last parts are spent for objectives an syllabus design of English grade 12; recommended test specification of end-term achievement tests on English grade
12 and components‟ contents of end-term achievement tests
Chapter 4 is the data analysis and findings, describing the analysis of data in details and discussing the results of the study
Chapter 5 is the conclusion, providing the summary of the main issues and some pedagogical implications This chapter also includes limitations and some suggestions for further studies
LITERATURE REVIEW
This chapter outlines the theoretical framework of the research, beginning with the interplay between language testing and the teaching and learning processes It then delves into achievement tests, including test specifications and the assessment of language components and skills A critical focus is placed on validity, particularly content validity and reliability Finally, the chapter addresses the objectives and syllabus design for English in grade 12, examining the current test specifications and end-term achievement tests at the research school, along with the content components of the studied end-term achievement test.
2.1 The relationship of language testing with teaching and learning
Testing is a crucial component in the teaching and learning process, serving as a tool to evaluate learners' abilities Harrison (1983) highlights that tests effectively measure students' capabilities in classroom settings Heaton (1975) views tests as a means to assess student performance and motivate learners Oller (1974) emphasizes that testing fosters a sense of competition among students, rather than merely assessing their performance Through testing, teachers can make informed decisions regarding courses, objectives, materials, and student needs Brown (1994) defines a test as a method for measuring an individual's knowledge or ability in a specific area, while Bachman (1990) notes that language tests can focus on particular areas of interest Overall, testing is an effective means of assessing students' language knowledge and skills, enabling educational leaders to enhance course design, syllabi, and teaching materials, ultimately improving the roles of teachers, learners, and administration in the assessment process.
6 teaching and learning process Testing is also seen as the last stage of teaching process
Teaching, learning, and testing are intricately connected, with changes in one significantly impacting the others Among these, language testing exerts the most pronounced influence on both teaching and learning Heaton (1988) emphasized that testing and teaching are so interrelated that one cannot effectively engage in either without considering the other He also highlighted the critical role of testing in the learning process, noting that tests can serve to reinforce learning and motivate students, as well as assess their performance in language acquisition.
According to Davies (1996, p.5), effective English language testing plays a crucial role in fostering positive attitudes towards learning Well-constructed tests provide students with a sense of achievement and ensure that teacher evaluations align with the instruction provided Additionally, quality English assessments encourage diligent study, highlight course objectives, and identify areas for improvement, ultimately enhancing language acquisition.
In the teaching field, testing plays a crucial role in helping educators assess students' proficiency in target language knowledge and skills According to Bachman (1990), the primary purpose of testing within an educational program is to provide valuable information for decision-making and evaluation However, teachers often face challenges in obtaining accurate, reliable, and valid test results due to the diverse interests, attitudes, and backgrounds of test takers This can lead to disappointment when test outcomes do not meet teachers' expectations A viable solution is to incorporate a range of test items, including easier questions to support weaker students and progressively more challenging items for advanced learners.
For learners, testing helps teachers find out their weak and strong points, from which they may develop the most suitable learning strategies themselves; testing
Testing can motivate students to maintain or improve their class rankings, but if the difficulty level is inappropriate, it may lead to disinterest or boredom in the learning process Hughes (1989) discussed the concept of backwash in testing, highlighting its potential to be either beneficial or harmful, with a focus on the negative aspects He argued that when test content does not align with course objectives, harmful backwash occurs, suggesting a disconnect between testing and teaching For instance, if writing skills are assessed solely through multiple-choice questions, students may focus on practicing these items rather than developing their actual writing abilities.
Testing is crucial in the teaching and learning process, as it allows educators to assess both their instructional methods and student performance effectively A well-designed test can mitigate negative backwash effects, ensuring that assessments contribute positively to the educational experience.
A test is a systematic tool used to observe and describe specific characteristics of a student, employing either a numerical scale or a classification scheme It is important to note that testing is a more focused concept compared to assessment.
Language tests can be categorized based on their purposes, as outlined by Hughes (1989) These include proficiency tests, achievement tests, diagnostic tests, and placement tests Proficiency tests assess language ability without considering prior training and are not aligned with specific course content, exemplified by tests like FCE, CPE, TOEFL, and IELTS In contrast, achievement tests are closely tied to language courses and evaluate whether students have met the course objectives They are divided into final achievement tests, administered at the end of a course, and progress achievement tests, which monitor students' development throughout the course.
Progress achievement tests assess students' development to enhance their performance in final exams Diagnostic tests identify learners' strengths and weaknesses, helping to determine necessary areas for further learning Placement tests are utilized to align students with the appropriate level of the teaching program based on their abilities, while also classifying them into different levels for targeted instruction Additionally, aptitude tests evaluate an individual's capacity to acquire specific skills and predict future learning performance.
Besides, based on manner in which are tests are scored, they divided into objective and subjective testing
Objective tests are designed to provide scores that are independent of the scorer's judgment, as all items are objectively scored (Davies, 1999, p 132) These tests specify correct responses, eliminating the need for markers to make subjective assessments They serve as an effective method for evaluating English study results, assessing both communicative skills and language knowledge Additionally, objective tests can encompass a broader range of grammar, vocabulary, and phonology points compared to subjective tests.
The techniques employed in objective testing include multiple-choice items, true or false questions, matching items, sentence transformations, rearrangement tasks, and fill-in-the-blank exercises Test-takers must select their answers from a range of options, typically with only one correct answer Objective tests require significant time and effort for both test-takers and examiners, necessitating thorough preparation Many institutions favor this testing format due to its reliability and consistent scoring The primary purpose of these tests is to assess language structures, vocabulary, comprehension, and sound discrimination.
This test format promotes guessing, making it easier to answer and score, while being suitable for a large number of test-takers Additionally, it can be efficiently scored using computer-based MCQ testing software.
Objective tests, while useful, have notable drawbacks, particularly concerning their validity Heaton (1998, p 26) highlights that these tests are often criticized for being easier to answer than subjective tests However, the difficulty of objective test items can be adjusted by the test designer Critics of multiple-choice formats argue that they encourage guessing; yet, Heaton asserts that providing four or five answer options can significantly minimize this issue He further notes that test-takers typically do not guess randomly but rather rely on their partial knowledge when making selections.
Designing effective objective tests poses a significant challenge for test-makers, as it requires a deep understanding of testing techniques and the ability to create a diverse range of test items To develop a high-quality objective test, it is essential for test-makers to master these skills and prepare a comprehensive set of testing materials.