Nghiên cứu về hoạt Động Đánh giá kỹ năng nói của giảng viên Đối với sinh viên không chuyên tiếng anh tại một trường Đại học Ở hà nội
INTRODUCTION
Rationale for the study
In contemporary education, teaching and assessment are viewed as interconnected, with student performance assessment being vital to effective teaching (Mertler, 2009) Recently, stakeholders in Vietnam's education system have shown heightened interest in language testing and evaluation, recognizing that assessment information influences decisions about students, curricula, programs, and educational policies (Nitko, 2009) Effective assessment and marking strategies are essential for fostering positive changes in teaching, such as enhanced instruction, increased motivation, and improved student outcomes (Mellati & Khademi, 2018) However, research indicates a disconnect between teachers' beliefs and their actual assessment practices, revealing inconsistencies in how assessment methods are applied in the classroom (Karavas-Doukas, 1996; Phipps & Borg, 2009).
Contemporary language education significantly prioritizes the assessment of learners' efforts and achievements, a shift from practices of previous decades The English teaching curriculum was notably revised with the launch of Project 2020, aimed at enhancing foreign language proficiency within the National Formal Educational System from 2008 to 2020 This initiative sought to ensure that by 2020, most learners in vocational and higher education would attain foreign language skills, enabling Vietnamese youth, civil servants, and professionals to communicate, study, and work independently by 2025.
This project has highlighted the importance of teaching and learning spoken language skills, particularly at the university level in Vietnam, where students must achieve a level 3 proficiency in the English competencies framework upon graduation Teachers are also required to have a C1 proficiency level and participate in regular assessment training While numerous studies have focused on high school contexts, speaking assessment practices in higher education remain underexplored.
Aims, objectives and research questions of the study
This research addresses the gap in speaking assessment practices at the tertiary level by investigating the methods used by university teachers It focuses on the goals, objectives, metrics, interpretation, and application of speaking assessments in classroom settings, as well as the factors that influence these practices.
This study explores the speaking assessment practices of teachers for non-major English students at a university in Hanoi, addressing a significant research gap It aims to answer two key questions related to these assessment practices.
1 What are the current practices of classroom speaking assessment conducted by teachers of English at the Faculty of English education of a University in Hanoi?
2 What factors affect the practices of classroom speaking assessment by teachers of English in the Faculty of English education of a University in Hanoi?
Scope of the research
This study focuses on the speaking assessment practices of in-service university teachers, specifically examining in-class speaking evaluations and the factors influencing oral assessment It excludes other skill assessments and types of evaluations outside the classroom The research targets speaking assessment at the tertiary level, involving 32 instructors, which represents 85% of those teaching English to non-major students in the most recent semester Additionally, 9 teachers participated in semi-structured interviews, responding to survey questions and interviews conducted at the end of the spring term in early 2023, following a one-and-a-half-month term at the beginning of the year.
The study focuses on English for Occupational Purposes (EOP) within the university's curriculum, highlighting the factors that influence speaking assessment practices in the classroom.
Methods of the study
This study employs a mixed-method approach, integrating both quantitative and qualitative data collection and analysis A survey questionnaire was distributed to university teachers to collect insights on their speaking assessment practices and the influencing factors Additionally, qualitative data was gathered through interviews to explore the teachers' perspectives and experiences in greater depth.
The questionnaire comprises 13 questions, with the first 12 focused on exploring speaking assessment practices To ensure relevance to the Vietnamese tertiary teaching environment, it also addresses challenges faced by speaking teachers and the factors influencing their decisions Additionally, to enhance the clarity of the survey results, interviews were conducted with 9 instructors.
Significance of the study
This project aims to provide comprehensive insights into the speaking assessment practices of in-service university English teachers and the factors influencing their decisions The findings will facilitate the creation of effective plans and necessary revisions for assessing and teaching speaking skills at the university level, particularly in English for Occupational Purposes Importantly, this study can inform the development of training programs and courses focused on speaking assessment practices and teaching methodologies for in-service educators.
Structure of the thesis
This research consists of five chapters presented as below:
Chapter 1 – Introduction – presents the rationale, aims, scope, significance, method and structure of the study
Chapter 2 – Literature review – provides the definition of key terms as well as the framework of the study and includes reviews on related studies to the research topic
Chapter 3 – Methodology – gives information about the setting of the research, sampling method, participants‘ information, data collection instruments, and data analysis methods of the research
Chapter 4 – Findings and discussion – presents, analyses and discusses the research findings
Chapter 5 – Conclusion – provides the summary of findings; implications, limitations of the study; and suggestions for further research.
LITERATURE REVIEW
Definition of language assessment and classroom assessment
Assessment in language education lacks a fixed definition, but it is generally understood as the systematic collection of data regarding specific aspects of language learning (Bachman, 2004) This data can be obtained through formal tests or informal methods, such as teachers' checklists of student performances (Purpura, 2016) According to Chapell and Brindley (2010), language assessment involves gathering information and making judgments about a learner's understanding and ability to use the language effectively.
Assessment is a comprehensive process aimed at making informed decisions about students, curricula, programs, schools, and educational policies, as noted by Brookhart and Nitko (2008) This definition highlights the expanded scope of assessment while maintaining a focus on the essential steps involved in the assessment process.
Classroom assessment, as defined by McMillan (2015), is primarily managed by teachers and tailored to the unique needs of each class This approach allows educators to implement diverse assessment methods and set specific learning objectives that cater to individual students Furthermore, it encompasses a range of strategies and techniques utilized by both teachers and students to effectively gather, analyze, and report on student achievements The study aims to adopt McMillan's (2015) definition of assessment.
Classroom assessment procedure
Language assessment is a systematic procedure that begins with establishing its purposes, defining the language knowledge to be evaluated, and selecting suitable assessment methods According to Shohamy and Inbar (2006), this process involves creating and developing specific tasks and items, administering the assessment instrument, and evaluating its quality, validity, and reliability It is crucial to acknowledge potential challenges during administration Ultimately, the findings from the assessment are analyzed and reported to the relevant authorities.
Table 2.1 Components of classroom assessment (Shohamy and Inbar, 2006)
1 Determining the purpose of assessment
2 Defining language knowledge to be assessed
6 Determining the quality of the language sample/answers produced
7 Assessing the quality of the procedures
McMillan (2015) suggested a simpler version with five components of classroom assessment including purposes, learning objectives, measurement, interpretation and use They are illustrated in the figure 1 below:
Figure 2.1 Components of classroom assessment (McMillan, 2015)
The procedure of McMillan (2015) is quite similar with that of Shohamy and Inbar (2006) The third component ―measurement‖ in the procedure of McMillan
(2015) contains ―selecting the assessment procedure‖, ―designing items and tasks‖,
This study will utilize McMillan's (2015) methodology to evaluate the procedures teachers implement in their classrooms, focusing on administering assessment tools, determining the quality of language samples and responses, and assessing the effectiveness of these procedures, thereby simplifying the process compared to previous methods.
Purpose Learning objectives Measurement Interpretation Use
2.2.1 Identifying purposes of classroom assessment
Identifying specific assessment purposes is crucial in the assessment process, as emphasized by McMillan (2015), who defines purpose as the reason for conducting assessments Classroom assessments can be categorized as formative or summative based on their objectives Formative assessment, as described by McManus (2008), involves feedback from both teachers and students during lessons to improve the learning and teaching process The main goal of formative assessment is to enhance student learning and motivation (McMillan, 2015).
Brown (2004) defines assessment of learning, or summative assessment, as a method used to evaluate students' academic performance, typically conducted at the conclusion of a term, semester, or year In contrast, formative assessment is emphasized in in-class practices to support ongoing learning.
Apart from the above-mentioned classification, according to Bachman
The primary objective of the assessment, as noted in 1981, is to make informed assumptions about language proficiency to guide future decision-making This involves linking specific assessment purposes to decision-making categories, which are divided into two levels: micro-evaluation, focusing on individuals, and macro-evaluation, which pertains to the overall program A flow chart is included to summarize these key purposes.
Figure 2.2 Purposes of Assessment (Bachman, 1981)
Bachman (1981) and McManus (2008) share similar assessment purposes, but it is essential to consider a broader range of goals When making decisions regarding students, Bachman's insights are particularly relevant.
In 1981, four key purposes of assessment were identified: selection, placement, diagnosis, and progress grading Selection often necessitates a single overall score, while placement involves multiple levels to help teachers group students by similar abilities for presentations and activities Additionally, the primary focus of diagnosis is to evaluate students' strengths and weaknesses.
Cheng, Rogers, and Hu (2004) identify three primary purposes commonly found in classrooms: instructional, student-centered, and administrative A summary of these purposes is presented in the table below.
Table 2.2 Purposes of Assessment (Cheng, Rogers and Hu, 2004)
Instructional purpose Student-centred purpose Administrative purpose
Teachers must gather information about students' knowledge, comprehension, and skills
For instance, teachers are obliged to ask students to complete a task or watch them practice in class to gauge their understanding
Teachers can decide how to prepare and adjust their instruction after examining the data they have acquired
Assessment aims to identify students' strengths and weaknesses Teachers can utilize diagnostic evaluation methods to adjust course content, ensuring it aligns with students' learning needs in the classroom.
Teachers are obligated to report the statistics and make public comments on student achievement to parents and principals after assigning pupils their final grades
According to Mertler (2003), the instructional process is structured chronologically into three phases: before, during, and after instruction, differing from the approach of Cheng, Rogers, and Hu (2004) During the planning phase, key decisions include drafting material, creating lesson plans, selecting additional resources, assigning projects, and determining assessment methods, all influenced by teacher expertise, district curriculum, and student needs Throughout instruction, ongoing evaluations are essential to assess teaching effectiveness and motivate students, while teachers must remain aware of students' challenges and behavioral issues Finally, post-instruction analysis is crucial for identifying necessary adjustments for future teaching.
2.2.2 Identifying learning objectives and targets
Learning targets are essential declarations outlining what students are expected to learn (McMillan, 2015) To create clear and detailed learning objectives, teachers should utilize Bloom's modified taxonomy, which categorizes cognitive processes into six hierarchical levels: remembering, comprehending, applying, analyzing, evaluating, and creating (Anderson & Krathwohl, 2001) This structured approach ensures that assessments align with the intended learning outcomes, helping both teachers and students understand expectations and effectively measure learning success.
Classroom assessment methods can be categorized into four main types: selected response, constructed response, teacher observation, and student self-assessment (McMillan).
Peer assessment, while not commonly utilized, can be considered a fifth category in evaluation methods The table below clearly outlines the specific tasks along with the objectives and targets that each type of assessment can measure.
Table 2.3 Categories of assessments (McMillan, 2015)
Task multiple-choice, matching, true– false, short answers, and completion sections performance, essay, and oral questioning
Performance assessments involve comprehensive responses or presentations that necessitate the completion of meaningful tasks, often guided by a scoring rubric While these assessments can be formal, they are frequently informal, allowing students to evaluate their own performance.
Features best at measuring subject matter and procedural knowledge and comprehension, especially when students need to recognize or remember facts, dates, people,
A comprehensive understanding of student dispositions—such as attitudes, cooperativeness, effort, trust, and responsibility—is crucial for effectively measuring their learning progress By gathering data on these dispositions, educators can assess students' abilities to comprehend learning criteria and track their advancements.
Student self- assessment historical events, spellings, and concepts and what else needs to be learned
Factors affecting teachers‘ classroom assessment practices
Formative assessment is gaining increased focus in classroom assessment practices Clarke and Hollingsworth (2002) identified four key components influencing the adoption of formative assessment: personal, contextual, external, and resource-related factors These components are visually represented in Figure 3, which outlines a framework for understanding the factors that impact teachers' implementation of formative assessment.
Personal factors, as noted by Clarke and Hollingsworth (2002), encompass teachers' perceptions of formative assessment, while other factors pertain to the change environment The school context, which includes the learning environment and policies, significantly influences teachers' actions and is linked to contextual components Additionally, educational features at both local and national levels are shaped by external variables such as high-stakes and accountability assessments Professional development activities exemplify resource-related aspects, requiring essential resources like funding, time, and information.
McMillan and Nash (2000) developed a model that highlights the key factors affecting instructors' decision-making regarding classroom assessment, particularly in grading Their model identifies three primary influences: the realities of the classroom environment, external circumstances, and the knowledge, beliefs, and expectations of teachers.
"evaluation decision making is characterized by friction between the teachers' inherent ideas and values and external factors that are placed on them" (McMillan,
External pressures, such as district regulations, extensive testing programs, parental expectations, and real classroom situations, compel educators to adopt assessment methods that conflict with their personal beliefs and values.
Figure 2.5 Teachers’ classroom assessment decision making (McMillan and
Assessment practices are influenced by three primary factors: internal factors, including teachers' knowledge, beliefs, expectations, and values; external factors from contexts outside the classroom; and the realities within the classroom This study will utilize the McMillan and Nash (2000) model as a theoretical framework to formulate interview questions, owing to its clear explanation of each aspect and the evident connections among them.
Speaking assessment
Defining speaking is challenging, as it is often perceived as a simple activity, yet it is inherently complex (Galaczi, 2010) According to Burns and Joyce (1997), speaking is an interactive process that involves the production, reception, and processing of information to create meaning Various internal and external factors, such as the participants, their shared experiences, the physical environment, and the purpose of speaking, significantly influence this activity Additionally, Fulcher (2003) presents a framework for understanding the construct of speaking, illustrated in the accompanying figure.
Figure 2.6 Framework for speaking construct (Fulcher, 2003)
Fulcher (2003) highlights the importance of communication strategies, while Bachman (1990) defines strategic competence as the ability to effectively utilize one's skills to accomplish tasks, allowing speakers to engage in conversations despite limitations in grammar or vocabulary This includes the use of code-switching and non-verbal cues such as gestures Additionally, textual knowledge pertains to the organization of speech, whereas pragmatic knowledge emphasizes the ability to accurately receive and convey intended messages.
Sociolinguistic knowledge, as defined by Bachman (1990), refers to the ability to navigate language use based on the specific context, allowing individuals to perform language functions appropriately This knowledge enables people to select language that fits the situation and to recognize cultural differences in communication.
This study utilizes Fulcher's (2003) speaking construct, which is instrumental in developing assessment criteria for the questionnaire Bachman (1990) defines sociolinguistic knowledge as the ability to navigate language use based on specific contextual factors, allowing individuals to perform language functions appropriately Those with sociolinguistic awareness recognize cultural communication differences and can choose suitable language for various situations.
This investigation utilized Fulcher's (2003) speaking construct, which is instrumental in establishing grading standards for various questions in the questionnaire.
Any developers of a speaking assessment would concern the kind of speaking, the methods of assessment and the rating criteria (Luoma, 2004)
Fluency is closely linked to pragmatic ability, a crucial yet challenging aspect of evaluating speaking skills (Luoma, 2004) According to Koponen (1995), fluency is often described through terms such as flow, smoothness, speech rate, minimal pauses, absence of hesitation markers, utterance length, and connectivity These descriptions are intricate, as they reflect both the speaker's delivery and the listener's perception Stockdale (2009) further illustrates that fluency involves the capacity to communicate in a foreign language with minimal silent pauses, filled pauses (like "oooh" and "emm"), self-corrections, false starts, and hesitations.
Research shows that even when there are multiple analytic criteria used in the test, raters tend to pay a lot attention to grammar (Brown and McNamara,
The accuracy of grammar reflects how well students apply grammatical rules, while the range of grammar indicates the variety of rules they can use.
Vocabulary plays an inevitable role in speaking assessment especially in English for occupational purposes According to August Carlo, Dressler, and Snow
In 2005, it was observed that language learners with limited vocabulary face challenges in acquiring new words and often refrain from engaging in oral communication with their peers Vocabulary encompasses a student's capacity to utilize a diverse and precise range of words relevant to a specific subject during speaking assessments For example, when given a particular topic, students can effectively express their opinions by using contextually appropriate language.
Clear pronunciation, while demanding significant focus, can be a strong point for weaker examinees (Luoma, 2004) To effectively assess pronunciation, it is essential to consider phonemic differentiation, word and phrase stress, rhythm, intonation, and the use of linking, elision, and assimilation (Knight, 1992).
Teachers should implement speaking techniques that promote student interaction to assess conversational skills effectively This includes the ability to sustain a conversation, maintain coherence in dialogue, take turns, self-correct, use pause fillers, seek clarification, and expand on discussion topics.
Bachman and Palmer's CLA (1996) emphasizes that sociolinguistic knowledge, while often implicitly assessed, plays a crucial role in examinees' performances during interactions Students are expected to employ a polite and familiar register suitable for supervised school interactions.
This test evaluates functional knowledge indirectly, as it does not directly elicit specific functions Examinees must describe, interpret, and justify their responses during the interaction, with their ability to express these ideas assessed through vocabulary, grammar, pronunciation, and fluency, rather than through a separate criterion related to language functions (Bachman and Palmer, 1996).
With regard to speaking assessment methods, Luoma (2004) indicates that assessment tasks for oral performance contains nine main categories which are clearly presented in the following table
Table 2.5 Assessment tasks for oral performance (Luoma, 2004)
Speaking tests often feature description tasks that require examinees to provide detailed responses to brief prompts When designing these tasks, developers must consider the inclusion of images, as they can increase the complexity of the task while also offering greater flexibility in terms of size and duration, particularly in paired activities.
One-to-one interviews and with pairs, tape- based testing
Narrative tasks are used to assess examinees' recall of events from the present or past, often utilizing picture sequences that guide the narrative Alternatively, subjects may be asked to share personal experiences These tasks should focus on key narrative elements, such as setting the scene, consistently naming characters, identifying significant events, and presenting them in a logical sequence Prior to testing, it's advisable to pilot the picture sequences to ensure their effectiveness.
Look at the sequence of pictures and tell a story
The primary goal of providing directions and instructions is to effectively communicate the message and ensure comprehension This often results in brief interactions between the speaker and listener Conversely, interactive tests evaluate the dynamics of communication between speakers as well.
Look at maps and give instructions
Comparing Analysis and debate of similarities and differences are Examinees
Factors affecting teachers‘ assessment and grading practices
Investigating factors which have impacts on teachers‘ assessment practices is a topic which is always being paid attention to According to McMillan and Nash
In 2000, three primary factors significantly influence teachers' decisions regarding classroom assessment and grading: teacher beliefs and values, classroom realities, and external factors, as illustrated in Figure 2.7.
Figure 2.7 A model of Teachers’ Assessment and Grading Practices Decision
A teacher's philosophy of teaching and learning significantly influences their decisions regarding classroom assessment and grading practices (McMillan and Nash, 2000) This philosophy provides a rationale for selecting assessment and grading methods that align with the teacher's educational priorities.
Classroom realities, as defined by McMillan (2005), encompass factors beyond teachers' control, including social promotion, excessive absenteeism, disruptive behavior, and student heterogeneity Research by Richards and Pennington (1998) further highlights the detrimental effects of student-related issues on teaching practices, such as large class sizes, unmotivated students, low English proficiency, and resistance to new learning methods.
External factors are influences originating outside the classroom that do not directly impact a student's behavior, such as the home environment (McMillan and Nash, 2000) Although these external elements, like certain classroom realities, are beyond a teacher's control, they still significantly affect the assessment and grading processes within the classroom.
This study would employ the model of McMillan and Nash, 2000 to investigate the factors influencing the teachers‘ practice of speaking assessment in class.
Speaking assessment practice at tertiary level in the context of Vietnam
At the tertiary level, speaking evaluation has attracted much attention Its implementation helps students learn more effectively and develops their speaking abilities (Ounis, 2017)
Van et al (2006) highlighted that tertiary foreign language education in Vietnam is of inadequate quality, failing to meet the demands necessary for the country's socio-economic growth Additionally, Tien supports this view, emphasizing the need for improvement in language education to align with current developmental goals.
In 2013, the quality of teaching and learning at the tertiary level was found to be inadequate, with students exhibiting extremely low language competency Many college graduates failed to meet recruiters' foreign language requirements, primarily due to a focus on grammar and writing over oral skills Assessment practices tended to evaluate language knowledge rather than practical language skills Additionally, significant challenges in English Language Teaching (ELT) at this level included insufficient instructional time, varying starting English levels among students, unclear assessment criteria, and low motivation among learners.
Recent interest in in-class speaking assessment in Vietnam has led to a thorough investigation of the topic, especially at the university level This study focuses on evaluating classroom speaking practices in higher education.
English for Occupational Purposes (EOP)
2.7.1 English for Occupational Purposes (EOP)
At the university conducting this study, instructors are delivering English for Occupational Purposes (EOP) tailored to students' specific fields The primary aim of the EOP course is to meet the English language requirements relevant to learners' professional environments As a subset of English for Specific Purposes (ESP), EOP is utilized in both business and academic contexts, preparing students for their future careers (Dudley-Evans and St John, 1998) For example, effective communication in a retail setting necessitates that students acquire specific conversational skills.
Greet the customer and offer help
Introduce new items to the customer
EOP prioritizes learner-centered practices, highlighting the importance of addressing students' needs and interests in curriculum development A well-structured EOP curriculum should be based on thorough needs analysis rather than mere perceptions At the university level, diverse English courses are available for students across various faculties, including English for the Automobile Industry, Mechanical Engineering, Business, and Accounting These courses are designed to enhance students' communication skills, particularly their oral abilities, thereby providing valuable opportunities for their future career paths.
2.7.2 Speaking assessment practices in EOP
Language assessment for professional purposes refers to the evaluation process conducted by stakeholders to gauge an individual's proficiency in understanding and using language within a specific professional context, ensuring it meets established standards or requirements (Knoch and Macqueen 2020, 3).
Both language for specific purposes assessments and general language assessments share common considerations, such as the test's purpose, characteristics of test-takers, target language-use situations, and the reliability, validity, and impact of potential tests (Douglas, 2013) However, they differ primarily in two areas: the authenticity of the tasks involved and the connection between language proficiency and domain-specific content knowledge.
2.7.3 Issues in Speaking assessment practices in EOP
The demand for performance-based English assessments in the workplace is increasing, yet creating effective speaking assessments for English for Occupational Purposes (EOP) remains a significant challenge This difficulty arises because English for Specific Purposes (ESP) assessment creators often lack expertise in the specific fields of the learners, hindering their understanding of workplace communication Language professionals may not be familiar with the technical terminology, necessary language functions, or real dialogues relevant to various industries such as automotive, engineering, or business Furthermore, developing authentic contexts and appropriate speaking assessments for EOP learners poses an additional challenge for language teachers.
Douglas (2000) emphasized that the development of evaluation criteria or rating scales is crucial and challenging in creating speaking tests for specific purposes He further noted in 2001 that test content should be aligned with assessment criteria formulated in the target language While language experts excel in designing speaking exam tasks and evaluating oral performances, they lack expertise in the specific fields of the test-takers Recent studies have introduced more field-specific assessment criteria for English for Occupational Purposes (EOP) speaking assessments (e.g., Friginal 2013; Manias and McNamara 2016; Wette and Hawken).
Recent studies, as noted by Hijikata and Min (2021), have expanded the evaluation of test-takers beyond general effectiveness and communicative competence to include context-specific characteristics For example, health professionals must communicate effectively not only with their peers but also with patients who may not have medical expertise To better address the actual language needs of test-takers, two new criteria—clinician engagement and management interaction—were introduced (e.g., Manias and McNamara 2016; O'Hagan et al 2016).
Language assessments for specific purposes and general language assessments share several similarities, including the consideration of test purposes, characteristics of test-takers, target language-use situations, and the reliability, validity, and impact of the tests However, the most notable difference is found in the methods used to evaluate profession-specific communication tasks and the authenticity of those tasks.
Previous studies
Research has explored the relationship between various factors and teachers' speaking assessment practices in classrooms, particularly in contexts like Korea and Norway Kim (2003) examined the assessment tasks used by Korean junior secondary school English teachers, revealing a reliance on traditional, teacher-centered evaluation methods that prioritize grammatical accuracy and pronunciation over communicative skills The study identified challenges such as difficulties in establishing appropriate criteria and grading assessments Similarly, Agasster (2015) evaluated oral English assessment practices in lower secondary schools in Norway, focusing on assessment criteria, procedures, feedback methods, and teachers' perceptions of assessment's role in education, along with the challenges they face in implementing effective practices.
Studies on in-class speaking evaluation have been conducted in Vietnam, including those by Truong (2010), Tran and Nguyen (2017), and Nguyen and Tran
Nguyen and Tran (2018) conducted a study on the perceptions of EFL teachers regarding in-class English speaking evaluation They initially collected survey data from 42 teachers across various high schools in Quang Tri Province, followed by interviews with five selected participants The findings revealed that while the teachers had a good understanding of in-class speaking evaluation, they required additional theoretical and practical knowledge about language assessment.
Teachers expressed their unfamiliarity with portfolios and student self-evaluations in speaking assessments While researchers have shown some interest in in-class speaking evaluation, it remains a relatively underexplored area, particularly in higher education Most existing studies focus on high school classroom speaking assessment methods Consequently, this study aims to investigate the evaluation procedures for classroom speaking at the university level.
There is a notable lack of research focused on speaking assessment practices for occupational purposes In Vietnam, while many researchers have investigated in-class speaking assessments, the exploration has been limited, particularly concerning English for Occupational Purposes.
Summary
This chapter provides a brief overview of key terms related to the study's focus, including general assessment, classroom assessment, factors influencing classroom assessment, and speaking assessment The theoretical framework is based on five elements of classroom assessment identified by McMillan (2015) The research follows the paradigm established by McMillan and Nash (2000) to examine the variables affecting classroom speaking assessment practices Subsequent chapters will delve into the study, its methodology, and findings within the context of the aforementioned theories.
METHODOLOGY
Settings of the study
This study was carried out in the Faculty of English for occupational purposes at a university in Hanoi The average class size in this university is about
The university has developed a specialized program and curriculum for English for occupational purposes, emphasizing speaking skills from the first to the third year Initially, general speaking skills are prioritized, while the focus shifts to occupation-specific speaking skills in the subsequent years Teachers utilize the curriculum and course outline for assessment practices, allowing them to adapt evaluation tasks and activities to meet students' needs and language proficiency, thereby facilitating the achievement of learning objectives.
Sampling and participations
The study focuses on 32 English teachers from the Faculty of English at a university in Hanoi, with a significant majority being female, comprising 29 out of the 32 participants Most of these educators hold master's degrees, while a small number possess bachelor's degrees.
The Faculty of English maintains an average class size of 35 students and requires teachers to manage a workload of 20 periods per week across five classes Most educators have over six years of experience teaching English Additionally, all teachers must attend an annual training session focused on language assessment.
All 32 teachers participated in the questionnaire, which was provided with a Vietnamese translation to ensure full comprehension of the questions The survey focuses on two main topics: the current speaking assessment practices of instructors and the challenges they face when evaluating speaking skills in their English for Occupational Purposes (EOP) classes.
In an interview focused on factors influencing current speaking evaluation practices, only 9 out of 32 teachers were able to participate due to scheduling conflicts.
Data collection
This study utilizes questionnaires and interviews as data collection methods, focusing on the effectiveness of questionnaires in efficiently gathering extensive data from participants Additionally, interviews provide the researcher with the opportunity to obtain more detailed and specific information.
According to Dornyei (2007), the results of surveys would guarantee both a high number of responses in a short amount of time and the generality of the study
In this study, a questionnaire was selected as the primary data collection instrument, with careful attention given to each step of its design process The survey questions were developed based on extensive literature research and subsequently translated into Vietnamese to avoid misunderstandings The author created a comprehensive questionnaire, which underwent two rounds of piloting to evaluate its validity and reliability Initially, feedback was sought from a supervisor, followed by a second round of piloting with an instructor at a university in Hanoi After incorporating the feedback received, the researcher revised the questionnaire to finalize the version used for the study.
The questionnaire begins with an overview of the research title and its objectives, ensuring participants understand the study's purpose This section also emphasizes the confidentiality of the data collected and the commitment to utilizing the information solely for research purposes.
The questionnaire consists of two sections: the first part collects confidential personal information from participants, while the second part includes 13 questions focused on assessing their current speaking assessment practices.
Questions 1 to 12 in particular go in-depth on the speaking assessment procedure including purposes, construct definition, speaking assessment methods, designing speaking items and tasks, speaking assessment administration, assessing the speaking assessment procedure’s quality, interpreting results and grading and feedback This section's final question looks into speaking assessment issues for EOP classes The table below presents specific construction guidelines for this questionnaire
Table 3.1 Criteria for designing Questionnaire of English teachers’ speaking assessment practices
Indicators/ items Question Item number
II Objectives and targets of speaking assessment
IV Methods of speaking assessment
VI Procedures of assessment and evaluation
VII Designing speaking assessment tasks
IX Feedback in speaking assessment
XI Interpretation of speaking assessment results
XII Issues of speaking assessment of EOP
The second data collection method in this study involved interviews modeled after McMillan and Nash (2000) Teachers were prompted to discuss their reasons for choosing specific evaluation methods for classroom speaking and the time allocated for these assessments Furthermore, the interviews explored their views on speaking assessment for English for Occupational Purposes (EOP) and the challenges they encounter during the evaluation process.
Table 3.2 Frameworks for Interview of factors affecting teachers’ speaking assessment practice
External factors McMillan and Nash (2000)
Classroom realities McMillan and Nash (2000)
Richards and Pennington (1998) Factors related to EOP speaking assessment
Richards and Pennington (1998) Hijikata, Y., & Min, J (2021) Douglas (2000)
A total of nine participants engaged in semi-structured interviews, which were recorded with their full knowledge and consent to ensure that all important details were captured.
The process of collecting data for this research is described in details below
Step 1: Preparing official documents and ask for consent
The researcher obtained an official introduction letter from the University of Languages and International Studies to seek approval for the research, as required by the faculty Following this, the Head of the Faculty of English Teacher Education received the letter and granted permission for data collection Subsequently, invitations to participate in the research, along with consent inquiries, were sent to teachers, who agreed to take part in the study.
Questionnaires were handed out directly to all of teachers through Google docs Before they began to answer, careful instruction and further explanation for any question were provided
The interviewing procedure for this study happened in the order listed below
In order to check their agreement with semi-structured interviews, the research contacted 8 teachers Following that, specific information regarding the date, time, and location of interviews was given
To aid in data analysis, each teacher's interview was conducted separately and was recorded
Following the teacher interview, the recording was transcribed to create data that was available for analysis.
Data analysis
The research utilized a mixed-methods design, incorporating both descriptive statistics and interpretive techniques for data analysis The analysis of phase 1's questionnaire data involved three essential steps.
Step 1: Examine the data: When the questionnaires were received, the researcher carefully checked to see if the following conditions were met: (1) the questionnaire was completed; (2) the answers provided were accurate and easy to comprehend; and (3) the given data addressed the questions The researcher may go to the next stage only after making sure the surveys met all three requirements
Step 2: Report the data: According to each section of the questionnaire, analysis and syntheses of data from 32 surveys were done Using Excel, the frequency and importance of each choice in the particular section was computed in percentages The first study issue, which concerned the teachers' procedures for evaluating classroom speaking, was addressed with the use of the questionnaire results
Step 3: Visualize the results: The outcome was represented in bar charts, tables, and mixed charts in addition to being presented in written form with descriptions and numbers to help compare and generalize the data
To explore the factors affecting instructors' practices in classroom speaking evaluation, qualitative data from interviews were collected alongside quantitative data from questionnaires The analysis of the qualitative evaluation utilized an inductive approach as proposed by Thomas, D.
R (2006) was applied in this study to analyse information collected from the participants The procedure included five main steps which are shown in the following figure
Figure 3.1 The coding process in Inductive Analysis (Thomas, D R., 2006)
Step 1: Preparation of raw data files (data cleaning):
Initially, all recordings were transcribed into a standardized written format The researcher aimed to ensure accuracy in this process to preserve the original messages and ideas expressed by the participants.
Step 2: Close reading of text:
Once the material is prepared, the researcher thoroughly reviews the raw text to gain familiarity with its content, themes, and events.
The researcher identified and defined categories or themes by analyzing repeated patterns in the transcription data Key features were highlighted, and higher-level categories were derived from the evaluation's objectives In vivo coding, which involves lower-level categories, emerged from multiple readings of the raw data Inductive coding generated categories based on actual phrases or meanings found in specific text segments During this phase, participant-provided data were compared to identify potential themes, with Microsoft Word's review feature aiding the process.
Step 4: Overlapping coding and uncoded text:
In this phase, themes and codes were developed, adhering to qualitative coding principles that differ from quantitative methods Notably, a single text segment can be assigned to multiple categories, and a significant portion of the text—potentially over 50%—may remain uncategorized due to its irrelevance to the assessment objectives These insights, along with the established framework, guided the creation of themes and codes, as exemplified in the table below.
Figure 3.3 Example of themes and codes
Step 5: Continuing revision and refinement of category system:
The researcher aimed to explore the connections among codes, themes, and various levels of themes According to Thomas (2006), it is essential to identify subtopics within each category, such as contrasting perspectives and fresh insights Selecting relevant quotations that encapsulate the core idea of a category is crucial When categories share similar meanings, they can be merged under a broader category The study identified three primary themes: classroom realities, external factors, and EOP factors.
Summary
In summary, this chapter outlined the research design methodology, highlighting the research contexts and the participation of 32 tertiary-level teachers It also provided insights into the instruments used, as well as the processes of data gathering and analysis.
FINDINGS AND DISCUSSSION
Teachers‘ practices of Speaking assessment
In the initial section of the questionnaire, participants provided insights into their current practices regarding classroom speaking assessment Specifically, the first question focused on how often participants utilize various purposes in their classroom settings The analysis of their responses is depicted in the figure below.
Figure 4.1 Actual speaking assessment purposes
According to the analysis result, the ratios of three most regular purposes that participants chose were all above 70 precent The percentage of teachers regularly
Give feedback during lessons to arrange the learning and teaching process
Diagnose students’ strengths and weaknesses
Divide class into groups of similar ability, organise students for groups’ presentation and activities
Motivate my students to learn
Plan and adjust my instruction
In a recent survey, the use of speaking assessments to provide feedback during lessons and enhance the learning process was highlighted, with 78.1% of teachers indicating they use this method regularly Additionally, 71.9% of educators reported using speaking assessments to improve student achievement Furthermore, 70.1% of teachers regularly utilize these assessments to plan and adjust their instructional strategies.
A significant 66 percent of teachers utilize speaking skills to inspire student learning, while 65.6 percent aim to identify students' strengths and weaknesses Additionally, approximately 55 percent of teachers regularly employ speaking assessments to evaluate students' academic progress.
Less than 50% of instructors regularly utilized speaking assessments to group students by ability or organize presentations and activities, while 12% of teachers reported rarely using this method In contrast, for other purposes, no teachers or fewer than 10% indicated infrequent use.
Surveyed teachers showed a greater emphasis on formative assessment in classroom evaluations compared to summative assessment This observation aligns closely with the findings of McMillan (2015), McManus (2008), and Brown (2004).
The bar chart below illustrates current practices of participants related to objectives and targets used for oral assessment in their classroom
Figure 4.2 Actual speaking assessment objectives and targets
A significant majority of surveyed participants, 81.2 percent, regularly link their assessment targets with students' learning objectives, while 18.8 percent do so occasionally According to Nitko (2009), students enhance their focus on practice and study when they understand the skills required for performance All 32 instructors emphasized the importance of informing students about assessment targets prior to evaluating their speaking skills, as this influences their choice of appropriate assessment methods However, a small minority of teachers, 9.4 percent, rarely utilize assessment targets when selecting methods for speaking assessment.
A recent survey revealed that 62.5 percent of teachers frequently use action verbs from Bloom's taxonomy when writing assessment targets However, over 6 percent of respondents indicated that they rarely incorporate these action verbs in their assessments, highlighting a gap in effective assessment practices.
2001 claimed that teachers are advised to determine the objectives and targets based
Identify assessment targets before assessing speaking skill.
Utilize action verbs on Bloom’s Taxonomy in assessment targets
Link my assessment tarrgets with my learning objectives.
Utilize my assessment targets to choose assessment methods.
Inform students of my assessment targets before assessing their oral skills.
Never Rarely Sometimes Regularly on Bloom's modified taxonomy so that they re easy for teachers to follow and that helps students comprehend what is expected of them
Figure 4.3 Actual choice of Speaking construct
When designing speaking tasks, teachers prioritize five key factors, with over 90% regularly considering language competence as the foundation Fulcher (2003) defines language competence as encompassing phonology, accuracy, and fluency, which significantly impact students' conversational abilities, even when grammar and vocabulary are lacking Textual knowledge, reflecting the organization of speech, ranks second, influencing 75% of instructors In contrast, only about half of the teachers regard sociolinguistic knowledge as important, with 9% rarely considering it According to Bachman (1990), sociolinguistic knowledge involves understanding cultural differences.
Strategic capacity Textual knowledge Pragmatic knowledge
Never Rarely Sometimes Regularly communication and teachers also shared that the speaking construct in their EOP class have not paid much attention to this factor
The bar graph below demonstrates the choices of methods during the actual speaking assessment practices of surveyed instructors
Figure 4.4 Methods of speaking assessment
In EOP classrooms, role-play emerged as the predominant teaching method, utilized regularly by over 70% of teachers, while 28.1% employed it occasionally Interviews ranked second, with half of the surveyed instructors applying this method sometimes Additionally, more than 50% of teachers reported using presentations occasionally in their teaching practices.
In addition to the four primary methods discussed, picture-description and oral reading were occasionally utilized by many instructors, with 56.3% and 50% of teachers employing these techniques, respectively Furthermore, nearly 35% of educators also incorporated these methods into their teaching practices.
Never Rarely Sometimes Regularly percent of surveyed participants reported that they rarely used oral reading for speaking assessment
In contrast, the two methods ―debate‖ and ―story-telling‖ were least employed with the same figure 56.3 percent of participants rarely applying them in speaking assessment practices in their class
In comparison with previous research, the results shown in methods used for speaking assessment practices showed both similarities and differences Louma
Role-plays are frequently utilized in English for Occupational Purposes (EOP) classrooms because they effectively simulate various real-world situations that the target group may face, as noted by (2004).
In contrast to the findings of Munoz et al (2003), which indicated that the method interview was among the least frequently utilized tasks by teachers, this study reveals a significant shift, placing the method interview as the second most commonly used approach.
Teachers in the English faculty at the university utilized diverse oral assessment methods, with the most prevalent being role-play, interviews, and presentations This approach highlights the instructors' focus on enhancing students' communication skills in real-world situations.
―debate‖ and ―storytelling‖ are not preferable in EOP classroom as they are not much related to the skills students may need in their future career
4.1.5 Criteria for classroom speaking assessment
In the current study, teachers identified multiple criteria used to evaluate students' speaking abilities in the English for Occupational Purposes (EOP) class The accompanying figure illustrates the significance of each criterion employed in the assessment process.
Figure 4.5 The level of importance for each criterion to assess speaking performance
Specifically, there were 7 criteria which were taken into consideration including ―fluency‖, ―grammar‖, ―vocabulary‖, ―pronunciation‖, ―conversational skill‖, ―sociolinguistic knowledge‖ and ―functional knowledge‖ Among 8 criteria,
―vocabulary‖ received the most attention from surveyed participants with more than
Eighty percent of teachers indicated that field-specific criteria are crucial in English for Occupational Purposes (EOP) speaking assessments Friginal (2013) emphasized the importance of utilizing these criteria, highlighting that instructors particularly prioritize context-related vocabulary in their evaluations.
Followed that were two other criteria namely ―Pronunciation‖ and ―fluency‖ which were considered important with 56.3 percent and 53.1 percent of teachers reporting, respectively Three other criteria which are ―conversational skill‖,
―sociolinguistic knowledge‖ and ―functional knowledge‖ received the same level of attention with nearly 50 percent of teachers investigated (46.9 percent) stating that they were important when assessing oral performance
Fluency Grammar Vocabulary Pronunciation Conversational skill Sociolinguistic knowledge Functional knowledge
Very important Important Neutral Not important
On the other hand, more than 60 percent of instructors indicated that
Factors affecting the practices of classroom speaking assessment
Participants were surveyed on the significance of various factors influencing classroom speaking assessment practices, as illustrated in the accompanying bar chart.
Norm-referenced interpretations Criterion-referenced interpretations
Figure 4.10 Factors affecting teachers in their classroom speaking assessment
The interview examines the challenges teachers face during oral assessments in their English for Occupational Purposes (EOP) classes Transcriptions and coding of the teacher interviews have been completed.
From the results from the questionnaire and the interview, these factors can be divided into three categories including classroom realities, external factors and EOP factors
In a recent survey, nearly 60 percent of teachers identified students' linguistic proficiency as the most crucial factor in classroom dynamics Participants noted that linguistic proficiency significantly impacts the assessment of speaking skills, often creating challenges in selecting appropriate assessment items and tasks Ms Loan Do highlighted the importance of ensuring that assessment items are both authentic and aligned with students' language proficiency and syllabus requirements.
Class size Students’ linguistic proficiency
Very important Important Neutral Not important
Students' language proficiency significantly influences the assessment of speaking skills in the classroom Given the varying levels of linguistic ability among students, creating appropriate tasks that cater to everyone can be challenging.
As a teacher of freshmen, I observe significant variations in language proficiency among my students, both individually and across different skills Many students, during their high school years, prioritized grammar and reading over speaking and listening Consequently, I must implement additional tasks to evaluate their pronunciation and provide constructive feedback.
Class size emerged as a significant factor influencing classroom oral assessment practices, with 56.3% of teachers rating it as very important and 40.6% as important Teachers reported an average class size of 30 to 35 students, which considerably increased the time required for speaking assessments For instance, Teacher 1 noted that each student must submit a video clip for speaking practice weekly, making it time-consuming to evaluate all submissions Additionally, Teacher 2 highlighted that class size impacts the choice of assessment methods, leading teachers to select approaches that minimize time while maintaining assessment quality.
Assessing students' speaking skills through progress tests requires a minimum of two sessions per class, leading to significant waiting times for students This process can be time-consuming and often places students under pressure as they await their turn to speak.
Large class sizes can hinder effective speaking assessments, often relying on group or pair work that may not yield desired results A significant issue arises when students with lower language proficiency resort to using their native language during these activities, making it challenging for instructors to manage multiple groups effectively.
Teacher 5 stated that class size had a minimal impact on her speaking assessment method, as she typically selected her approach based on the syllabus and course organization, making the decision process straightforward.
Over 40 percent of teachers believe that student motivation plays a crucial role in their speaking assessment practices In interviews, five out of nine teachers indicated that motivation significantly influences their chosen assessment methods Teacher 3 noted that incorporating a variety of assessment methods can help maintain student motivation during the speaking assessment process.
To effectively engage students with varying levels of proficiency, I tailor speaking tasks that capture their interest It is essential for all students to have the opportunity to enhance their speaking skills, as this ability is crucial for their future careers Encouraging them to learn and improve in this area is a priority.
A significant factor influencing the assessment of students' oral skills is their behavior, with over half of the teachers (53.1 percent) rating it as important Teacher 1 emphasized the necessity of ensuring full participation in group and pair work activities for speaking assessments, stating that she consistently adapts her teaching methods to achieve this goal.
Nearly 60 percent of instructors considered teachers’ workload important when carrying out speaking assessment In the interview, all 9 teachers said that their workload was heavy at university On average, interviewed participants taught
20 periods per week (5 classes) In addition to teaching, they also had to fulfil scientific research every semester As a result, the excessive workload influences teachers a lot on classroom speaking assessment
Each week, my students must submit at least one video related to the topic they studied or engage in a conversation with their peers This results in approximately 30 videos per class for me to evaluate weekly, leaving me with insufficient time to thoroughly assess all submissions.
In addition to my teaching responsibilities, I participate in university meetings and workshops, and engage in scientific research through journal writing and data collection Consequently, this often leaves me with limited time to explore new methods for assessing students' speaking skills.