04051001955 Localizing the CEFR spoken production and spoken interaction descriptors to the Vietnamese context = Bản địa hoá bản đặc tả nói tương tác và nói sản sinh theo khung CEFR trong bối cảnh Việt Nam.
INTRODUCTION
Rational of the study
Assessing oral production is challenging due to the dual nature of language, which includes explicit knowledge gained through formal education and implicit knowledge acquired from multimedia and real-life interactions During oral tests, students must effectively process information to utilize grammar, vocabulary, and phonology, while also demonstrating sociolinguistic competence These demands create significant challenges for students striving to achieve native-like speech Consequently, language examiners face the critical task of establishing clear and objective criteria for evaluating oral skills that encompass all facets of effective speaking ability.
Language scholars have emphasized the importance of developing criterion-referenced assessments for speaking skills (Liu & Jia, 2017) Influential international frameworks, such as the Common European Framework of Reference (CEFR), have shaped language assessment practices, yet there is a growing demand for localized tests to cater to diverse learner needs in EFL countries, exemplified by Taiwan's General English Proficiency Test (GEPT) and China's Fudan English Test (Wu, 2012; Fan & Ji, 2014) Each country faces unique challenges in creating frameworks for English proficiency assessment, particularly in evaluating Spoken Production and Spoken Interaction Descriptors For instance, Taiwanese students often lack opportunities for social communication in English, hindering their expressive abilities In China, the development of assessment frameworks is complicated by the education system's nature and resistance from stakeholders The Ministry of Education of China is tasked with further research and implementation of the CEFR framework, which many educators believe requires adjustments and innovative perspectives to align educational goals with the diverse social and educational contexts of each country (Read, 2014).
The Vietnamese Standardized Test of English Proficiency (VSTEP), approved in 2015, is increasingly adopted by local educational institutions in Vietnam as a substitute for international English proficiency exams It serves as an alternative to well-known tests like TOEIC, PET, KET, and IELTS.
Since the introduction of VSTEP, numerous training programs have been implemented for test examiners, writers, and validators Several studies have highlighted the positive impact of these training programs on test validity and ratings (T N Q Nguyen, 2018; Nguyen et al., 2020; T P T Nguyen, 2018).
Despite previous studies, there has been a lack of focus on adapting the CEFR spoken production and interaction descriptors to the Vietnamese context This gap in research prompted the author to select the topic "Localizing the CEFR Spoken Production and Interaction Descriptors for Vietnam."
Spoken Interaction Descriptors to the Vietnamese Context” as the topic of my master's thesis.
Aim, objective and research questions
The study was conducted to collect participants' perceptions about localizing the CEFR spoken production and spoken interaction descriptors to the Vietnamese context
(1) Investigating the teachers‟ opinions on the need, feasibility, and suitability of the CEFR spoken production and spoken interaction descriptors to the Vietnamese context
(2) Investigating the teachers‟ suggestions on the adjustment of the CEFR spoken production and spoken interaction descriptors to the Vietnamese context
To achieve the goal of the study, the topic focuses on clarifying specific research questions as follows:
(1) What are the teachers‟ opinions on the need, feasibility, and suitability about the CEFR spoken production and spoken interaction descriptors to the Vietnamese context?
(2) What are the teachers‟ suggestions on the adjustment of the CEFR spoken production and spoken interaction descriptors to the Vietnamese context?
Scope of the study
Spatial scope: The study was conducted within CEFR and Vietnam's VSTEP English proficiency assessment framework
Time range: Primary data is expected to be collected through survey questionnaires and semi-structured interviews from December 15, 2022, to January 15, 2023.
Methodology
This study will utilize a quantitative research design to gather statistical data relevant to the research questions As noted by Creswell (2014), quantitative research tests objective theories by exploring relationships among variables This approach employs various measurement methods to interpret the connections between factors, particularly in phenomena that can be quantified (DeFranzo, 2011) Additionally, the quantitative method is integral to the deductive research approach, aimed at testing established theories.
Quantitative research relies on numerical data, including quantities, proportions, and measures, which allow researchers to assess degrees and make comparisons This data enables the calculation of average values, providing valuable insights into the subject of study.
This study utilized a survey questionnaire to gather participants' opinions on the successes and limitations of localizing the CEFR Spoken Production and Spoken Interaction Descriptors within the Vietnamese context during the VSTEP speaking assessment.
Qualitative research, as defined by Creswell (2014), is an approach aimed at exploring and understanding the significance of individuals or groups in relation to social or human problems This research method enables researchers to gain insights into the opinions, motivations, behaviors, and attitudes of their subjects DeFranzo (2010) notes that qualitative methods not only provide valuable information about issues but also support quantitative research by generating new ideas and theories Qualitative data is collected through various methods, including unstructured and semi-structured interviews and focus groups One of the key advantages of qualitative research is its ability to offer detailed insights into participants' feelings and opinions, thereby enriching the understanding of a phenomenon (Rahman, 2017).
On the downside, it takes a long time to collect data Research focuses on experience, so data can be subjective and biased (Silverman, 2010)
This study involved semi-structured interviews with experienced lecturers who assessed speaking skills based on Vietnam's VSTEP standards The focus was on their insights regarding the functional status, successes, and limitations of adapting the CEFR Spoken Production and Spoken Interaction Descriptors within the Vietnamese context.
Thesis structure
The thesis consists of four main chapters as follows:
Chapter 1: Introduction -provides an overview of the research, including: research background, research objectives, problem statement, research questions, scope and significance of the study, and thesis structure
Chapter 2: Literature review provides the theoretical basis system related to the topic
Chapter 3: Research Methods- focuses on introducing research methods, data collection tools, research data collection, and analysis procedures
Chapter 4: Results and Discussion focuses on analyzing research data collected from research tools and conducting research discussions
Chapter 5: Conclusion presents the research's findings and suggests some solutions Chapter 5 also discusses research limitations as well as future research directions.
LITERATURE REVIEW
Overview of the CEFR
2.1.1 Circumstances of birth of the CEFR
In the 1990s, the need for a comprehensive framework for language learning, teaching, and assessment led to the introduction of the Common European Framework of Reference for Languages (CEFR), which marked a significant shift in language education philosophy The CEFR emphasizes the interaction between different languages and promotes multilingualism, rather than focusing solely on mastering individual languages Language plays a vital role in developing cultural customs, conveying information, and facilitating communication By encouraging multilingualism, the CEFR enables individuals from various nations to connect and understand one another, addressing a societal need and providing numerous benefits.
The Common European Framework of Reference for Languages (CEFR) was first proposed in 1991 at a Council of Europe symposium in Rüschlikon, in partnership with Swiss authorities In 1992, a working party was formed to collaborate with a Swiss research group, funded by the Swiss National Science Foundation, aimed at creating and standardizing language proficiency descriptors Four members of this working party were chosen to author the CEFR.
The CEFR framework marks a pivotal shift in language teaching philosophy, moving away from traditional grammar-translation methods towards a more functional and communicative approach This transition highlights the evolving methodologies in language education, emphasizing practical communication skills.
The CEFR framework serves as a comprehensive tool for assessing the needs of second language (L2) learners, establishing learning objectives, and guiding the creation of relevant activities and materials It focuses on analyzing language use through the strategies learners employ to activate their communicative competencies, enabling them to effectively engage in tasks across various social contexts This framework also provides criteria for evaluating L2 learning outcomes, ensuring that learners can successfully navigate communicative situations.
This toolkit describes the use of language both vertically and horizontally
The vertical axis categorizes communication competence into six levels, organized into three tiers: A1 and A2 for basic users, B1 and B2 for independent users, and C1 and C2 for proficient users This classification highlights the varying degrees of language proficiency.
“there is a high consensus on the number and content of qualifications suitable for the organization of language teaching and the general recognition of learning outcomes” (Council of Europe, 2001a, p.22f.)
Verbal communication encompasses reception, production, interaction, and mediation, with a focus on assessing receptive skills like listening and reading, productive skills such as speaking and writing, and interaction skills for information exchange The user-centered vertical scales are designed to reflect communicative behavior, highlighting what learners can achieve in their target or L2 language These scales are clear and practical for learners, facilitating their understanding and application.
The horizontal axis of the frame of reference highlights the importance of communicative linguistic competencies and strategies, which serve as connections between language knowledge and communicative activities While competencies and strategies are also represented on a scale, their significance is secondary to the scale of communication behaviors, which play a crucial role in how learners apply their linguistic knowledge.
The reference framework does not prescribe a specific language teaching method, yet it emphasizes communicative competence through task-based teaching and learning This perspective is supported by a thorough analysis of the roles and tasks of teachers in language education While the framework does not dictate how to assess learners' communicative competence, it aligns with a communication-oriented approach rooted in action theory Additionally, the toolkit outlined in the framework reflects European language diversity and curriculum policy.
2.1.3 Purpose and principles of the CEFR framework
The reference framework serves as a foundational tool for creating lesson plans, curriculum guides, examinations, and textbook compilations across Europe, aligning with the continent's political, cultural, and educational objectives (Council of Europe, 2001a, p.1).
Language learning is a long-term endeavor that encompasses various aspects of the process The ultimate aim is to foster a multilingual and multicultural European society that benefits both education and the daily lives of its citizens Consequently, the focus is not on achieving "absolute competence" in a language, as many learners prioritize specific skills, such as listening and speaking, or aim to enhance their reading abilities through conversation.
2.1.4 Scales in the frame of reference
The frame of reference plays a crucial role in language description, despite not being designed to define scales in language use Understanding the various scales within this framework, such as the broad scale, self-assessment scale, and those related to language activities like listening, reading, speaking, and writing, provides significant advantages These scales are interconnected with the language competence scale, which encompasses aspects like vocabulary range, grammatical accuracy, and sociolinguistic knowledge Additionally, the communication strategy scale is essential, as it emphasizes the use of effective strategies during communicative acts to optimize language resource utilization.
Scales effectively illustrate learning outcomes, beginning from lower levels that align with teaching requirements outlined in lesson plans and textbooks It is crucial to understand that these scales do not merely request language acquisition; instead, they describe the process of acquiring language Consequently, learning outcomes represent an expansion of the communicative abilities that learners can achieve, rather than just a progression within a single area of communication This perspective was highlighted by the editors in the frame of reference.
Scale is not a hierarchical ranking system; within the same language class, learners can exhibit varying proficiency levels, such as C2, C1, and B2 This variation highlights that the scale represents a sequence of learning outcomes achieved over many years of study.
Behavioral aspects at the highest scales of language proficiency encompass learner maturity, overall academic achievement, and professional experience Speaking at the A1 level allows learners of any age to use simple phrases to describe their surroundings and acquaintances, while C2 level speaking skills involve the ability to clearly and fluently summarize complex ideas in a contextually appropriate manner This advanced level of communication often exceeds the cognitive capabilities of elementary and middle school students, and much of the C2 content is typically beyond the grasp of most high school learners Consequently, the proficiency scales can be adjusted to better meet the needs of young learners Thus, effective communication does not always necessitate a high level of language competence.
2.1.5 Actual situation of using CEFR to assess foreign language competence in the world
The CEFR framework is adopted by countries such as Taiwan, Japan, China, and Vietnam to evaluate English proficiency based on their unique needs In Taiwan, it is specifically used to create equivalence among different tests administered by educational institutions, notably the National General English Proficiency Test (GEPT) (Wu, 2012).
The Vietnamese Common European Framework of Reference for
2.2.1 The National Foreign Language 2020 Project
The Vietnamese Government has consistently prioritized the enhancement of foreign language education, making it a mandatory subject from general education through to graduate levels since the 1960s This ongoing commitment has led to notable advancements in foreign language teaching across the country Recognizing the importance of foreign languages, the Prime Minister of Vietnam signed Decision No 1400/QĐ-TTg on September 30, 2008, which initiated a National Project aimed at improving the teaching and learning of foreign languages.
Languages in the National Education System, Period 2008- 2020 (namely
2020 Project) was promulgated then The goal of the 2020 Project is
To enhance the teaching and learning of foreign languages within the national education system, a comprehensive program will be implemented across all school levels and training degrees By 2015, the goal is to significantly improve professional skills and language competency, particularly in prioritized sectors By 2020, it is expected that most Vietnamese youth graduating from vocational schools, colleges, and universities will possess the ability to use a foreign language independently This proficiency will boost their confidence in communication and expand their opportunities for study and work in diverse, multicultural environments Ultimately, this initiative aims to position language skills as a key advantage for the Vietnamese population, supporting the country's industrialization and modernization efforts.
The 2020 Project undergoes three phases The first phase is from 2008 to 2010; the second phase, from 2011 to 2015; and the third phase, from 2016 to
In 2020, the initiative began with a focus on establishing optimal conditions for the development and piloting of a new language program, aimed at preparing for its widespread implementation in general schools The second phase emphasizes the large-scale rollout of a comprehensive ten-year language program for general education, alongside intensive language training for various educational levels Finally, the third phase prioritizes the nationwide implementation of the ten-year language program and the introduction of intensive language courses across all training centers, vocational schools, colleges, and universities.
The Ministry of Education and Training (MOET) has adopted the Common European Framework of Reference for Languages (CEFR) and established the CEFR-V to provide a standardized guideline for measuring and assessing language proficiency The CEFR aids in curriculum design, teaching practices, and evaluation, promoting democracy and accountability in education It is user-friendly for teachers, supported by extensive teaching and research materials, and encourages student independence in learning Given its widespread use in Europe and beyond, the CEFR is deemed an ideal standard for implementation in Vietnam.
On January 24, 2014, the Ministry of Education and Training (MOET) signed Circular No 01/2014/TT-BGDĐT to establish the CEFR-V framework, aimed at standardizing foreign language proficiency requirements within the national education system This framework is based on the Common European Framework of Reference for Languages (CEFR) and introduces three main proficiency levels: Elementary Level (CEFR Basic User), Intermediate Level (CEFR Independent User), and Advanced Level (CEFR Proficient User) Each of these levels is further divided into two sublevels, resulting in a comprehensive 6-level proficiency framework for foreign language education.
Table 1.1 The 6 levels of the CEFR-V
Level group Level Level name
A – Basic user 1 (A1) Breakthrough or beginner
B – Independent user 3 (B1) Threshold or intermediate
C – Proficient user 5 (C1) Effective operational proficiency or advanced
2.2.3 The use of the CEFR –V
Since 2008, the NFL 2020 Project has significantly impacted English language training for teachers at all levels, establishing standards for general and higher education, as well as for government officials According to MOET documents of the CEFR-V, high school English teachers must possess a bachelor's degree in English or related fields and a master's degree in TESOL or equivalent for tertiary education High school teachers are expected to achieve Level C1 proficiency, while secondary and primary teachers must reach Level B2 Although the CEFR-V has mainly been applied in education, it is anticipated to have a broader national application The foreign language proficiency requirements for postgraduate candidates (B1 for master's and B2 for doctorate degrees) are expected to enhance human resource development across various industries, particularly for professionals seeking further education Thus, implementing the CEFR-V in education is a strategic choice to equip future generations for regional and international integration.
To achieve level 3 (B1), students must comprehend the main points of clear standard input related to familiar topics in work, school, and leisure They should effectively handle most situations encountered while traveling in areas where the language is spoken Additionally, students need to produce simple connected text on familiar subjects or personal interests Ultimately, they should be able to describe experiences, events, dreams, hopes, and ambitions, while also providing brief reasons and explanations for their opinions and plans.
Theory of spoken interaction and spoken production
Bygate (1987) considers Speaking as a form of expression of ideas that uses language to express emotions In addition, Bygate (2001) also states that
Speaking is a fundamental language skill that enables learners to express their thoughts and ideas effectively According to Bygate and Bailey (2005), speaking should primarily be analyzed by focusing on its key aspects Harmer (2007) and Karim & Haq (2014) emphasize that this skill is essential for meaningful communication in conversations.
Speaking is an interactive activity that involves generating, receiving, and expressing information (Nasiri & Gilakjani, 2016) It is characterized by specific patterns influenced by the purpose, topic, and setting of the conversation (Correia, 2016) The spectrum of spoken interaction ranges from casual to everyday talk, highlighting the diverse nature of communication (Eggins & Slade, 1997, as cited in Itkonen, 2010).
Research in various countries has examined students' self-assessment of their spoken interaction skills based on the Common European Framework of Reference (CEFR) descriptors A notable study by Asdar (2017) in Indonesia, titled "Students' Self-assessment on Their Spoken Interaction Using CEFR," involved 30 first-semester students from the English Education Department The findings revealed that 27 students rated their spoken interaction skills below the B1 (Threshold User) level, with many identifying themselves at A1 (Breakthrough level) or A2 (Way Stage User) Only 3 respondents considered themselves at the B1 level, highlighting a general perception of low spoken interaction proficiency among the students.
A study conducted in Turkey by Korkmaz (2017) revealed that 86% of respondents felt capable of identifying discussion topics when people spoke slowly and could express their likes and dislikes during spoken interactions Additionally, 82% believed they could understand simple daily conversations if the speaker repeated the information Furthermore, 78% of participants felt they could describe themselves, their family, and others effectively.
A study by Fasoglio and Tuin (2018) involving forty secondary schools in the Netherlands revealed that nearly 40 percent of secondary school students self-assessed their English speaking skills as below level A2, while the remaining students rated themselves at level A2 according to the CEFR proficiency levels.
Oral production, as defined by the Council of Europe (2011), involves speaking based on an oral text to an audience, which can include activities like speeches, sales presentations, public addresses, and sports commentaries During these tasks, speakers are not required to use perfect sentences; instead, they should adapt their speech to fit the context.
Research by Geranmayeh et al (2014) and Nishimaki (2014) indicates that spoken production can be disorganized due to the complexities of linguistic processes and brain control over human utterances Numerous studies have investigated the speaking production skills of English second language learners, focusing on students' self-assessments of their oral skills, their perceived challenges during presentations, and their views on the effectiveness of speech training programs.
Rian, Hinkelman, and Cotter (2015) conducted research on students' self-assessment of their presentation videos, revealing that students rated their performance lower than both their peers and lecturer Despite initial feelings of embarrassment, the students valued the opportunity to watch and evaluate their own presentations.
2.4.2 Assessment of speaking skills based on CEFR descriptors
When evaluating speaking skills, it is essential to examine the detailed components of speaking ability, including sound, grammar, vocabulary, and features of language production, as highlighted by Luoma (2004) Key aspects to assess include pronunciation, oral proficiency, and sociolinguistic factors These elements serve as the foundation for designing and developing effective speaking skills assessments, determining the content that will be tested.
Lado (1961) emphasizes that while speaking a foreign language is a highly valued skill, it remains the least assessed and practiced aspect in language testing.
Speaking involves using verbal language for communication Fulcher (2003) identifies key components of speaking skills assessment, which include audio characteristics such as pronunciation, intonation, accuracy, and fluency Additionally, psychological factors, speaking strategies like word creation and correction, structural elements of speech including openings and closings, pragmatic aspects, vocabulary, and overall speech production are essential for effective communication.
Field (2004) develops a speech production model based on Levelt's framework (1989, 1999), emphasizing that all speech production, whether in a first or second language, involves specific stages.
- Conceptual stage: this is the stage when speech concepts begin to form in the speaker's head
- Systematic stage is when the speaker chooses a suitable frame to create words, emphasizing the harmony between verbs, etc
- Vocabulary stage: when the speaker searches for words or words in his vocabulary, this process is aided by signs in the form, such as words or syllables
- Sound generation stage: this is the stage when information is gathered so that it can be transformed into speech
- Pronunciation stage: this is the stage when the characteristics of the sound are formed
Speech-making stage: at this stage, speech is formed
Weir's sociological model of thinking emphasizes the importance of the validity of the thought process As noted by Weir (2005), this validity is established through the representation of the thinking process during the pre-exam period Additionally, the analysis of test content in relation to candidates' scores provides post-test evidence of this validity.
Language test designers must grasp the candidate's thinking process, ensuring it mirrors real-life language use (Taylor, 2011) Field (2011) utilizes Weir's research to implement Levelt's model of speech production (1999), creating a practical framework for understanding the cognitive processes involved in speaking skills.
The unique processes studied in this model include:
- Concept formation: is the stage of generating speech ideas
- Grammatical coding: is the stage of forming a syntactic framework and arranging the sounds that will be used
- Audio coding: is the stage of converting the abstract concepts of the previous stage into a sequence of words and sounds
Pronunciation coding involves organizing sounds to facilitate quick pronunciation It connects sounds into a sequence of instructions and pronunciations, while also storing these instructions in a waiting area until the prepared clause is articulated.
- Pronunciation: is the stage of speech production
- Self-control: this is when the speaker focuses on the verbal message just before speaking and immediately after speaking to check the verbal message's accuracy, clarity, and appropriateness
Key terms
In this study, a number of terms appear that will be explained below:
The urgency of establishing evaluation criteria is essential for enhancing the effectiveness of Spoken Production and Spoken Interaction Descriptors in English teaching and learning in Vietnam.
Feasibility : the possibility of specification skill to be reasonable to be used in the teaching and learning context of Vietnam
Suitability : this is suitable for implementing assessment criteria in the current context of teaching and learning English in Vietnam
Adjustment: A desire for changes to be made so that the CEFR specification model will be more suitable to the foreign language teaching and learning process of the Vietnamese context
This study aims to gather teachers' opinions on the necessity, feasibility, and suitability of adjustments to the specification framework Additionally, it will collect suggestions from teachers regarding these adjustments to better align with their needs and preferences.
Related studies
Many countries, including Vietnam, have adapted the CEFR framework to better fit their specific contexts, leading to numerous published studies on its localization.
Duong and Trinh (2014) examined the perceptions of lecturers and students regarding EFL policy and practice at Can Tho University Their study utilized both quantitative and qualitative data from questionnaires and in-depth interviews The findings indicated that the majority of the policies, particularly those focused on student engagement and improving the collaboration among student support units, were deemed effective in enhancing the teaching quality of Basic English courses at the university.
The CEFR-V has not been effectively integrated into university English teaching, with both lecturers and students lacking adequate mastery of it To enhance proficiency assessment and teaching, it is essential to implement the CEFR-V while strengthening its policies and investing in lecturers' professional development Key recommendations for effective implementation include publicizing the CEFR-V, providing lecturer training, selecting appropriate materials, and establishing suitable facilities and test formats Additionally, the findings highlight the need for pedagogical practices that align with the CEFR-V syllabus, advocating for a blend of traditional methods and communicative approaches to enhance classroom instruction and foster professional learning communities (Duong & Trinh, 2014).
Nguyen's 2015 study examines educational policy borrowing in a globalized context, focusing on the CEFR implementation at a Vietnamese university Through qualitative analysis of policy documents, classroom observations, and in-depth interviews, the research suggests that adopting the CEFR serves as a "quick-fix" solution to the intricate challenges of improving English language education However, it highlights that this approach does not adequately tackle significant issues in the practice of foreign language teaching and learning in Vietnam.
Nguyen (2016) explored the challenges faced by ethnic minority students in implementing national proficiency standards at Tay Bac University The study utilized both quantitative and qualitative data from placement tests, questionnaires, and interviews Findings indicated that these students encountered several difficulties, including a tri-language barrier, limited English proficiency, negative attitudes, lack of motivation, ineffective learning strategies, inactivity, and excessive anxiety in learning English Additionally, they faced challenges related to a difficult curriculum, unfamiliar textbooks, teacher behavior, and insufficient time for study.
A study by Al-Nouh, Abdul-Kareem, and Taqi (2015) examined students' perceptions of the challenges associated with oral presentations The results indicated that students viewed these difficulties as moderate, with 57.4% expressing that the conversation course did not sufficiently enhance their speaking skills Additionally, 44.4% attributed their low performance to a lack of oral-skill courses Students believed that their presentation skills could be improved by engaging in presentations in other classes.
A study conducted by Syafryadin, Nurkamto, Linggar, and Mujiyanto (2016) explored students' perceptions of the significance of speech training The findings revealed that students were enthusiastic about participating in the training, responding positively to its implementation in the classroom They believed that the speech training enhanced their speaking skills and helped alleviate their speech anxiety.
Adapting the CEFR framework to guide teaching to suit Taiwan's social and cultural conditions is also very difficult (Wu, 2012) Cheung
In 2012, it was noted that Taiwanese students, learning English as a foreign language, require opportunities for social communication in English to effectively demonstrate their skills according to the CEFR framework Cheung emphasized that adapting the CEFR framework to accurately assess the English proficiency of primary school students involves significant effort.
Between 2008 and 2011, researchers from the Tokyo University of Foreign Studies developed the CEFR-J framework, which adapts the Common European Framework of Reference (CEFR) to better fit the unique educational context of Japan (Negishi & Tono, 2014).
Educators generally believe that the CEFR framework requires adjustments to better align with the diverse social and educational contexts of different countries There is a recognized need for innovative thinking to redefine educational goals and programs However, recent research on localizing the CEFR framework has not addressed the localization of the Spoken Production and Spoken Interaction Descriptors.
RESEARCH METHODOLOGY
Research design and method
This study employs a mixed methods approach for data collection and analysis, integrating both qualitative and quantitative research tools such as questionnaires and interviews As a result, it falls under the category of "mixed methods research," a term that is gaining prominence in the research community While definitions may vary, mixed methods research fundamentally involves the combination of different research methodologies.
Mixed-methods research involves the integration of qualitative and quantitative approaches by researchers to achieve a comprehensive understanding of a subject This methodology utilizes both qualitative and quantitative perspectives, along with diverse data collection and analysis techniques, to enhance the depth and breadth of insights and validation.
Mixed methods research recognizes the inherent strengths and weaknesses of various methodologies, enhancing the richness and utility of the data collected (Johnson, 2007) By integrating both quantitative and qualitative approaches, researchers gain diverse perspectives, leading to a deeper understanding of the subject matter Additionally, the combination of different methods bolsters the reliability of the findings (Allwright, 1991; McDonough, 1997; Silverman, 2008).
This study employed both quantitative and qualitative research methods to evaluate the status, achievements, and challenges faced by English language teachers in localizing the CEFR Spoken Production and Spoken Interaction Descriptors within the Vietnamese context The qualitative approach was particularly valuable for gaining deeper insights into teachers' perceptions regarding the localization process.
Participants of the study
The study involved 42 English language teachers, all of whom hold bachelor's or master's degrees in English and are well-versed in the Common European Framework of Reference for Languages (CEFR), having achieved B2 or C1 levels These educators utilize the CEFR framework for training and assessing their students Additionally, all participants have over five years of experience in teaching English, with a specific focus on teaching and testing English speaking skills, as confirmed by the screening questionnaire.
Research instruments
Questionnaires are an effective tool for collecting vast amounts of information simultaneously, as highlighted by Brown (1988), who noted their efficiency in large-scale data gathering Key advantages of this method include its practicality, scalability, and the ability to maintain high levels of confidentiality and anonymity (Fowler, 2009; McDonough, 1997) Respondents can choose whether to disclose personal information, minimizing the risk of identity exposure Additionally, questionnaire responses are easily quantifiable and can be processed using software, allowing researchers to quickly and clearly present their findings.
Questionnaires can include a range of items, such as closed questions, open questions, dichotomous options, multi-option selections, and Likert scale items with varying points (Cohen, 2011) Closed questions provide quantifiable data that is easier to analyze due to their limited and predetermined choices (Creswell, 2008; Wiersma, 2008), while open questions allow respondents to express their thoughts in their own words (Bransford, 2000; Punch, 2009) Seliger and Shohamy (1989) highlight the efficiency of closed-ended questions for analysis, whereas Gillham (2000) suggests that open questions can facilitate deeper insights (Gillham, 2000; Seliger, 1989).
A questionnaire was developed to gather data for this study, focusing on participants' evaluations of the necessity, feasibility, and relevance of the CEFR Spoken Production and Spoken Interaction Descriptors within the Vietnamese context The questionnaire's primary content is organized into assessment criteria for levels B2, B1, A1, and pre-A1.
(4) Sustained monologue: putting a case (e.g in a debate)
The scales used are as follows:
The urgency of establishing evaluation criteria is essential for enhancing the effectiveness of assessing Spoken Production and Spoken Interaction in the context of English teaching and learning in Vietnam.
Feasibility : including all factors related to the implementation into the analysis, including both internal and external resources, to evaluate the successful performance of the project
Suitability : this is suitable for implementing assessment criteria in the current context of teaching and learning English in Vietnam
The second question in the survey aimed to gather teachers' insights on suggested changes to the CEFR Spoken Production and Spoken Interaction Descriptors, specifically tailored for the Vietnamese context.
Interviews serve as an effective tool for gathering in-depth opinions and feelings on specific topics, events, or actions (Kvale et al.) They enable a thorough exploration of situations through carefully crafted questions, allowing for flexibility in the conversation This adaptability permits interviewers to modify the sequence of questions, skip certain inquiries, or delve deeper into specific responses, particularly in less structured formats (Roulston, 2010).
Interviews can be categorized as structured, primarily used in quantitative research, or unstructured, allowing for a more flexible dialogue To achieve a balance between these approaches, semi-structured interviews are recommended, as noted by Bryman (2008) and Kvale (2009) In semi-structured interviews, the interviewer is not confined to a strict set of questions; instead, they can provide clarifications, delve deeper into specific inquiries, or pose additional questions, encouraging the interviewee to explore various perspectives and offer more comprehensive responses.
Considering the above considerations, a semi-structured interview was applied in this study It was conducted face-to-face randomly, with 03 out of
A survey involving 42 teachers was conducted to maximize the use of collected data The researcher developed four questionnaires addressing three research questions, focusing on the necessity, feasibility, and relevance of the CEFR Spoken Production and Spoken Interaction Descriptors within the Vietnamese context Additional questions may be included during interviews, allowing participants to respond in Vietnamese to express their thoughts and feelings more effectively and comfortably.
Procedures
Following the design of the questionnaire, a pilot study was conducted with five experienced English teachers to evaluate the appropriateness of the survey questions The findings revealed that all participants found the questionnaire to be well-structured, easy to comprehend, and aligned with the research objectives Consequently, the questionnaire was incorporated into the formal study.
The research utilized convenience sampling, where the researcher leveraged personal relationships to invite participants to partake in the study Upon agreement, a printed questionnaire was distributed to 42 teachers who consented to participate in the survey.
The researcher warrants that all personal information collected will be kept confidential and used for research purposes only All questionnaires were completed and returned to the researcher
After collecting the questionnaires, three semi-structured interviews were conducted with a randomly selected group of three teachers from the initial 42 participants Each interview involved a sequence of eight or three questions from the Interview Sheet, with the teachers taking turns to respond The researcher also had the flexibility to ask additional questions that arose during the discussions, allowing for the inclusion of students' opinions According to Gorman and Clayton (2005), this hand-to-hand interviewing method is effective for data collection, as it yields rich qualitative data with a moderate time commitment from both participants and researchers This approach facilitates the emergence of diverse attitudes and beliefs, while also providing opportunities for immediate feedback and clarification, allowing the researcher to observe nonverbal cues such as gestures and facial expressions.
Data analysis
The study employed a concurrent triangulation design, incorporating a simultaneous analysis of both qualitative and quantitative data (Creswell, 2007; Guest, 2013) Initially, separate analyses were conducted for each dataset, followed by a merging of the two to create a comprehensive understanding from both data sources (Creswell & Clark).
A comprehensive evaluation was conducted among 42 English teachers, all of whom hold bachelor's or master's degrees and possess over five years of teaching experience The assessment focused on the necessity, feasibility, and suitability of specific skills, as well as the need for any adjustments Following the survey, the researcher will calculate the average scores for necessity, feasibility, appropriateness, and the extent of required changes based on the collected data.
The data collected from questionnaires were analyzed using SPSS software, which provided the distribution of teachers' responses for each item as percentages Additionally, descriptive statistics, including means and standard deviations for Likert-scale items, were computed The quantitative findings will be displayed in tables and charts for clearer visualization.
The interviews were transcribed and translated into English by the researcher, following the practice recommended by Kvale and Brinkmann (2009) for easier reference and analysis The transcripts underwent thematic analysis based on the research questions, and the data were subsequently quoted and analyzed qualitatively.
In this study, the researcher conducts an interview with a participant and subsequently takes notes on paper A blue pen is used to highlight appropriate specification frames, a yellow pen marks limitations, and a red pen indicates directives for modification.
The quantitative and qualitative results were analyzed individually and then compared and combined to represent the participants' evaluation of the research issue This approach, referred to as merging two databases, is commonly used in concurrent mixed methods design, as noted by Creswell and Clark (2011).
FINDINGS AND DISCUSSION
Findings
4.1.1.1 Descriptive statistics of the study sample
Information about the surveyed subjects is as follows:
Chart 4 1 Information about the age of the surveyed subjects
Chart 4 2 Information about the level of the surveyed subjects
The survey results indicate that the majority of respondents, 52%, are aged between 31 and 35 years, while 21% fall within the 36 to 40 age range Overall, 73% of the teachers surveyed are between 31 and 40 years old, with the remaining 27% being over 40 Notably, there were no participants under the age of 25.
In terms of qualifications, the majority of teachers have Master's Degree (90%), 10% Bachelor's Degree (10%)
Chart 4 3 Information about the experience of the respondents
In terms of experience, the majority of teachers have 11-15 years of experience, accounting for 33%; 6-10 years accounted for 31%
Thus, the surveyed teachers have an average age of 31 to 40 years old, a Master's Degree (90%), and 11-15 years of experience
The survey results of 42 teachers showed that:
Table 4 1 Overall spoken production results from Pre-A1 to B2
Avarage Ranking Avarage Ranking Avarage Ranking yes No
Can give clear, systematically developed descriptions and presentations, with appropriate highlighting of significant points, and relevant supporting detail
Can give clear, detailed descriptions and presentations on a wide range of subjects related to their field of interest, expanding and supporting ideas with subsidiary points and relevant examples
Can reasonably fluently sustain a straightforward description of one of a variety of subjects within their field of interest, presenting it as a linear sequence of points
Can give a simple description or presentation of people, living or working conditions, daily routines likes/ dislikes, etc as a short series of simple phrases and sentences linked into a list
Skills Necessity Feasibility Suitability Adjust
Avarage Ranking Avarage Ranking Avarage Ranking yes No
Can produce simple, mainly isolated phrases about people and places
Can produce short phrases about themselves, giving basic personal information (e.g name, address, family, nationality)
The survey results indicate that the assessed skills of the qualifications are deemed necessary, feasible, and sustainable, with average scores of 1.98, 1.86, and 1.95, respectively However, there is still a need for adjustment, as reflected by an average score of 52%.
At the B2 level, two key skills are assessed: the ability to provide clear and systematically developed descriptions and presentations, emphasizing significant points and relevant details, and the capacity to deliver detailed descriptions and presentations on various subjects within their field of interest, supporting ideas with subsidiary points and relevant examples These skills are currently evaluated at an average necessity level of 1.83.
Feasibility: 1.85 and Sustainable: 2.1 These 2 skills need to be adjusted with an average score of 62%
Table 4 2 Overall Spoken Production results at level B2
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking yes No
Can give clear, systematically developed descriptions and presentations, with appropriate highlighting of significant points, and relevant supporting detail
Can give clear, detailed descriptions and presentations on a wide range of subjects related to their field of interest, expanding and supporting ideas with subsidiary points and relevant examples
B2-level spoken skills are highly regarded by educators, as individuals at this level can clearly describe and express opinions on a wide range of general topics with minimal vocabulary challenges They demonstrate a solid grasp of grammar, making only occasional errors that may cause misunderstandings, yet they are typically able to self-correct most of their mistakes.
At the B1 level, the ability to fluently provide a straightforward description of various subjects within one's field of interest is deemed essential, receiving the highest necessity ranking of 1 and an average grade point of 2.1 The skill's possibility is rated at 1.88, while its sustainability stands at 2.12 There is a consensus for adjustment, with 57% of respondents in agreement.
Table 4 3 Overall Spoken Production level B1
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking yes No
Can reasonably fluently sustain a straightforward description of one of a variety of subjects within their field of interest, presenting it as a linear sequence of points
Teachers agree that individuals at level B1 possess sufficient language skills to discuss common topics related to work, family, personal interests, and travel However, they often experience hesitation and repetition in their speech Effective and precise use of vocabulary, along with familiar sentence structures, typically emerges in predictable situations.
At the A2 level, the skill of providing a simple description or presentation of people, living or working conditions, and daily routines was rated by teachers with an average score of 2.048, ranking it third The possibility of this skill was rated at 1.93, placing it first, while its suitability received a score of 1.9, ranking fourth Notably, 52% of teachers assessed this skill without any adjustments.
Table 4 4 Overall Spoken Production level A2
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking yes No
Can give a simple description or presentation of people, living or working conditions, daily routines likes/ dislikes, etc as a short series of simple phrases and sentences linked into a list
Most teachers concur that the A2 level in Vietnam allows learners to use simple sentences and memorized phrases for basic communication in everyday situations While A2 learners can construct simple sentence structures, they often require assistance with fundamental concepts Additionally, they can comprehend short sentences, even when these sentences are interrupted or contain errors.
At the A1 level, teachers assessed the skill of producing simple, isolated phrases about people and places as essential, with an average score of 1.92, ranking it fifth The possibility of achieving this skill received a score of 1.9, placing it second, while its suitability was rated at 1.67, ranking sixth Notably, 79% of teachers believe that no adjustments are necessary for this skill.
Table 4 5 Overall Spoken Production level A1
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking yes No A1
Can produce simple, mainly isolated phrases about people and places
Teachers recognize that A1 level learners possess a basic vocabulary and can use simple phrases to discuss personal information and specific situations They are familiar with fundamental grammar structures and simple sentence patterns, which many Vietnamese learners can easily master Overall, these skills are achievable for most learners in Vietnam.
At the Pre-A1 level, teachers assessed the skill of producing short phrases about personal information, such as name, address, family, and nationality, with average scores indicating a need for improvement The rankings were as follows: Necessity (2.095; 2), Possibility (1.74; 6), and Suitability (1.88; 5) Notably, 54% of teachers concurred that this skill requires adjustment.
Table 4 6 Overall Spoken Production level A1
Skills Necessity Feasibility Suitability Adjust
Can produce short phrases about themselves, giving basic personal information
At this level, teachers believe that learners primarily communicate using short, memorized sentences When attempting to express new ideas or complex vocabulary, learners often experience hesitation While they can ask and answer detailed questions about themselves, their communication remains reliant on repetitive sentence structures Despite their ability to interact, learners frequently make mistakes and require guidance to correct these errors, as they struggle to self-correct.
Table 4 7 Results of Sustained monologue: describing experience
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking Yes No
Can give clear, detailed descriptions on a wide range of subjects related to their field of interest Can describe the personal significance of events and experiences in detail
Can clearly express feelings about something experienced and give reasons to explain those feelings
Can give straightforward descriptions on a variety of familiar subjects within their field of interest
Can reasonably fluently relate a straightforward narrative or description as a sequence of points Can give detailed accounts of experiences, describing feelings and
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking Yes No reactions Can relate details of unpredictable occurrences, e.g an accident
Can relate the plot of a book or film and describe their reactions Can describe dreams, hopes and ambitions
Can describe events, real or imagined Can narrate a story
Individuals can effectively narrate stories or outline details through concise bullet points They are capable of depicting everyday elements of their surroundings, such as people, locations, work, or educational experiences Additionally, they can provide brief and straightforward descriptions of events and activities Furthermore, they can articulate plans and arrangements, discuss habits and routines, and reflect on past activities and personal experiences.
Can use simple descriptive language to make brief statements about and compare objects and
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking Yes No possessions
Can explain what they like or dislike about something
Can describe their family, living conditions, educational background, present or most recent job Can describe people, places and possessions in simple terms,
Can express what they are good at and not so good at (e.g sports, games, skills, subjects)
Can briefly describe what they plan to do at the weekend or during the holidays
Individuals can articulate their identities, occupations, and locations They are capable of describing basic elements of their daily lives through a series of straightforward sentences, utilizing simple vocabulary and fundamental phrases, especially when given the opportunity to prepare beforehand.
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking Yes No
Can describe themselves (e.g name, age, family), using simple words/signs and formulaic expressions, provided they can prepare in advance Can express how they are feeling using simple adjectives like
“tired”, accompanied by body language
The survey results indicated that teachers assessed skills related to Sustained Monologue: Describing Experience as a necessity, with an average score of 2.13, while feasibility and suitability were rated at 2.3 and 2.16, respectively For level B2, the skills of providing clear, detailed descriptions on various subjects and expressing personal significance of events were rated as necessary with an average score of 2.07, while feasibility and suitability were rated at 2.26 and 2.19 Notably, 51% of teachers agreed on the need to adapt these skills to the B2 level.
Table 4 8 Results of Sustained monologue: describing experience level B2
Skills Necessity Feasibility Suitability Adjust
Average Ranking Average Ranking Average Ranking Yes No
Can give clear, detailed descriptions on a wide range of subjects related to their field of interest Can describe the personal significance of events and experiences in detail
Can clearly express feelings about something experienced and give reasons to explain those feelings
Discussion
4.2.1 Teachers’ opinions on the need, feasibility, and suitability about the CEFR spoken production and spoken interaction descriptors to the Vietnamese context:
After conducting interviews and collecting Vietnamese teachers' opinions on the localization of the CEFR Spoken Production and Spoken
In the Vietnamese context, the CEFR framework demonstrates a notable level of suitability, necessity, and feasibility Teachers rate the skill of providing clear and detailed descriptions on various subjects as the most suitable, with an average score of 2.19 The necessity of skill specifications at level B1 ranks highest, averaging 2.1, while level A2 skills lead in feasibility with a score of 1.93 This indicates that the CEFR specifications are generally appropriate for Vietnam, particularly at medium and low levels However, improvements are necessary to enhance the framework further, as some differing opinions persist, particularly regarding the Overall Spoken Production section of Level B2, where the disagreement on the first skill adjustment is minimal, at around 10%.
The ability to provide clear and detailed descriptions on various subjects is crucial, with a high demand for improvement, as 69% of individuals express a desire to modify this skill The B1 level skill is deemed the most essential, scoring an average of 2.1, while the A2 level is viewed as reasonable and feasible, with an average score of 1.93 and 52% of respondents preferring no changes Interestingly, at the A1 level, only 21% of survey participants indicated a desire for change, highlighting varying perceptions of the CEFR Spoken levels.
In the Vietnamese context, Production and Spoken Interaction Descriptors have shown effective performance, with many teachers believing that the CEFR accurately evaluates candidates at basic levels However, higher CEFR levels, such as B1 and B2, face challenges like unfair grading and poorly timed assessments, necessitating updates to the CEFR Speaking Assessment Skills Framework Candidates often dread speaking exams due to issues like vocabulary recall, sentence structure errors, and direct translation from Vietnamese to English This common struggle among Vietnamese candidates lacks a clear solution A recent research paper gathered teachers' insights on the necessity and feasibility of a specification framework, highlighting common student errors and suggesting straightforward improvements for the localized foreign language certification exam These enhancements aim to bridge the gap between Vietnamese candidates and foreign language certifications, facilitating their integration into the global community.
The speaking test for the localized Vietnamese foreign language certificate closely resembles international tests like IELTS and TOEIC, featuring three parts with similar content and exam formats However, it is essential to adapt these assessments to suit the unique context of Vietnam, necessitating specific adjustments.
4.2.2 Teachers’ suggestions on the adjustment of the CEFR spoken production and spoken interaction descriptors to the Vietnamese context:
Many respondents feel that limiting the foreign language certification exam to a single discussion topic is unfair, as it may result in low scores due to incomplete understanding To address this, they suggest providing candidates with a selection of speaking topics to choose from during the test Additionally, candidates often struggle with expression errors, hesitation, and missing words, which can hinder their performance To improve the testing experience, some have proposed a system where points are deducted only for specific errors and extending the speaking test duration by 2 to 3 minutes.
The unfairness of speaking exam grading is highlighted by the experience of a candidate who received a low score due to an unfamiliar topic, specifically "Describe a live sport event you watched before." She expressed her struggle, stating, "I am a person who rarely watches sports, and of course I do not know an athlete or a team of athletes When it comes to this topic, I don't know where to start, what sport to talk about and how to talk, etc My head just goes blank and I don't know anything." In contrast, her friend, who regularly watches sports, answered the same question effectively and achieved a high score, demonstrating the impact of familiarity on performance in such assessments.
The evaluation of the candidate's speech appears to be unfair, as noted in the previous example Most educators agree that providing students with a list of themes to choose from for parts 2 and 3 of the speaking test would be a beneficial solution This approach would offer students greater flexibility, autonomy, and a more equitable assessment.