Investigating the impacts of more 3 students book user for cambridge ket on students learning outcomes at tri viet language center

3 STUDENT’S BOOK USED FOR CAMBRIDGE KET ON STUDENTS’ LEARNING OUTCOMES AT TRI VIET LANGUAGE CENTER A thesis submitted to the Faculty of English Linguistics & Literature By NGUYỄN HỒ

i

VIETNAM NATIONAL UNIVERSITY- HO CHI MINH CITY

UNIVERSITY OF SOCIAL SCIENCES AND HUMANITIES

FACULTY OF ENGLISH LINGUISTICS AND LITERATURE

INVESTIGATING THE IMPACTS OF MORE! 3 STUDENT’S BOOK USED FOR CAMBRIDGE KET

ON STUDENTS’ LEARNING OUTCOMES

AT TRI VIET LANGUAGE CENTER

A thesis submitted to the Faculty of English Linguistics & Literature

By

NGUYỄN HỒNG THẮM

Supervised by

TRAN THI THANH DIEU, Ph.D

NGUYEN DANG NGUYEN, Ph.D

HO CHI MINH CITY, AUGUST 2023

ii

ABSTRACT

This study investigated the impacts of MORE! 3 student’s book used for Cambridge KET on students’ learning outcomes at Tri Viet Language Center in Tay Ninh province, Viet Nam This research was conducted because of some initial potential gaps that the researcher discovered when teaching MORE! 3 to KET students at TVC To help point out factors that led to the impacts of the book on students’ learning and hopefully offer practical recommendations for better use of the book, the researcher employed a case study design and a mixed methods approach using questionnaires and observations administered to 05 KET teachers and 75 KET students at TVC for this student's book evaluation The data from the questionnaires were analyzed and grouped in themes using a basic sum through frequency and percentage of responses The results from participants’ opinions revealed that MORE! 3 student’s book was to some extent helpful to support users with necessary components to achieve the course’s communication objectives However, as the book did not contain enough necessary aspects for testing for learners to achieve KET Consequently, it caused more many negative impacts rather than positive ones on students’ learning outcomes (SLOs)

Key words: MORE! 3 student’s book, evaluation, impacts, students’ learning

outcomes (SLOs), KET

Besides, I am also grateful to Tri Viet Language Center in Tay Ninh province to enable me to conduct my study, research valuable findings and receive constructive feedback from colleagues Also I would like to sincerely say thank you

to teacher and student participants and people at Tri Viet Language Center, who directly or indirectly contributed the data to this study

Finally, I would like to express my heartfelt thanks to my beloved niece who quitted her own job to help me to take care of my little daughter so that I could have enough time focusing on this research study And special thanks also go to all members in my family, my good friends and my colleagues who gave me strong encouragement to finish the thesis Without their unfailing love, encouragement and support over time, I could not have done this research study

or other content from other people's work used within my thesis, whether published

or unpublished, is fully acknowledged in accordance with standard referencing practices

I certify that this is a genuine copy of my thesis, along with final revisions,

as accepted by my thesis supervisor and the administration staff of the English Faculty in USSH, and that this thesis has not been submitted to any other University

or Institution

v

RETENTION OF USE

Being a candidate for the MA degree, I agree to the University's standards for the preservation and use of theses deposited in the Library I agree that the original of my thesis deposited in the Library should be available for study and research purposes, subject to the Librarian's standard rules for the preservation, lending, or replication of theses

I agree to follow any general terms imposed by the University for the preservation, lending, or replication of theses, as well as any specific usage conditions in regard to this thesis titled

By signing this form, I agree that a copy of my thesis will be preserved in the University of Social Sciences and Humanities, and allow it to be published online in full text I confirm that the necessary copyright licenses for any original materials included in my thesis have been secured

Date: 11/8/2023

Candidate’s signature _

vi

TABLE OF CONTENT

Table of Contents……… ……… …… ………… vi

LIST OF ABBREVIATIONS ix

LIST OF TABLES x

LIST OF FIGURES ……… ……… ……….….xi

LIST OF APPENDIXES……… ……….…… xii

CHAPTER 1: INTRODUCTION ……….……… ……….1

1.1 Background to the study……… ……….…….….1

1.2 Statement of problem 5

1.3 Aims of the study 6

1.4 Significance of the study 7

1.5 Scope of the study 7

1.6 Organization of thesis 8

CHAPTER 2: LITERATURE REVIEW ….…… ………9

2.1 Definition, strength and weakness and design of textbooks…… ……… 9

2.1.1 Definition of the textbook ……… 9

2.1.2 Strength and weakness of textbooks ………10

2.1.3 Textbook Design ……….…….11

2.2 Textbook evaluation ……… 13

2.2.1 Definition of textbook evaluation ……… 13

2.2.2 Role of textbook evaluation in EFL classroom……… 13

2.2.3 Approaches and Criteria for Textbook Evaluation ……… …15

2.3 Students’ learning outcomes (SLOs) and Assessment of SLOs: ………17

2.3.1 Definition of student’s learning outcomes: ……… 17

2.3.1.1 Definition of student learning outcomes: ……… 17

2.3.1.2 Cambridge Key English Test (KET) ……… 18

2.3.2 Assessment of Cambridge Key English Test (KET) ……… 20

2.3.2.1 The structure of KET exam………20

2.3.2.2 Assessment of four skills in KET format ……….……22

2.3.2.2.1 Reading and Writing skills ……….……22

2.3.2.2.2 Listening skill ……… 23

vii

2.3.2.2.3 Oral Competency ……… … 24

2.4 Impacts of MORE! 3 on students’ KET exam achievement ……… 25

2.5 A review of previous studies ……… 26

2.5.1 Studies in Vietnam ……… 26

2.5.2 Studies in Western countries ………28

2.6 Theoretical framework of the study ……… 29

CHAPTER 3: METHODOLOGY ……….33

3.1 Research design ……… 33

3.2 Context of the study……… 36

3.2.1 Sampling of TVC and Participants ……… 36

3.2.1.1 Sampling of TVC ……….36

3.2.1.2 Participants ………38

3.2.1.2.1 Teacher Respondents ……….38

3.2.1.2.2 Student Respondents ……….40

3.3 Research instruments ……….41

3.3.1 Classroom observation ………41

3.3.1.1 Classroom observation checklist ……… 41

3.3.1.2 Classroom observation procedure ………43

3.3.1.3 Methods of analysis ……….43

3.3.2 Questionnaire for teachers and students ……….44

3.3.2.1 Questionnaire for teachers ……….… 45

3.3.2.2 Questionnaire for students ……….… 45

3.4 Data collection procedure ……….….47

3.4.1 Phase 1: The preparation stage ……… 47

3.4.2 Phase 2: The observation stage ……… ….47

3.4.3 Phase 3: The questionnaire stage ……… ….47

3.4.3.1 The questionnaire stage for teachers ……… 47

3.4.3.2 The questionnaire stage for students ……… 47

CHAPTER 4: RESULTS AND DISCUSSION 4.1 RESULTS ……….49

viii

4.1.1 Theoretical Evaluation of MORE! 3 student’s book ……….….…49

4.1.1.1 Aims and objectives of MORE! 3 ……….….49

4.1.1.2 Organization and structure of MORE! 3 ……… 49

4.1.1.3 Physical appearance ……….…51

4.1.1.4 Content of MORE! 3 and supplementary resources ………51

4.1.1.5 Comparison of MORE! 3 to KET format ………53

4.1.1.5.1 Ket format ……….….53

4.1.1.5.2 Comparison of MORE! 3 to KET format ……….…55

4.1.2 Findings from Classroom Observation and Questionnaire ……….…………58

4.1.2.1 Findings from Classroom Observation ……….58

4.1.2.1.1 A summary of general information about classroom observation ………58

4.1.2.1.2 Detailed evaluation of a typical unit in classroom observation: Unit 6 …60 4.1.2.1.3 A summary of physical conditions ……… 66

4.1.2.1.4 A summary of teaching and learning process ……… 66

4.1.2.1.5 First investigation results from classroom observation ……….67

4.1.2.1 Findings from Questionnaire ………73

4.1.2.1.1 Questionnaire Findings from teachers’ responses ……….73

4.1.2.1.2 Questionnaire Findings from students’ responses ……… 83

4.2 DISCUSSION ……… 93

CHAPTER 5: CONCLUSION……….………… …966

5.1 Summary of the findings……….… 96

5.2 Implications……… 98

5.3 Limitations and recommendations ………99

5.3.1 Limitations……… ……….……… ………… 99

5.3.2 Recommendations 99

5.3.2.1 Recommendations from teachers 99

5.3.2.2 Recommendations from students 100

ix

LIST OF ABBREVIATIONS

CEFR: Common European Framework of Reference for languages

EFL: English as a Foreign Language

EIL: English as an international language

ELT: English Language Teaching

ESOL: English for Speakers of Other Languages

ESL: English as a Second Language

FCE: First Certificate in English

IELTS: International English Language Testing System

KET(s): Key English Test(s)

MOET: Ministry of Education & Training

RQ(s): Research Question(s)

SL: Second-Language

SLO(s): Students’ Learning Outcome(s)

TOEFL: Test of English as a Foreign Language

TVC: Tri Viet Language Center

x

LIST OF TABLES

Table 2.1: 7 steps of material writing stage (Jolly and Bolitho, 2011)

Table 2.2: Six principles of materials by Nunan (1988)

Table 2.3: Criteria for evaluating KET writing skill

Table 2.4: Criteria for evaluating KET Speaking skill

Table 3.1: Correspondence of RQ, analysis procedure & data source Table 3.2: Summary of background information of TVC

Table 3.3: General information about the teacher participants

Table 3.4: General information about the student participants

Table 3.5: Classroom observation checklist

Table 4.1: A summary of the grammar points in the book

Table 4.2 Activities and tasks in Unit 1 in MORE! 3

Table 4.3: Comparison between the test format and MORE! 3

Table 4.4 Background details of classroom for observation

Table 4.5 Information about the lessons observed

Table 4.6: A summary of teachers’ responses on Question 1

Table 4.7: A summary of teachers’ responses on Question 2

Table 4.8: A summary of teachers’ responses on Question 3

Table 4.9: A summary of teachers’ responses on Question 4

Table 4.10: A summary of teachers’ responses on Question 5

Table 4.11: A summary of teachers’ responses on Question 6

Table 4.12: A summary of teachers’ responses on Question 7

Table 4.13: A summary of students’ responses on Question 1

Table 4.14: A summary of students’ responses on Question 2

Table 4.15: A summary of students’ responses on Question 3

Table 4.16: A summary of students’ responses on Question 4

Table 4.17: A summary of students’ responses on Question 5

Table 4.18: A summary of students’ responses on Question 6

Table 4.19: A summary of students’ responses on Question 7

Table 5.1: Summary of main findings

12 12

xi

LIST OF FIGURES

Figure 1.1: Evaluation Standards in English language

proficiency

Figure 2.1: “Can-Do summary” Figure 2.2: Common European Framework for References (CEFR) Figure 2.3: A2 Key for schools exam format Figure 2.4: Materials analyzing procedures Figure 2.5: Littlejohn’s (2008 & 2011) framework Figure 2.6: The Tomlinson’s (2003 & 2013) framework Figure 2.7: Conceptual Framework of the study ……… 2

……… 19

……….20

……… 21

……… 30

……… 30

……… 31

……… 32

Figure 3.1 7: steps to conduct mixed methods research (Cresswell, 2012, p.555) ……… 35

Figure 4.1: KET format 54 … ……… 54

xii

LIST OF APPENDIXES

Appendix 1: Respondent Information Sheet for Teachers

Appendix 2: Participant Information Sheet For Students

Appendix 3: Questionnaire Instruction

Appendix 4: Questionnaire to Teachers

Appendix 5: Questionnaire to Students

Appendix 6: Classroom Observation Sheet

Appendix 7: CEFR Level A2 (Key English Test: KET)

Appendix 8: MORE! 3 student’s book: The book cover

Appendix 9: The map of the book/ The content pages

Appendix 10: Wordlist of MORE! 3 Student’s Book

Appendix 11: Units 1, 6 & 10 observed in classrooms

Appendix 11A: Unit 1: I’ve lost my wallet!

Appendix 11B: Unit 6: Teens around the world

Appendix 11C: Unit 10: It must be good for you

Appendix 12: Cambridge KEY English Test Sample – KET

…… 109

…… 110 …… 111

1

CHAPTER 1: INTRODUCTION

This chapter consists of background to the study, statement of the problem, aims of the study, research questions, significance of the study, scope of the study, and organization of the thesis

1.1 Background to the study

Within the context of Decision 1400-QD-TTg by the Prime Minister approving the Project of foreign language teaching and learning in the national system for the 2008-2020 periods, the Ministry of Education and Training had issued the Common Framework of Levels of Foreign Language Proficiency, based on the Common European Framework of Reference (CEFR) for Languages: Learning, Teaching, and Assessment CEFR was developed by the Council of Europe (MoET, 2008c) It comes with a general framework which indicates what language learners need to learn to be able to use a foreign language effectively in practice This Framework includes 6 levels equivalent to those of CEFR and the national foreign language education curriculum will be designed accordingly The new curriculum defines that students at the completion of Primary education should reach Level 1 (A1 of CEFR), at the completion of Lower secondary education should reach Level 2 (A2 CEFR) and General school leavers should reach Level 3 (B1 CEFR) According to this CEFR framework, students who finish grade 6 are expected to achieve level A2+ For tertiary education, graduates from vocational colleges should reach level A2 In CEFR framework, candidates who pass Key English Test (KET) mean they pass Level of A2 and achieve KET certificate This certificate has an eternal value in the world

This standard was used as the basis foundation for constructing the curriculum, textbook development, and standards for its level To meet the state decree, Level of A2 (KET) was measured in the following figure:

2

Figure 1.1: Evaluation Standards in English language proficiency

in comparison to the international standards (Source: MoET, 2008c)

Teaching material used for this level has become popular and diverse, too

In different language training centers, there were different materials, even for the same level Also it is a key for them to succeed in learning outcomes (Asfiah et al., 2013; Pramana and Dewi, 2014; Amir et al., 2015; Daud and Rahmadana, 2015; Nisa’ et al., 2015; Panjaitan et al., 2019) The textbooks have become the key source in learning process (Mahmood, 2011; Thang et al., 2013; Anisah and Azzah, 2016) They can be the best when they were well prepared (Allenhyania

et al., 2017) The textbooks should focus on students’ needs of learning and should be a guide for them to learn efficiently (Allehyani et al., 2017) However,

in many private language centers, the textbooks were chosen freely by officials

as they were widely used or introduced by Oxford University Press Moreover, they were encouraged to be used because their publishers were well-known or they were taught by many famous foreign language centers Then, the national curriculum program was textbook-based The instruction of program delivery

3

was based on each section of the textbook content Each DoET could adjust to the framework to suit some conditions in the local (Dugan, 2001) In fact, there were no significant standards for choosing textbooks There was also no serious investigation of the textbooks’ impacts on learners’ outcomes

Sana Baig et all., 2021 conducted a study to analyze an English textbook of Grade 8 in Pakistan in the light of student learning outcomes of four language competencies: reading and thinking skills, writing skills, oral communication skills, and formal and lexical aspects of language of the latest English curriculum It was a mixed-methods research involving both content analysis and survey questionnaire obtaining opinions of learners regarding the competency The findings indicated that the textbook only focused on reading skill The others, such as thinking skills, writing skills, oral skills, and formal and lexical aspects of language in the textbook were not much paid attention Finally, the book was recommended by the researchers to update the book in congruence with students’ learning outcomes The second study about a textbook evaluation conducted by Haris (2015) to investigate the effectiveness of Intermediate English compulsory syllabus in an Iqbal Open University The findings showed a fulfillment of the basic English language skills and other sections like grammar, communication, local culture and context, and sociocultural background in the books However, the speaking skills were not executed

Denham (1992) states that English has become the most popular foreign language since 1989 Besides, David Nunan (2003) also had a study about the important role of English In a study, the impact of English as a global language

on educational policies and practices in the Asia-Pacific Region was emphasized There was the rapid growth and expansion of English in many different aspects

in Vietnam Particularly, in education, English is now a compulsory subject in the curriculum for general education (from grades 6 to 12) In higher education, English is one of five obligatory foreign languages (English, French, Russia, German and Chinese) taught on all majors with the highest enrolments (MoET, 2008c) To evaluate foreign language ability among Vietnamese citizens, the

4

MoET issued an evaluation system, ranked from level A to level C in 1994 This system awards the National Certificate to those who meet the requirements at a certain level

According to Government Decision (2008), the reform also emphasizes the establishment of national evaluation standards in foreign language proficiency, based on international tests such as TOFEL, IELTS, PET, etc The national standards are used as the basic foundation for the construction of the school curriculum, developing textbooks and standards for each level of education to ensure continuity of learning at all levels of education To meet the requirements

of the government decree, the authorities have established six levels of foreign language (FL) proficiency to measure language ability of the Vietnamese population (see Figure 1 for more details) Evaluation standards in English language proficiency (Source: MoET, 2008c) The reform sets targets for FL proficiency at each level of education, which must be based on the Vietnamese Evaluation Standard System Therefore, students should be at level 1 after finishing primary school; at level 2 by the end of junior high school; and at level

3 by the end of senior high school The levels that graduating foreign language major university students should reach are 4 and 5 This is the first time the government has established clear English proficiency targets, which are considered to be practical and possible for each level of education

The CEFR “provides a common basis for the elaboration of language syllabuses, curriculum guidelines, examinations, textbooks, etc across Europe” (Council of Europe, 2001, p.1) It describes in a comprehensive way what language learners have to learn to do in order to use a language for communication and what knowledge and skills they have to develop to be able to act effectively (Yamat, Umar & Mahmood, 2014).The CEFR is derived from the international global scales where it distinguishes five communication skills, which are listening, reading, spoken interaction, spoken production, and writing

It also gives suggestive activities which are deemed appropriate for that particular skill Language proficiency is measured in relation to the five skills on

a scale beginning with A1, and progressing through bands A2, B1, B2 and C1 to

5

C2 Proficiency in each skill is defined at each level by a series of “can do” statements An important reason to set up and follow international standards is that they can be used to compare different language education systems, and to assess a single country’s language education system relative to international best practice This process is referred to as “benchmarking” (English Language Standards and Quality Council, 2015, p.64)

In short, choosing a textbook should depend on learners’ needs When the content in the textbook was not suitable for students to learn and practice English, they may affect learners’ results in a negative way The teaching material is considered as a benchmark for learners to achieve an effective and efficient learning process (Purwanto and Rizki, 2015)

In the context of Tay Ninh province, among textbooks used for KET, MORE! 3 was popular and widely used in some institutes, especially in Tri Viet Language Center (TVC) To achieve the course objectives under this book, students in KET classes needed to pass the two tests: a midterm test and a final test designed similarly with an actual KET of Cambridge University (see Appendix 12: Cambridge KEY English Test Sample) After completing this final KET level, the students would register and take part in an actual Cambridge KET organized by the representative of Cambridge University of UK in Vietnam However, MORE! 3 was mainly designed for communication achievement only The components and revision lessons only supported learners to develop communicative skills (see Appendix 9: The map of MORE! 3) It seemed there

In other words, the achievement of communicative competence was focused, and the teachers taught what were in the book and were listed in the course outline Consequently, there was a big gap between teaching this book for developing students’ communicative achievement or for preparing them to pass KET There were a number of studies working on textbook evaluation with different purposes However, there was not much literature on exploring impacts

of the books on students’ learning outcomes, in the case the textbooks were used

to prepare for an international test

1.3 Aims of the study

This study investigated MORE! 3, a student’s book from a series of three student’s books: MORE! 1, MORE! 2 and MORE! 3 for KET students at TVC Particularly, MORE! 1 was used for levels of KET 1 and KET 2 MORE! 2 was used for KET 3 and KET 4 Especially, MORE! 3 was used for final KET level MORE! 3 had been used for more than 9 years at TVC until now Investigating MORE! 3 was important to help point out factors that led to its impacts on students’ learning and hopefully could offer practical recommendations for better use of this student’s book

Basing on the aims of the study, the following research questions would guild the investigation:

1 To what extent is MORE! 3 student’s book suitable for Key English Test teaching and learning at TVC?

1.4 Significance of the study

There were many areas where this study possibly made significant contribution to the current ELT Firstly, its empirical evidence would provide better understanding of the impacts of MORE! 3 students’ book used for KET on students’ learning outcomes Due to the fact that final KET students learnt MORE! 3 at TVC for many years, finding out positive or negative impacts of MORE! 3 on students’ learning outcomes could help to identify appropriation of the MORE! 3 student’s book Thanks to the findings, training managers could adjust their KET learning programs for their better training purposes Secondly, the current study was intended to suggest ways of solving the problem as well as recommendations From the students’ and teachers’ ideas about the problem, the researcher would experiment KET practice-supported MORE! 3 teaching as a supplementary technique to improve students’ learning outcomes This contribution would be of practical value to teachers and students Finally, this study would serve as a base for future studies or references on other tests in contemporary use such as Cambridge KET In Vietnam and the world scale, there could not have related literature conclusive regarding the impacts of MORE! 3 used for Cambridge KET on students’ learning outcomes And there were not any studies inside and outside Vietnam conducted about this problem; therefore, the present study was intended to contribute to the literature in this respect The researcher hoped that the study would make a small but valuable contribution for ELT in a real situation

1.5 Scope of the study

With an effort to form the basis for ELT improvement, the study was primarily defined to explore impacts of MORE! 3 student’s book used for Cambridge KET on students’ learning outcomes This study only looked at MORE! 3 because it was currently used to prepare students for taking part in an

8

actual KET Cambridge examination Additionally, its findings would provide suggestions to the academic administrators of TVC context only in order to promote learner-centerness in learning English for their tests As a result, this study, however, would not be carried out to confirm or reject the Cambridge KET utilization and English courses in other general situations

1.6 Organization of thesis

The present thesis has the following chapters:

Chapter one: It is the introduction chapter which introduces the study’s

background, aims of the study, statement of the problem, research questions, significance of the study, scope of the study, and organization of thesis chapters

Chapter two: It is about literature review chapter which represents the

definitions of items relating to the study, review the studies related to the present study, indicates the gap that this study aims to fill

Chapter three: It is methodology chapter which focuses on the

methodology including research design, context of the study, participants, research instruments, data collection procedure and data analysis scheme

Chapter four: It is results and discussion chapter which include the results

of the study and the findings’ discussion

Chapter five: It is conclusion chapter which summarizes the research

findings, discusses the results, points out some implications and the study’s limitations, and offers recommendations

CHAPTER 2: LITERATURE REVIEW

This chapter describes the literature on textbooks and textbook evaluation including a review of textbook; textbook evaluation; textbook evaluation approaches and criteria, previous relating studies, and a theoretical framework of the study

2.1 Definition, strength and weakness and design of textbooks

2.1.1 Definition of textbooks

In teaching a foreign language, teaching materials is indispensable component Many researchers have defined this concept in various ways, but they have shared the same opinions that materials can be anything used for facilitating teaching and learning of language (Littlejoin 1998, Tomlinson 1998, and McGrath 2002) According to Littlejoin (1998) and Tomlinson (1998), materials could be student’s book, workbook, teacher’s book, videos, CDs, DVDs, lesson plans, internet sources, activities and others Also, McGrath (2002) suggested that they could be real objects, drawings, photographs, worksheets, computer software and recordings Besides, Hutchinson and Torres (1994) stated that a textbook was a teaching guide for teachers to instruct learners, a tool for learners to review knowledge and also considered a record to evaluate the knowledge More importantly, Cunningsworth’s (1975) generally defined that a textbook was an effective resource for self-directed learning, an effective source of presentation of materials, a source of ideas and activities, a reference source for students, a syllabus where they reflected predetermined language objectives, and support for less experienced teachers who had yet to gain confidence (Cunningsworth, 1975 as cited in Awasthi, 2006: 2) Awasthi (2006) said that a textbook was a teaching and learning material for teachers and learners to use in the teaching and learning process In addition, Ur (1996) and Tomlinson (2011), in accordance with the definition in the dictionary, added textbooks were considered the core materials for a language course A textbook did not often include only language components like vocabulary, grammar points, pronunciation but also functions and skills of the four skills

And Tomlinson (2011) defined the textbook as one of the materials used to help teachers teach learners As one of teaching materials, a student’s book was

seen as a textbook that students and teachers basically used as a heart of the course (Collinsdictionary.com, 2018) Similarly, a student’s book was used as a book for students to take part in a specific course of study (Dictionary.cambridge.org, 2018)

In brief, different writers presented their own definitions of textbooks, based on how and where they were used However, there was one thing they had in common: The textbooks were a source for both the teachers to transfer the knowledge to the learners and the learners to master the knowledge and used the knowledge for study, work or life purposes This definition of the textbooks seemed to be appropriate for this explosive study

2.1.2 Strength and weakness of textbooks

Cunningsworth (1975); Hutchinson and Torres (1994), and Tomlinson (2003) agreed that textbooks were widely used by teachers and had many typical advantages Firstly, Donoghue (1992) surveyed 76 experienced and inexperienced teachers and showed that the majority of teachers used teachers’ guides at least once

or twice a week This concluded that textbooks, with their supplementary components were an important source According to Crawford (2002); Littlejohn (1992); McGrath (2002) and Tomlinson (2003), textbooks and their supplementary components, such as teachers’ guides, workbooks, tests, and other audio-visual devices could provide useful and necessary classroom work for general language courses Textbooks with their teacher’s manuals give teachers instructions on how

to carry out the lessons That’s why textbooks help to “re-skill”, rather than skill” teachers Additionally, they also help teachers save time on lesson preparation and materials production, as some textbooks provide photocopied activity sheets and pictures to introduce new language material to learners (Tomlinson, 2003) Bell and Gower (1998 & 2011) also stated that textbooks helped teachers have more time to think about creating interesting and effective classroom activities to introduce the given content of knowledge to learners According McGrath, (2002), textbooks were sometimes considered to offer a clear map for the teacher and learner to follow They indicated to both the teachers and learners where they were, what they had done and what needed to be done to fulfill a specific course aim They also helped learners review and practice what they learned

“de-Besides strong points of the textbooks, many other scholars also pointed out some weaknesses of textbooks According to Block (1991); Maley (2011); and Tomlinson (2003), they argued that textbooks did not meet all teachers’ and learners’ needs as they could not be relevant to all teaching and learning circumstances or to the diversity of learners’ backgrounds Moreover, Block (1991); Maley (2011); and Tomlinson (2003), tended to follow a middle course between these different points of view, saying that, if the textbook was adapted and used flexibly in the classroom, it could provide relevant knowledge to meet the demands

of specific purposes Wade and Moje (2000) supported Tomlinson’s view, adding that flexibility in textbook use could help the teachers deliver knowledge suitable to learners’ needs (Awasthi 2006) However, in contexts such as Vietnam where English is not used outside the classroom, classroom instruction should rely on different kinds of materials, including textbooks, reference books, articles, and journals to take advantages of both authentic and non-authentic texts Moreover, a lecturer or authors who wrote a textbook could limit the diverse user background, teaching and learning culture and context

2.1.3 Textbook Design

According to Tomlinson (2003: 107), ‘many of the materials developers describe that materials writing processes which are ad hoc and spontaneous and which rely on an intuitive feel for activities which are likely to work Besides, Tomlinson and Masuhara (2010) do mention a number of issues which need to be carefully considered before starting to write a textbook including needs analysis, language learning principles, target students and teachers These can be grouped into two main categories The first one indicates the steps people should take when they are writing materials, and the second one focuses on the specific principles for writing the content of textbooks

In this part, seven steps of the material writing stage of Jolly and Bolitho (1998 & 2011) and principals of Nunan (1988) for developing materials/ textbooks

are presented in brief in the following Table 1: Seven steps of the material writing stage

1 Identification of need for materials

2 Exploration of need

3 Contextual realization of materials

4 Pedagogical realization of materials

5 Production of materials

6 Student use of materials

7 Evaluation of materials against agreed objectives

Table 2.1: Seven steps of the material writing stage (Jolly and B: olitho, 2011: 113)

The textbook is appropriate to the context in which it is to be used Unclear instructions, exercises and activities are likely to reduce learners’ comprehension, participation and confidence in classroom activities They may also demotivate them Inappropriate topics should not be included in a textbook taught because learners do not feel familiar with It is hard for them to understand and they hardly ever use that knowledge to describe their surroundings According to Dat (2008), common topics, such as social issues, economic achievements, environment, etc… should be included in the materials However, controversial issues related to religion or superstition should not be included then Additionally, the physical appearance of the textbooks is also important as it makes the first impression on users before they consider other characteristics In Bell and Gower’s framework for materials design, they focus on the content of materials The physical appearance and the preparatory steps are not mentioned Having a similar ideas with Bell and Gower in materials development, Nunan (1988) proposed six principles of materials design in the following table 2:

1 Materials should be clearly linked to the curriculum they serve

2 Materials should be authentic in terms of text and task

3 Materials should stimulate interaction

4 Materials should allow learners to focus on formal aspects of the language

5 Materials should encourage learners to develop learning skills

6 Materials should encourage learners to apply their developing skills to the

world beyond the classroom

Table 2.2: Six principles of materials by Nunan (1988)

2.2 Textbook evaluation

2.2.1 Definition of textbook evaluation

There has been many different definitions of textbook evaluation by diverse writers There are some definitions of textbook evaluation as follows

According to Tomlinson (2011), textbook evaluation is an attempt to measure the potential value of textbooks It involves making judgments about the impacts of textbooks on its users These impacts may be measured through such features as the: credibility, validity, flexibility, etc of the textbook Rea-Dickins and Germanie (1994: 28) define evaluation ‘as the means by which we can gain a better understanding of what’s effective, what’s less effective and what appears to be no use at all’ Weir and Roberts (1994) specify that textbook evaluation involves a systematic analysis of all relevant information necessary to improve the textbook Nevo (1977: 127)

According to Cunningsworth (1975), “Materials evaluation is a complex matter” (p.35) because there are various factors which affect or determine the success or failure of the textbooks However, it specifically helps teachers and students to get a useful and understandable vision of the materials that they are going to use Besides, Tomlinson (2003) also states that “materials evaluation is a procedure that involves measuring the value (or potential value) of a set of learning materials” (p 50) After choosing the textbook for a particular language learning program, it should be evaluated then In addition, there are three types of evaluation: pre-use evaluation which mainly focuses on exploring the potential worth of the textbooks, while-use evaluation which especially allows a special attention to the learners while using the instructional materials in the classroom setting, and after-use evaluation which describes and selects what will happen after using the textbooks

2.2.2 Role of textbook evaluation in EFL classroom

Evaluating a textbook is important for many different reasons Cunningsworth (1975) lists two reasons that acceptance of textbooks and awareness of the specific advantages and disadvantages of existing textbooks are very essential for textbook evaluation Moreover, Littlejohn (2011, p 183) asserts that "material analysis and evaluation allows us to look inside materials and have more control over their

design and use." “Textbook evaluation is very important because an inappropriate choice can waste time and money and this can have a de-motivating effect on both students and other teachers.” (McDonough & Shaw, 2003: 60)

According to Cunningsworth (1975) and Ellis (Mukundan, 2007; Sheldon,

1988, 1997), textbook evaluation helped teachers move beyond dramatic assessments and continue to provide teachers with accurate, useful, systematic, and contextual insight into the overall nature of the material “We ensure that careful selection is made and that materials are selected closely reflect the learners' needs, methods and values of the curriculum” (Cunningsworth, 1975: 7) Ellis (1997) proposed that if teachers agreed with the value of textbooks in ELT, they should definitely follow the stipulation that they met the standards of quality, usefulness, and situational relevance

Sheldon (1988) also addresses the important role of textbook evaluation, arguing that educational administrators and teachers can make decisions about which textbooks to use by using the results They are able to understand the textbook market and clearly distinguish between a wide variety of textbooks Choosing the right textbook can be very difficult as there are many textbooks for ELT materials on the market When choosing textbooks, one must be able to make informed and idealistic choices (Cunningsworth, 1975) In that respect, the quality

of a textbook can be so important that it can determine the success or failure of an ELT course (Mukundan, 2007) However, buying textbooks without in-depth reviews is common Textbook selection is sometimes influenced by factors other than their inherent educational value, such as the reputation of the author or publisher or the publisher's marketing prowess McGrath, 2002) Attractive color textbooks are preferred, and teachers can use widely used bestsellers elsewhere without question

Another important function of textbook review is that it can aid teachers' professional development by providing them with information to test their own assumptions about how the curriculum works (Hutchinson, 1987) In addition, it can inform prospective teachers about key features to look for in books It provides them with detailed information about the assessment criteria so that they can critique more published textbooks (Litz, 2005)

Overall, textbook evaluation plays an important role in learning and teaching

in any ELT program as it helps to select the most appropriate textbooks for use in a

particular teaching and learning context

2.2.3 Approaches and Criteria for Textbook Evaluation

There are different writers and researchers who had different typical types of evaluation It depends on the specific circumstances of teaching and learning contexts, so the evaluation criteria may vary

Firstly, some approaches of Cunningsworth’s (1975), Skierso’s (1991) Ur’s (1996) and Byrd’s (2001) approaches for textbook evaluation are similar These approaches similarly provide some general evaluation criteria, such as layout, prints, periodic review lesson and tests, supporting materials, illustrations, etc., alongside some very specific ones, such as the objective; the variety of interesting topics and tasks; clear instructions; sequencing and grading; vocabulary and grammar, etc For more explanation, Cunningsworth’s (1975) approach offers 8 criteria of textbook evaluation: Aims and approaches; Design and organization; Language content; Skills; Topic; Methodology; Teacher’s guide; and Practical considerations Moreover, Byrd’s (2001) approach shares similar features to that of Cunningsworth (1975) She provides the seven following categories for textbook evaluation: Physical appearance, topics, vocabulary, linguistic structure, exercises, illustrations and teacher’s manual Addition, Skierso (1991) is similar to Cunningsworth (1975) in suggesting six criteria with clear explanations: Bibliographical data, aims and goals, subject matter, vocabulary and structures, exercises and activities, and layout and physical makeup Briefly, they all give detailed criteria for evaluation, but they do not give a clear framework for the whole process of evaluation

Besides, Tomlinson (2003) suggests a three-stage process of evaluation: use, Whilst-use, and Post-use Pre-use evaluation mainly focuses on exploring the potential worth of textbooks Artwork, illustrations, appearance, content pages, etc

Pre-in the textbook is analyzed to gaPre-in an impression of its potential value Then, criteria about instructions, layout, texts, tasks, and others are evaluated in whilst-use stage This second stage especially allows a special attention to the learners while using the instructional materials in the classroom setting Finally, impact of the

textbook on teachers and students is discovered The after-use evaluation describes and selects what will happen after using the textbooks Although Tomlinson provides useful criteria for evaluating a textbook, he does not specify what to look for to measure each criterion However, to contribute to a more comprehensive textbook evaluation for a particular context, his approach should be supplemented

by more specific criteria and elements from other authors

Additionally, Littlejohn’s (2008) analyzes a framework including 3 levels Level 1 generates superficial features Lever 2 and 3 are about the design, including teaching methodology, learners’ roles and materials’ roles More importantly, Littlejohn (1998 & 2011) also employs some more evaluation criteria, such as appearance (quality of paper, binding, etc.), practical issues (pricing), organization, etc to evaluate a textbook However, he states somewhat different approaches from Tomlinson’s (2003) Although, Tomlinson (2003) points out that the levels of textbook evaluation starts from more objectives criteria to less objective ones, Littlejohn (1998 &2011) suggests a three-level evaluation checklist: What is there?, What is required of users? , and What is implied? Then he goes slightly deeper into

an analysis of what is assumed to be the most important aspect of textbooks

The next approach to textbook evaluation suggested by McGrath (2002) is similar to Littlejohn’s (1998; 2011) Particularly, he also suggests three types of textbook evaluation methods Firstly, an impressionistic method refers to the impression of the textbook, includes general criteria of evaluating a textbook through the first glance Secondly, an in-depth method deals with textbook evaluation in details, refers to a large evaluation scale, and deals with learning and valuable assumptions that the materials are based on Finally, a checklist method refers to items, criteria and activities lists as a discipline to decide the value of evaluation According to McGrath (2002), this is an in-depth evaluation It can address the aims and content of the textbook, requirement, the functions, the interests and the needs of students, and their approaches to learning a language and the approach in teaching and learning in the teacher’s classroom

Typically, McDonough and Shaw’s (1998 & 2003) approach to textbook evaluation has three stages: External evaluation similar to McGrath’s (2002) ‘first-glance evaluation, Internal evaluation resembling McGrath’s (2002) ‘in-depth

evaluation’ and Tomlinson’s (2003) ‘whilst-use evaluation’, and Overall evaluation accounting factors of usability, generalizability, adaptability and flexibility of a textbook

Generally, we have seen many approaches and criteria to evaluate a textbook

of different writers In fact, there are a variety of evaluation approaches and criteria discussed in the literature Each of the mentioned approaches to textbook evaluation has strengths and weaknesses; therefore, evaluators should be aware of their purpose for evaluation because there are many purposes for evaluating textbooks (selection, balancing teaching, supplement, etc.) Also, textbook evaluation can be conducted either before the textbook is used, while it is being used or after it has been used (Tomlinson, 2003) And it's important for evaluators to combine different approaches in order to take full advantages of all approaches and criteria and avoid their disadvantages

2.3 Students’ learning outcomes (SLOs) and Assessment of SLOs:

2.3.1 Student’s learning outcomes:

2.3.1.1 Definition of student’s learning outcomes:

According to W.Winkel (1989), students’ learning outcomes are the success achieved by students, namely student achievement in school in the form of numbers While the definition of learning outcomes by Winamo Surakhmad (1980)

is the result of repetition, examination or test The purpose of the repetition is to obtain an index in determining student success

In Students’ learning outcomes, students’ learning products are focused more than the learning process This means student performance is more appreciated than what they intend to do during instruction For example, students who are able to apply what they have learned and experienced to communicate with other people in English will meet the target of a communication course rather than their course completion process

Students’ learning outcomes specify what students will know be able to do or

be able to demonstrate when their Course or Program has been completed Students’ learning outcomes specify an action that must be observable, measurable and able to

be demonstrated Grades are considered as the students’ learning objectives students’ learning outcomes are essential and enduring knowledge, abilities (skills)

and attitudes (values, dispositions) that constitutes the integrated learning needed by

a graduate of a course or a program” (by Mark Battersby and the Learning Outcomes Network, Center for Curriculum, Transfer and Technology, February, 1999)

From the above definitions, it can be concluded that the results of learning is the result of learning achieved by students in the process of teaching and learning activities are given in the form of numbers Students’ learning outcomes or students’ leaning results are a tangible result achieved by the students in the effort

to master the physical and spiritual skills in school which is manifested in the form

of report cards in each semester In this current study, the learning outcome of this program is level A2, according to CEFR framework If the students receive the score from 120 in the Basic level of CEFR which shows KET (A2) certificate achievement, it means they achieve successful learning outcomes

2.3.1.2 Cambridge Key English Test (KET)

A2 KET is an acronym for Key English Test, also known as Cambridge English This is an English test organized by Cambridge Assessment English to assess candidates' ability to communicate through communicating basic everyday situations KET is equivalent to A2 level on the Common European Framework of Reference for Languages (CEFR) Achieving KET exam helps students to achieve Level A2 of CEFR for Languages In some private and government schools, A2 is a compulsory certificate of English for their college students

At this A2 level, learners should be able to cope linguistically with daily situations and would be able to use English in their own with native and non-native speakers of English for general contact KET candidates could read simple texts including street signs and public notices, product packaging, forms, posters, brochures, city guides and instructions on how to make a phone call They should also deal with personal messages written as letters or postcards, and gain some information from informative texts taken from newspapers and magazines Where listening skills were concerned, a candidate needed to understand the basic facts given in announcements such as railway stations and airports, traffic information given on the radio, and public announcements made at sporting events or pop

concerts Ssee the “Can-Do summary” in Figure 3 adopted from Handbook for

teachers of Cambridge University on http://www.iltea.org/cambridge/KET/ket_handbook2007.pdf) below:

Figure 2.1 “Can-Do summary”

KET certificate (A2 Key) is equivalent to IELTS 4.0 Students are exempted from the national high school English exam The UK Immigration Department uses the KET certificate to prove English proficiency for international students applying for a visa to the UK for more than 6 months This is also an exam to practice psychology in the exam room for young candidates before facing other big and small exams related to English

At TVC in Tay Ninh province, learners whose ages were ranged from 9 to 12 could join A2 to pass A2 certificate to measure their English competency

Especially, students who completed Flyers (A1) or those could pass an entrance general KET replacement test of TVC would participate in KET program According to CEFR framework and the curriculum at TVC, the students who finished KET course could register and pass an actual KET If they could achieve KET marks from 120 points at Basic program, they received A2 certificate by Cambridge University See the Common European Framework for References

framework on Figure 2.2 (on tests/) below.

Figure 2.2 Common European Framework for References (CEFR)

To measure and evaluate students’ learning outcomes, it was usually done through the test Based on the objectives and the scope, the achievement test could

be classified into typical kinds of tests In KET assessment, the students’ language competence was measured by evaluating how their four language skill competence was through doing KET exam

2.3.2 Assessment of Cambridge Key English Test (KET)

2.3.2.1 The structure of KET exam

Based on A2 Key for schools exam format, the structure of the KET exam consists of 3 parts: Reading and Writing, Listening and Speaking The sample KET exam paper (see Appendix 12 on p.139) is the officially updated version of Cambridge from January Reading and Writing have got 7 parts/32 questions (60 minutes, accounting for 50% of the total score) and assess the learners’ ability to read and understand signs, advertising information and magazines Moreover, Listening receives 5 parts including 25 questions (25% of total marks – 30 minutes

of test) and assesse the ability to listen to information spoken at a slow speed such

as spoken materials Finally, Speaking has 2 parts (25% of the total score – 10 minutes) and measure how candidates demonstrate their ability to communicate in English by asking and answering simple questions One or two other candidates and examiners will sit face to face in the speaking test This makes the test more realistic and more reliable See A2 Key for schools exam format in Figure 2.3

(https://www.cambridgeenglish.org/exams-and-tests/) below:

Figure 2.3 A2 Key for schools exam format

Each of Language Centers has their own marking scales and rubrics to measure candidates’ competence of language skills and confidence As usual, there were from four to five rubrics to measure each skill of English For example, four rubrics for common Cambridge Writing test were Task Fulfillment, Organization, Vocabulary and Grammar These categories were created basing on Cambridge University’s rules about evaluation rubrics of each program However, each institute could adopt or adapt them then

2.3.2.2 Assessment of four skills in KET format

2.3.2.2.1 Reading and Writing skills

The A2 Key Reading and Writing have seven parts and different types of texts and questions Parts 1 to 5 are about reading and Parts 6–7 are mainly about writing

In Reading part, there is Multiple Choice, multiple matching, multiple choice-cloze, and open cloze Each correct answer will get a score listed in Cambridge A2 Key Reading is a complicated process that comprises of equally complicated skills that require dynamic participation from the reader’s part (Mancilla -Martinez & Lesaux, 2010) In other words, reading can be considered as a thinking process that requires the reader to comprehend, understand, and even employ different skills to gain the desired information from a text, such as concluding, predicting, questioning, and drawing conclusions (Zainal & Husin, 2011; Küçükoğlu, 2013) Hans & Hans (2015) define reading comprehension as the capacity at which

a person can comprehend a text This type of comprehension arises from the written words and how they bring about knowledge existing outside the text In other words, comprehension in this sense is a complex yet creative process that depends

on the four skills of language (phonology, pragmatics, semantics, and syntax) In addition, Bulut (2017) states that reading comprehension refers to the act of thinking and forming meanings before, during, and after reading He also states that reading comprehension is one of the main language skills that entail interpretations and identifying the details that are found in written materials, whether they are clear or not In brief, reading aims learners to discover and understand a variety of texts, which entail reading and thinking strategies for understanding, fluency, and extensive reading It is also make learners read and analyze literary texts for seeking information and ideas, which will help them develop meaningful associations

Writing skill a productive mode of English in which students' language knowledge is effectively reinforced, writing represents a significant set of skills through which language learning and communication can be achieved (Weigle,

2002, p.9) Besides, Writing is viewed as a basic skill of language in which students should be competent so as to achieve the aims of EFL language learning and/or academic success (Kurt & Atay, 2007, p.15) Competency in writing requires cognitive skills as understanding, applying, and synthesizing knowledge (Defazio et al., 2010, p.34) That's why producing a written text which is accurate, organized, clear, and fluent is regarded as a challenging task (Weigle, 2002: 11) Moreover, writing cannot be separated from the acquisition of other EFL skills Alharbi (2015) also adds that writing skill in the textbook understudy has controlled and guided compositions in the early stages and do not demonstrate techniques for writing compositions Finally, as being mentioned previously, Parts 6–7 are mainly about writing There is guided writing and picture story It will take about 1 hour to finish the two tasks There are usually four rubrics to mark the writing skill: Content, Communicative achievement, Organization-structure and Language-grammar and vocabulary Each of categories is scored out of 5, so candidates can get a maximum

of 20 marks where 3/5 basically means that they have passed this part of the

writing See Table 2.3 below:

Criteria for evaluating writing skill Score Scale (1 to 5)

1 Content: answering the task,

supporting ideas

1 2 3 4 5

2 Communicative achievement –

register, tone, clear ideas, conventions

of the specific task type

1 2 3 4 5

3 Organization-structure of the text,

logical order, connected ideas

1 2 3 4 5

4 Language-grammar and vocabulary 1 2 3 4 5

Table 2.3 Criteria for evaluating KET writing skill

2.3.2.2.2 Listening skill

Listening comprehension is an active process of constructing meaning and this

is performed by applying knowledge to the incoming sounds (Buck, 2001) Gary

(1978) describes listening as an active process in which students’ listening competence can be expanded by orally giving them non-verbal tasks to carry out Lynch and Mendelsohn (2002) claim that listening comprehension consists of a variety of related processes comprising oral word recognition, perception of intonation patterns and interpretation of the relevance of what is being said to the current topic In addition, for O’Malley and colleagues (1989), what makes listening

an active process is that listeners focus on selected aspects of the aural input and construct meaning by relating what they hear to their prior knowledge Lastly, the A2 Key Listening paper has five parts: Multiple Choice, Gap fill, and matching The allotted time for this part takes around 30 minutes, including 6 minutes’ transfer time For each part, candidates have to listen to a recorded text or texts and answer some questions The candidates can hear each recording twice The misspelling of words is not accepted Each correct answer will get a certain score

2.3.2.2.3 Oral Competency

Fisher (2007) defines that speaking refers to the distinctive human use of oral competence in exchanging and sharing his or her opinion, information and feeling Furthermore, speaking is reported by Second-Language students as the hardest skill

to be proficient in To be good at speaking skill, students should have full knowledge in vocabularies, grammar and pronunciation Therefore, speaking is the skill of putting the correct words in the right order with the proper pronunciation and function (transaction and interaction) According to Mckee (2012), reading comprehension is defined as the capability of understanding texts, analyzing the information found in these texts, and the ability to interpret the writer’s statements and intents correctly

The A2 Key Speaking test has two parts: interview and discussion and the candidates take it with each other There are common topics: Daily life, Personal details, Home, Food and drink, Hobbies and leisure, Sport, Personal feelings, opinions and experiences, Health and exercise… in A2 Key It will take from 8 to10 minutes per pair of candidates; 13-15 minutes per group of three There are two examiners One examiner talks to the candidates and the other examiner listens Both examiners give marks for the candidates’ performance

How do the examiners assess candidates’ speaking? Particularly, the test examiners focus on candidates’ English Grammar, Vocabulary, Discourse Management, Pronunciation, Interactive Communication and Global Achievement The marks are for what the candidates do over the whole Speaking test, not for each part of the test See the Criteria for evaluating Speaking in Table 2.4:

Criteria for evaluating

Speaking

Score Scale (Example: Scale from 1 to 5)

1 Grammar and vocabulary 1 2 3 4 5

2 Discourse Management 1 2 3 4 5

3 Pronunciation 1 2 3 4 5

4 Interactive Communication 1 2 3 4 5

5 Global achievement 1 2 3 4 5

Table 2.4 Criteria for evaluating KET Speaking skill

2.4 Impacts of MORE! 3 student’s book on students’ KET exam achievement

As MORE! 3 was taught for many years at TVC with the main purpose to facilitate KET learners to pass KET exam However, there was a big gap between

an objective of learning this book (communication achievement) and an objective of KET course (KET exam achievement) Evaluating impacts of MORE! 3 on the

students’ learning outcomes was really important It should be carried out after the teachers and students used the book, especially after the students took their actual KET test so that they could reflect their evaluation of this book exactly and

completely Cunningsworth (1975) agrees that evaluation of the course books in the

post-use stage usefully helps the users decide to continue using them or not In the case that this student’s book made many benefits which helps the learners passed the KET exam effectively, it should be continuously recommended for use in next KET courses On the contrast, if it caused so many disadvantaged or did not support much for the learners to prepare for tests, it should be dismissed and changed as soon as possible

Tomlinson (2003) also states that defining the impacts of course books on users is probably valuable for the process of evaluation because it involves measuring the effects of the materials on users after its use A chosen student’s

book in a course helps the managerial and teaching staff select the most appropriate materials available for a particular program It also helps to identify the strengths and weaknesses of a particular student’s book that is already in use This is to inform teachers in the process of the student’s book adaptation and decision-making for the next course It is in this view that the present study mainly seeks to examine the suitability of MORE! 3’s for use and its impacts on students’ KET exam achievement

According to Littlejohn’s (2008) framework on evaluating materials, there would have 3 levels of analysis of the materials’ publication (superficial features), the roles of both teachers and learners (principles of second language acquisition) and the role of materials (three domains of Bloom (1965)) Each task and activity in MORE! 3 would be analyzed in order to find out whether they were appropriate for KET exam Even all of available proposed components in MORE! 3, such as language skills, language-grammar, vocabulary, supplementary practical materials, review lessons, task strategies and other related sections should be evaluated Most importantly, Littlejohn’s framework (2008 & 2011) revealed that the match and the evaluation of the materials’ aspects (design and publication) caused an action of rejecting, adopting or supplementing the materials then

Shortly, materials evaluation via analysis of materials and the target situation of use would help to decide to use the materials or not Moreover, it would bring benefits to help identify reasons the materials already adopted were not achieving

the educational aims or reasons the users wanted a change

2.5 A review of previous studies

This section reviews some research studies carried out on textbook evaluation

by different researchers in different parts of the world over the last 10 years The researcher would summarize and present the findings and assessment of these studies inside and outside Vietnam

2.5.1 Studies in Vietnam

Generally, almost researchers confirm that textbooks play a very important role and make strong effectiveness to English training these days Nguyen, T T M (2007) with her “Textbook evaluation: the case of English textbooks currently in use at Vietnam’s upper-secondary school Unpublished research report” examined

both the strengths and weaknesses of a new series of English textbooks currently developed for use at the upper secondary school level in Vietnam Nguyen’s study (2015) evaluated English 6, a textbook used for grade 6 students in the teaching and learning context of the Mekong Delta The evaluation intended to discover what users thought of the textbook, how it affected users, and their suggestions The researcher used both theoretical and empirical analysis For the empirical evaluation, information was gathered from 22 teachers and 313 students at 8 different secondary schools in four different provinces of the Mekong Delta using questionnaires, semi-structured interviews, documents, and classroom observations The results showed the textbook's advantages and disadvantages Even though Nguyen (2015) evaluated the English 6 textbook in-depth using different criteria suggested by different scholars and research tools like observation, questionnaires for students and teachers, and interviews for students and teachers, he mainly used Tomlinson’s (2003) approach and criteria, which is not updated Moreover, the textbook English 6 is not currently in use in Vietnam, therefore, his study is not meaningful at present Specifically, Hoàng Văn Vân (Ha Noi, 2015) on

“Development of the Ten-Year English Textbook Series for Vietnamese Schools under the National Foreign Language 2020 Project: A Cross-Cultural Collaborative Experience” pointed out the advantages of the cross-cultural collaboration in producing the textbook series In fact, there is one recently valuable study conducted by a learner of USSH, which is “Vietnamese public school English textbooks: Materials analysis and recommendations” by Phạm Hoàng Huy (TESOL

at USSH, 2018-2019) Similarly, Gokturk Saglam’s (2018) investigated the back effects of English language proficiency test at a university in Turkey on how and what teachers teach Almost the participants supposed that grammar and vocabulary presented in the textbook could be modified or supplemented 75% teachers believed that the grammar was not described with demotivating, long and uneasy examples They also believed that the book was not used to teach students to follow the exams Although there have been many studies about textbooks and course books which serve for English education, no study has been conducted about the impacts of MORE! 3 student’s book used for Cambridge KET on students’ learning outcome inside and outside Vietnam context

wash-2.5.2 Studies in Western countries

The researcher revisited some typical studies of Litz (2002) in South Korea, Jahangard (2007) or Alemi and Mesbah in Iran, Aftab (2011) in Pakistan, Syrbe, M., & Rose, H (2018) in German, and Ismaeva’s (2020) in Kyrgyzstan's public schools For more explanation, Litz’s study was conducted in 2002 to evaluate

‘English Firsthand 2’ buy collecting data from users’ views at a University in South Korea Through questionnaires, the author found out many advantages of the textbook (Litz, 2002: 33) besides drawbacks The study concluded with recommendations for supplementing, modifying and adapting problematic aspects The same as Litz’s study, Jahangard’s study was conducted in 2007 to evaluate the four ELT textbooks for Pre-University Students in Iranian High Schools He also found out strengths and weaknesses of these textbooks (Jahangard, 2007: 147) Because this was a theoretical and subjective evaluation, involving only the author’s opinions without feedback from users, so bias is unavoidable Particularly, the two studies by Litz’s and Aftab’s provide readers with general background information

of English language teaching and learning as well as the role of English in the contexts in which the textbooks are used However, the limitations of these studies are as follows:

Firstly, few of the studies mention the particular language of teaching and learning contexts (teaching and learning conditions; learners’ backgrounds; learners’ needs; and the role of textbooks in the contexts) in which textbooks are used None of the above-mentioned studies were conducted with classroom observation, in order to see how exactly the textbooks are used and what exactly happens in classroom lessons More importantly, none of these above studies analyzed specific units from textbooks, in order to strengthen their overall analysis and conclusions

Secondly, Jahangard (2007) evaluated the textbooks theoretically from their own perspective, adopting the evaluation criteria available in the literature They may therefore, be subjective to a certain extent

Finally, the impact of the textbooks on learners (learners’ study results, and learners’ use of English in communication) was not mentioned For the world-scale relevant research, Xuesong Wang (2011) in China on “Research