Teachers and students evaluation of the new tiếng anh 11 a study at three upper secondary schools in quang ninh province

VIETNAM NATIONAL UNIVERSITY, HA NOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST- GRADUATE STUDIES *************************** BÙI BÍCH PHƯƠNG TEACHERS' AND STUDE

VIETNAM NATIONAL UNIVERSITY, HA NOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST- GRADUATE STUDIES

***************************

BÙI BÍCH PHƯƠNG

TEACHERS' AND STUDENTS' EVALUATION OF THE

NEW TIẾNG ANH 11: A STUDY AT THREE UPPER

SECONDARY SCHOOLS IN QUANG NINH PROVINCE

(ĐÁNH GIÁ CỦA GIÁO VIÊN VÀ HỌC SINH VỀ CHẤT LƯỢNG

SÁCH GIÁO KHOA MỚI TIẾNG ANH 11: NGHIÊN CỨU TẠI

3 TRƯỜNG TRUNG HỌC PHỔ THÔNG Ở TỈNH QUẢNG NINH)

M.A MINOR PROGRAMME THESIS

Field: English Teaching Methodology Code: 60140111

HA NOI – 2017

Số thứ tự:……

VIETNAM NATIONAL UNIVERSITY, HA NOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST- GRADUATE STUDIES

***************************

BÙI BÍCH PHƯƠNG

TEACHERS' AND STUDENTS' EVALUATION OF THE

NEW TIẾNG ANH 11: A STUDY AT THREE UPPER

SECONDARY SCHOOLS IN QUANG NINH PROVINCE

(ĐÁNH GIÁ CỦA GIÁO VIÊN VÀ HỌC SINH VỀ CHẤT LƯỢNG

SÁCH GIÁO KHOA MỚI TIẾNG ANH 11: NGHIÊN CỨU TẠI

3 TRƯỜNG TRUNG HỌC PHỔ THÔNG Ở TỈNH QUẢNG NINH)

M.A MINOR PROGRAMME THESIS

Field: English Teaching Methodology Code: 60140111

Supervisor: Prof Dr Hoàng Văn Vân

HA NOI – 2017

Số thứ tự:……

i

DECLARATION

I hereby certify that the MA thesis entitled Teachers' and Students'

Evaluation of the New Tiếng Anh 11: A Study at three Upper Secondary Schools

in Quang Ninh Province is the result of my research for the Degree of Master of Arts

at University of Languages and International studies, Vietnam National University, Hanoi The thesis has not been submitted for any degree at any other university or tertiary institution

In terms of these conditions, I agree that the origin of my paper deposited in the library should be accessible for the purposes of study and research, in accordance with the normal conditions established by the librarian for the care, loan and reproduction of the paper

Hanoi, May 2017

Bùi Bích Phương

I also wish to thank all my lecturers at University of Languages and International Studies, Vietnam National University, Hanoi for their precious lectures whose ideas are partly employed in this study

I am also immensely grateful to 6 English teachers and 160 students in three top high schools in Quang Ninh province who were enthusiastic in their responses to my questionnaires Especially, I would like to express my special thank to 14 interviewees for spending time answering my questions Moreover, I would like to say “thank you”

to the boards of managers in these three schools who created convenient conditions in order that I was able to complete my surveys

Last but not least, I owe the completion of this study to my big family, my old teachers and my close friends who always stand by my side with encouragement and consideration Thanks for their spiritual support throughout this study

iii

ABSTRACT

Decisions related to textbook selection and evaluation are of great importance

The purpose of this study is to evaluate the new Tiếng Anh 11 compiled under the

Project 2020 used at several piloted high schools in Quang Ninh province by getting feedback from both students and teachers

For this purpose, a 48-item questionnaire for students and a 53-item questionnaire for teachers were developed about the physical features and organization, textbook contents (topics, texts, activities, skills, pronunciation, grammar, vocabulary), and supplementary materials, teacher’s book, objectives and curriculum fit 160 students in piloted classes and 6 high school English teachers using the new textbook

in three top high schools in Quang Ninh province were asked to reflect on the questions

by checking one of the five options included After, the interviews were conducted with 10 students and 4 teachers with more than five years of teaching experience to discuss in detail

The results indicated that the textbook’s qualities, in general, satisfy both students and teachers in terms of design, structure and contents The most remarkable improvement of the book is the way it is organized sections In addition, that the book tends to develop students’ communicative competences through a variety of activities

in class attracts them a lot Besides, there still have some limitations of the book that were discussed Two of them are the lengths of reading texts and the number of grammar introduced in each unit Finally, significant recommendations were also given

to get the textbook improved

Key words: Textbook evaluation, criteria, Project 2020, evaluate, strengths,

weaknesses

iv

TABLE OF CONTENTS

DECLARATION i

ACKNOWLEDGEMENTS ii

ABSTRACT iii

TABLE OF CONTENTS iv

ABBREVIATIONS vii

LIST OF TABLES viii

PART I INTRODUCTION 1

1 Research statement and rationale of the study 1

2 Aims and objectives of the study 2

3 Scope and limitations of the study 3

4 Significance of the study 4

5 The structure of the study 4

PART II DEVELOPMENT 6

CHAPTER 1 LITERATURE REVIEW 6

1 Introduction 6

2 Teaching and learning English in Vietnam under the National Foreign Language Project 6

2.1 The NFL 2020 Project 6

2.2 An Overview of the Pilot English Curriculum for Upper Secondary schools 7 2.3 An Overview of the Pilot Textbook Tiếng Anh 11 8

3 Key concepts 10

3.1 Definitions of textbook 10

3.2 Roles of textbooks 11

3.3 Textbook evaluation 12

3.4 Textbook evaluative criteria 13

3.5 Previous studies on textbook evaluation 18

v

3.6 Research gap 21

CHAPTER 2 RESEARCH METHODOLOGY 22

1 Research questions 22

2 Research design 22

2.1 Research design 22

2.2 Participants and sampling size 23

2.3 Data collection instruments 24

2.3.1 Questionnaires 24

2.3.2 Interviews 26

2.4 Data analysis methods 27

2.4.1 Statistics analysis 27

2.4.2 Content analysis 27

3 Research procedures 28

3.1 Data collection procedure 28

3.1.1 Stage 1 Questionnaire 28

3.1.2 Stage 2 Interview 28

3.2 Data analysis procedure 29

3.2.1 Stage 1 Analyzing collected data from questionnaires 29

3.2.2 Stage 2 Analyzing collected data from interviews 30

CHAPTER 3 MAJOR FINDINGS AND DISCUSSIONS 31

1 Overall rating 31

2 Students’ and teachers’ evaluation of the textbooks in each of 5 main categories 32

2.1 The evaluation in the “Physical features and organization” category 32

2.2 The evaluation in the “Textbook contents” category 34

2.2.1 The teachers’ and students’ evaluation of the “Topics” section 35

2.2.2 The teachers’ and students’ evaluation of the “Texts” section 36

vi

2.2.3 The teachers’ and students’ evaluation of the “Activities” section 37

2.2.4 The teachers’ and students’ evaluation of the “Skills” section 39

2.2.5 The teachers’ and students’ evaluation of the “Pronunciation” section 40 2.2.6 The teachers’ and students’ evaluation of the “Vocabulary” section 41

2.2.7 The teachers’ and students’ evaluation of the “Grammar” section 42

2.3 The teachers’ and students’ evaluation of the “Supplementary materials” category 43

2.4 The teachers’ evaluation of the “Teacher’s book” category 44

2.5 The teachers’ evaluation of the “Objectives and Curriculum Fit” category 46 CHAPTER 4 RECOMMENDATIONS 47

PART 3 CONCLUSIONS 51

1 Summary 51

2 Limitations 54

3 Recommendations for further research 55

REFERENCES 56 APPENDICES I

APPENDIX I QUESTIONNAIRE FOR STUDENTS I APPENDIX II PHIẾU KHẢO SÁT DÀNH CHO HỌC SINH VI APPENDIX III QUESTIONNAIRE FOR TEACHERS X APPENDIX IV QUESTIONS FOR INTERVIEWS XV APPENDIX V QUESTIONNAIRE RESULTS XXII APPENDIX VI THE INTERVIEWS’ SUMMARY XXIX

vii

ABBREVIATIONS

MOET: Minister of Education and Training VNU: Vietnam National University, Hanoi ELT: English Language Teaching

NFL 2020: the National Foreign Language 2020 Project CEFR: Common European Framework of References CLT: Communicative Language Teaching

viii

LIST OF TABLES Table 1 Textbook evaluation criteria by Mukundan, Jayakaran, and Nimehchisalem

(2011)

Table 2 Textbook evaluation criteria by Hoàng Văn Vân (2015)

Table 3 Evaluation on general features and organization

Table 4 Evaluation on Topics

Table 5 Evaluation on Texts

Table 6 Evaluation on Activities

Table 7 Evaluation on Skills

Table 8 Evaluation on Pronunciation

Table 9 Evaluation on Vocabulary

Table 10 Evaluation on Grammar

Table 11 Evaluation on Supplementary materials

Table 12 Evaluation on Teacher’s book

Table 13 Evaluation on Objectives and Curriculum Fit

1

PART I INTRODUCTION

1 Research statement and rationale of the study

The textbook is one of the crucial factors in determining the learners’ success in language course (Mukundan, Jayakaran, and Vahid Nimehchisalem, 2012) In a developing country like Vietnam, it has been a regular practice to consider textbooks as

a major source of the learning process to be undertaken in schools (Hoàng Văn Vân, 2012; Mahomood, 2010) Foreign language textbook is not an exception Vietnam is a country where English has become the compulsory subject in the educational curriculum from primary to upper secondary school In response to this fact, on 20 September, 2008 Decision No 1400/ QD-TT was issued by the Prime Minister on The Approval of the 10-year National Project for “Teaching and Learning Foreign Language in the National Formal Educational System Period 2008-2020” (the National Foreign Language 2020 Project, [NFL2020]) In implementing the Decision, be aware

of the significant role of textbook, the Vietnamese Ministry of Education and Training (MOET) started to design the new curriculum and new English textbook with an aim to

“form and develop English language communication competence” (Piloted English Language Curriculum for Lower Secondary Education, p.3)

In the first and second phases of this Project, the pilot teaching and evaluation of the new textbook are the focus According to Sheldon (1988), textbook evaluation is necessary as it will help the teacher or program developer in making decisions on selecting the appropriate textbook, and familiarize the teacher with its future instruction That is the reason why the book evaluation is not new in English Language Teaching (ELT) in the world For instance, Tomlinson et al (2001) evaluated some textbooks published by leading UK publishers: “Language in Use” and “True to Life” (by Cambridge University Press), “Cutting Edge” (by Pearson Longman), or textbook evaluation in Pakistan (Mahmood, 2010)

Tiếng Anh 11 is developed to comply with the NFL 2020 Project and the Piloted

English Language Curriculum for Vietnamese upper secondary schools It has been adopted as textbooks for many upper secondary schools including Ha Long High School for gifted students, Cam Pha High School and Uong Bi High School, top high schools in three big cities in Quang Ninh province Although the new textbooks have been piloted for several years, there are no studies conducted to evaluate these books in particular contexts It is necessary that an evaluation on the textbook from perspectives

of students and teachers should be carried out to discover whether the book is suitable within the context of teaching and learning It is also hoped that recommendations will

be made for the textbook writers to improve the textbook’s quality to meet the goal of the NFL 2020 Project

Based on the gap that needs to be met as presented above, the researcher would

like to conduct the study “Teachers' and Students' Evaluation of the New Tiếng

Anh 11: A Study at three Upper Secondary Schools in Quang Ninh Province”

2 Aims and objectives of the study

This research aims at evaluating the quality of the new Tiếng Anh 11 compiled

under the Project 2020 to find out the strong points and weak points of the new textbook Based on the findings, practical recommendations for textbook writers will

3

be made to improve the quality of the textbook to meet the requirements set by MOET (via the National Foreign Language 2020 Project): Upon finishing the secondary education, students’ English proficiency will reach Level 3 (equivalent to CEFR B1)

To achieve the above aims, the following objectives, presented in the form of two research questions, are set as follows:

1 What are the strengths and weaknesses of the new Tiếng Anh 11 as seen from the evaluation of the teachers and students who are using it?

2 What recommendations should be made for the textbook writers to improve the textbook’s qualities?

3 Scope and limitations of the study

In terms of theoretical framework for textbook evaluation, there have been various checklists and criteria suggested and developed by different scholars, such as Williams (1983); Sheldon (1988); Cunningsworth & Kusel (1991); Harmer (1998); Richards (2001); Driss (2006); Rahimy (2007) Lê Thị Bắc (2013) states that as any of the checklists aim at evaluating textbook in general with no specification for a specific textbook under a particular context of evaluation like this, the evaluation criteria used

in the study is a combination of evaluation frameworks and checklists proposed by Mukundan, Jayakaran, and Vahid Nimehchisalem (2011) and William (1983)

Besides, the population is restricted to 6 teachers and 160 students of grade 11 at Uong Bi High School, Ha Long High School for Gifted Students, and Cam Pha High

School in Quang Ninh province who are using the new Tiếng Anh 11

There are two factors limiting the results of the study: a short time span for the study and a limited sample size The researcher could not carry out the survey on a larger scale so more or less the results might not be reliable enough for such a

4

generalization In addition, combination of different evaluation checklists cannot ensure the best set of criteria

4 Significance of the study

Once this thesis is completed, it can be of both theoretical and practical significance In terms of theoretical contributions, the results are anticipated to provide

an appropriate textbook evaluation checklist basing on previous ones Besides, the findings should make an important practical contribution to establishment of qualities

for a better Tiếng Anh 11 It is also hoped that with the evaluation from direct textbook

users, students and teachers, the study will give positive feedback on the textbook so that it will soon be adopted to teach English for 11-graders in the nationwide In addition, this thesis is expected to serve as a reliable and useful reference material about textbook evaluation for other readers and researchers in their further studies on related subjects

5 The structure of the study

The study includes 3 parts as follows:

Part A – Introduction – presents the background to the study, the statement of

the problem, the aims, the research questions, and the scope and the significance of the study

Part B – Development

This part consists of 4 chapters

Chapter 1 – Literature review – provides a short introduction about NFL 2020

Project, an overview of the Piloted English Language Curriculum for upper secondary schools in Vietnam, a brief review of the new “Tiếng Anh 11” In addition, a number

5

of definitions of key terms by famous authors are also presented in this chapter The study, furthermore, applies some common sets of textbook evaluation checklists in order to create a questionnaire for data collection Besides, previous studies in the world and in Vietnam of textbook evaluation are also reviewed

Chapter 2 – Methodology – describes the research methodology employed in this study It introduces the research design, participants, sampling strategies, research instruments and data collection procedures

Chapter 3 – Findings and Discussion – presents data analysis, research results and discussion

Chapter 4 – Recommendations – gives several recommendations from students’ and teachers’ perspectives to improve the quality of the new textbook

Part C – Conclusion – summarizes what has been studied and gives

recommendations for the improvement of the textbook

2 Teaching and learning English in Vietnam under the National Foreign

Language Project

2.1 The NFL 2020 Project

The Vietnam MOET is currently launching the project entitled “Teaching and Learning foreign Languages in the National Education System Period 2008 – 2020” The NFL 2020 Project specifies the goals of teaching and learning English within the national education system and identifies seven tasks at both macro and micro levels for implementing the project in the three different phases, which is said to be worth 9.378 billion VND The first phase from 2008 to 2010 prioritizes to developing and perfecting the 10-year foreign language curriculum, focusing for exercising the pilot 10-year English program The second phase which extends from 2011 to 2015 focuses

on introducing the 10-year foreign language program throughout the whole general educational system And in the third phase, the focus is on perfecting the 10-year foreign language program throughout the whole general educational system and on developing intensive foreign language program for vocational schools, colleges and

universities (Hoàng Văn Vân, 2010) The main goals of the project is that ' by 2020

7

most Vietnamese youth whoever graduate from vocational schools, colleges and universities gain the capacity to use a foreign language independently This will enable them to be more confident in communication, further their chance to study and work in

an integrated and multi-cultural environment with variety of languages.' (Decision No

1400/QD-TTg) This goal also makes foreign language as an advantage for Vietnamese people, serving the cause of industrialization and modernization for the country

2.2 An Overview of the Pilot English Curriculum for Upper Secondary schools

Upper secondary education in Vietnam consists of three grades 10, 11 and 12 The upper secondary curriculum needs to give students more opportunities to enhance both accuracy and fluency in English through 315 periods in total (105 for each grade) Students practice and develop not only communicative competences (Reading, Speaking, Listening, and Writing), but linguistic skills (Pronunciation, Vocabulary, and

Grammar) with 4 main themes: Our lives, Our society, Our environment, and Our future The curriculum focuses on communicative competence so that when finishing

upper secondary school, they will have achieved level 3 of the Foreign Language Proficiency Framework for Vietnam (equivalent to B1 in the CEFR)

The pilot English for upper secondary schools includes two parts: general issues and suggestions for teaching contents for each grade In the first part, the principles to build and develop the curriculum and the objectives are presented Curriculum’s contents consisting of systems of themes and topics, communicative competences, linguistic knowledge, learning how to learn, the adoption of teaching method of Communicative Language Teaching (CLT) and learner-centered approach, testing and assessment procedures and conditions for curriculum implementation are further described in the curriculum The second part provides teachers with suggestions for teaching in each grade Themes and topics are clarified with specific suggested communicative competences and language items for each topic

8

2.3 An Overview of the Pilot Textbook Tiếng Anh 11

Tiếng Anh 11 is the second of a three-level English language set of textbooks for

the Vietnamese upper secondary school It is divided into two volumes, lasts 105 periods as the other 2 grades Communicative competences are taught through four language skills in relevance to four main themes mentioned above Especially, students have a chance to get to know cultures of Vietnam, of ASIAN countries, and of some main English-speaking countries

Under the curriculum for upper secondary schools, the textbook “Tiếng Anh 11” also contains four main themes Each theme is covered in three units with different topics

Theme 1: Our lives: The generation gap, Relationships, Becoming independent

Theme 2: Our society: Current social issues, Caring for those in need, Vietnam and ASEA

Theme 3: Our environment: People and the environment in conflict, Global Warming, Preserving our heritage

Theme 4: Our future: Further education, Health care and longevity, the future of cities

After each theme, there is a review Each unit is divided into five sections taught

in 8 lessons: getting started, language, four skills (reading, speaking, listening, and writing) communication culture, look back and project

Section 1 – Getting started – contains a menu listing the language and the

skills taught in unit, a conversation introducing the overall topic of the unit, some topic-related vocabulary and the main grammar points, and a number of activities and practice to check students’ comprehension

9

Section 2 – Language – includes vocabulary (in-depth practice of words and

phrases, additional vocabulary for use later in the unit, and word collocation); pronunciation (problematic aspects of pronunciation: sound clusters, weak forms, stress patterns, and intonation); and grammar (main grammar point in focus taught in meaningful contexts) All the sub-sections in the Language section are linked to the looking back section at the end of the unit

Section 3 – Skills – includes Reading, Speaking, Listening and Writing Reading texts provide an input of language and ideas for students to use in the other

skills The vocabulary and grammar points learnt in the previous sections are also recycled in the reading text The reading activities are designed in the order of three stages: pre-reading, while-reading and post-reading A variety of activities aim to develop reading skills such as skimming, scanning, through various types of tasks

Speaking follows the reading section, which links between two parts All speaking

activities aim to encourage students to interact with others to share their experience

with their partners Listening is also organized the same to reading, including

pre-listening, while-listening and post-listening Both post-reading and post-listening

stages create speaking environment for students And Writing guides students through

the writing process and focusing on the specific text types required by the syllabus Students are provided with a template and useful language before producing their own

writing

Section 4 – Communication and culture – includes Communication and

Culture Communication recycles what students have learnt in the previous sections in

terms of language items Students are encouraged to express their ideas freely without

or less control of the teacher Culture contains cultural information about the ASEAN

countries and English speaking countries around the world connected to the cultural

10

aspects of the unit topic It helps students to broaden their knowledge of the world cultures

Section 5 – Looking back and Project – includes two separate sub-sections:

Looking back and Project Looking back offers revision and consolidation of the

language learnt in the unit, including pronunciation, vocabulary and grammar Activities aims to check students’ pronunciation and understanding of the meaning and

use of the words or structures Project aims at providing students with an opportunities

to combine all skills and knowledge learnt in the unit to make their final product The project activities often involve teamwork which develop students’ collaborative skills and promote their team spirit

3 Key concepts

3.1 Definitions of textbook

According to Awasthi (2006), textbook can be defined as a teaching material for the teacher and a learning material for the learner A broader perspective has been adopted by Mahmood (2010) who argues that “textbook is a source of potential learning as to what students learn from textbooks and the practicality of that learning is mediated by the school context (teacher, peers, instruction, and assignments) This view is supported by some giants in the world to affirm the big contribution of textbook that it is “the visible heart of any ELT program (Sheldon, 1988: 237); “an almost universal element of teaching” (Hutchinson and Torres, 1994: 315); and “a guide for a teacher, a memory aid for the pupils, a permanent record or measure of what has been learnt” (Awasthi, 2006) From another aspect, Ur (1996: 183) emphasizes that textbook is a “framework” which regulates and times the programs Therefore, he points out the limitations of using textbook that inhibits the teachers’ creativity and cannot take account of individual differences in learners

11

While a variety of definitions of the term “textbook” have been suggested, this paper will use the definition first given by Awasthi (2006) and just consider the advantages of textbook

3.2 Roles of textbooks

Different writer have given different advantages of textbooks As mentioned in the definitions, textbooks are very useful tools in the hand of a teacher They play a necessary role in both teaching and learning They consist of detailed material on current topics and are designed to facilitate language development and knowledge acquisition They are central to the teaching and learning of any language program because they represent the appropriate content specifications selected from the total corpus of language for teaching purposes and for what is considered most useful to the learners as stated in the curriculum Richard (2001) lists the following considerable advantages of using textbooks:

- They provide structure and syllabus for a program

- They help standardize instruction

- They maintain quality

- They provide a variety of learning resources

- They are efficient

- They can provide effective language models and input

- They can train teachers

- They are visually appealing

The wide spread use of textbooks in different ELT contexts requires little further explanation Dendrinos (1992), Lee (1997) and William (1983) are unanimous

in their opinion that it continues to play an essential role in ELT classrooms all over the world The importance of textbooks in the ELT classroom is so extensive that it is

3.3 Textbook evaluation

According to Mahmood (2010), evaluation of textbooks is considered as one of the means for regulating and standardization of the textbooks to maintain their quality across the nation and/ or region In the same vein, Tomlinson (2003: 15) defines this activity as “procedure that involves measuring the value of a set of learning materials, making judgment about the effect of the materials on the people using them.” In short, both the authors consider textbook evaluation a way of judging the quality of a book Thanks to this assessment, “careful selection is made, and that the materials selected closely reflect (the needs of the learners) and the aims, methods, values of the teaching program” (Cunningsworth, 1995: 7) Textbooks found unsuitable for the use would have been screened out after this process (McGrath 2002)

Discussing the reason why the evaluation of textbook is necessary, Cunningsworth (1995) and Ellis (1997) believe that this activity helps teachers move beyond impressionistic assessments and it further facilitates them to acquire useful, accurate, systematic, and contextual insights into the overall nature of textbook materials (Litz, 2005) Through the textbook evaluation, a teacher knows the content of the book, the style in which it is written, and its strength and weaknesses, which facilitate the teacher to adapt it to suit the course aims, learners’ needs and the teachers’ belief In addition to this idea, Brown (1995) highlights that textbook evaluation is “the systematic collection and analysis of all relevant information

13

necessary to promote the improvement of a curriculum and assess its effectiveness within the context of the particular institution involved” Textbook evaluation has become such an important applied-linguistic activity that it has attracted the attention

of many linguistic researchers since the 1980s

In view of the above, it can be said that textbook evaluation is the process of judging the necessary information systematically to see the value of it on its specific users, taking the contextual factor into consideration Wrong choice of textbooks would

be likely to negatively affect both teaching and learning Financial resources would also be wasted (Mukundan 2007, Sheldon 1988)

3.4 Textbook evaluative criteria

Many big authors have proposed different models and frameworks for the evaluation of textbook or course book A checklist is an instrument that helps practitioners in ELT to evaluate language teaching materials, like textbooks (Mukundan, Jayakaran, and Vahid Nimehchisalem, 2011) These checklists may be quantitative or qualitative Quantitative scales have the merit of allowing an objective evaluation of a given textbook through Likert scales (Skierso, 1991) Qualitative checklists, on the other hand, often use open-ended questions to elicit subjective information on the quality of course book (Richards, 2001) While the qualitative checklists are capable of an in-depth evaluation of textbooks, quantitative checklists are more reliable instruments and are more convenient to work with, especially when team evaluations are involve Because of this advantage, in this study, the researcher decided to use the later type of checklist

Different researchers prefer to use different sets of evaluative criteria to evaluate

a textbook depending on the purpose of their study There are some researchers who are interested in evaluating the physical features of the textbook Some focus on

14

assessing the content areas of the book (themes, topics, grammar, vocabulary, skills, etc.) There are even researchers tending to assess textbook from as many categories as possible, not only physical futures but also language content, teaching and learning methods, and other practical considerations Preeminent theorists in the field of ELT textbook design and analysis such as William (1983), Sheldon (1988), Brown (1995), Cunningsworth (1995) and all agree that evaluation checklists should have some criteria pertaining to the physical characteristics of textbooks such as layout, organizational, and logistical characteristics Other important criteria that should be incorporated are those that assess a textbook’s methodology, aims, and approaches and the degree to which a set of materials is not only teachable, but also fits the needs of the individual teacher’s approach as well as the organization’s overall curriculum Finally, textbook evaluation should include criteria that pertain to representations of cultural and gender components in addition to the extent to which the linguistic items, subjects, content, and topics match up to students’ personalities, backgrounds, needs, and interests as well as those of the teacher and institution

William (1983) proposes a set of evaluative criteria which are derived from a combination of the linguistic and pedagogical components of language teaching analysis “General” criteria embrace global considerations of methodology, the needs

of the learner, the teacher, and the community “Technical” criteria are concerned with the quality of editing and publishing; the availability of supplementary materials; costs and dura-ability of the text; authenticity of language and style of the write, etc

Tomlinson (2003) defines three types of material evaluation: pre-use evaluation, while-use evaluation and post-use evaluation He also agrees that material evaluation is

a procedure to measure some or all of the aspects such as: appeal to learners; credibility

to learners, teachers and administrators; validity; reliability; ability to interest learners; ability to motivate learners; short-term and long-term learning value; learners' and

15

teachers' perception of the value of the material; assistance given to teachers in terms

of preparation, delivery and assessment, etc

As it has been argued by some scholars (e.g., Sheldon, 1998) that evaluative criteria of checklists “should be chosen according to the learning-teaching context and the specific needs of the learner and teacher” (Mukundan, Jayakaran, and Nimehchisalem, 2011) However, the review of textbook evaluation checklists across four decades (1970-2008) by Mukundan and Ahour (2010: 336-352) determines their length, the common criteria and their overall structure as they evolved across this period Most well-established checklist such as Cunningsworth (1984) also recommends a checklist of evaluation criteria consisting of seven sections (language content, selection and grading of language items, presentation and practice of new language items, developing language skills and communicative abilities, supporting materials, motivation and the learner, and overall evaluation

In addition to the criteria mentioned above, a checklist must take into account the background of the target students, and the language used in the various texts of the textbook under evaluation should be natural and authentic Most of these checklists, Mukundan, Jayakaran, and Nimehchisalem justify that, are either too short or too long and some criteria are vague, so they do not thoroughly meet the requirements of a good and applicable instrument for evaluation purpose Therefore, Mukundan and his coworkers designed a tentative checklist The main component of the criteria is clarified in the diagram bellow

17

Table 2 Textbook evaluation criteria by Hoàng Văn Vân (2015)

It is obvious that many identical evaluative criteria had been developed in different parts of the world for different learning-teaching situations and purposes These publications provide a useful starting point for any evaluation process

Lê Thị Bắc (2013) concludes that despite the fact that these checklists are developed under different rationales, they still share three common features:

 In terms of learners, textbooks should correspond to students’ interests, needs, level of proficiency and the socio-cultural background

18

 In terms of Curriculum, textbooks should correspond to the Curriculum Guidelines and reflect the aims of teaching

 In terms of methodology, textbook compilation should be based on the theories

of second language acquisition and on the latest language teaching methods This study will draw on the criteria established by Mukundan and his coworkers (2011) in combination with the work of Hoàng Văn Vân (2015)

3.5 Previous studies on textbook evaluation

Alamri (2008) conducted a study to evaluate the Sixth Grade English Language textbook for Saudi Boys’ schools A 63-item questionnaire was grouped under 12 main categories: the general appearance, design and illustration, accompanying materials, objectives, topic appropriateness, language components, socio-cultural contexts, skills development, teachability, flexibility, teaching methods, and practice and testing The findings of this study reveal that this is a good textbook because it satisfies almost all the categories listed Some recommendations are also given to improve the textbook better One of the strengths of this study is the checklist for textbook evaluation which consists of detailed criteria that the researcher could consider it a good reference for the current study

Another research carried out by Hashemi (2011) evaluates three English language textbooks currently used at high schools in Iran from the high school English teachers’ point of view He developed a 46-item questionnaire about five sections of the textbooks (vocabulary, reading, grammar, language functions, and pronunciation practice, their physical make-up, and practical concerns This checklist could be effective in Iran context, but for the purpose of the current study, it does not cover enough aspects that the researcher would like to evaluate

19

In Vietnam, textbook evaluation has been paid attention by many researchers and educators, especially after a new book is compiled and used in some piloted cases The study “The case of English textbooks currently in use in Vietnam upper-secondary schools”, (Nguyễn Thị Thúy Minh, 2007) seeks to evaluate a new series of English textbooks currently developed for use at the upper secondary school level in Vietnam, and examine the communicative nature of the textbooks and the case of their use with a view to proposing practical implications for both textbook authors and teachers who have been and will be using the books The research was conducted using the checklists provided by Cunningsworth (1995) with some modifications Instruments: Questionnaires (consist of both close and open-ended questions - 250 upper secondary school teachers and teacher trainers working throughout the country) & Interview (via phone) The findings show the strengths and weaknesses of the new textbook:

- Clear objectives

- Balanced coverage of language

skills and language elements

- Provision of systematic knowledge

- Training teachers in teaching

methods and procedures

- Teaching grammar communicatively

non Teaching too many grammar structures

in one unit

- Difficult and long reading texts

- Difficult listening materials

- Inadequate number of communicative speaking activities

- Inauthentic language

- Unhelpful teachers’ manuals and no reference books for teachers

Under The NFL 2010 project, Lê Thị Bắc (2013) conducted a study to evaluate

Tiếng Anh 6, Volume 1, to find whether the textbook is suitable to the requirements of

the 2020 Project as well as the Piloted Curriculum for lower secondary education in the context of teaching and learning this book in some lower secondary schools in Hai Duong province A survey questionnaire has been designed basing on a combination of evaluation checklists and criteria suggested by different authors, and informal talks with the respondents are utilized as the research instruments of the study The results show a very high level of agreement for the suitability of the book among the participants Recommendations and suggestions for future use of the book are also given in the study

In 2016, Đặng Thị Cẩm Tú also publicized a study on evaluation of Primary English Textbooks in Vietnam from a socio-linguistic perspective The evaluation focuses on four main sociolinguistic aspects: teaching approach, bilingualism, language variations, and intercultural communication reflected in the primary English textbooks

As indicated in the findings, the textbook design follows communicative language teaching However, English variations and cross-cultural knowledge are still limited in the textbook design In order to improve sociolinguistic understanding among young

of two or more checklists of criteria, depending on their specific purposes

3.6 Research gap

Textbook or course book evaluation is not a very new issue; however, only some studies evaluating the new textbooks have been available recently Though applying new textbook in teaching English in upper secondary curriculum under the NFL 2020 project has been conducted for several school years, evaluation of these books through particular research in specific context is not much enough to measure

the quality of the books Up to now, Tiếng Anh 11 has been evaluated by some

previous researchers So far, however, but it is only a case study concerned with some separate aspects such as speaking activities, grammar, and tasks No single study pursues a deeply systematic investigation into the whole textbook in accordance with a comprehensive set of criteria Moreover, different subjects may bring different results

in terms of their perspectives Therefore, in order to be able to make more sound judgments about characteristics of this book, more research of this kind needs to be conducted

22

CHAPTER 2 METHODOLOGY

This study explores the quality of the new Tiếng Anh 11 from the evaluation of

high school teachers and students in Quang Ninh province To achieve this aim, I will raise the research questions and present in some detail the research design

23

and powerful when both summary numbers and in-depth portraits of a setting are included However, the complexity of mixed-methods research is a disadvantage that requires more time and resources to complete Analyzing data would be more complex because the researcher not only works with statistics but has to decode qualitative information, which demands the researcher to be experienced (containing knowledge about this method and skills to deal with gathered data)

For this study, I combine both quantitative data and qualitative data collected in two phases in which the quantitative data will be collected first, then followed by the qualitative data The quantitative data from the survey are the major focus The

researcher would like to determine how the new Tiếng Anh 11 meets the textbook

evaluative criteria in both students’ and teachers’ perspectives to recognize what aspects the book strengthens in and what problems it may have A later time, qualitative data from the interviews are used to illustrate or further explain the quantitative findings This way of data collection and analysis is called explanatory design (Lingard, Albert, & Levinson, 2008)

2.2 Participants and sampling size

160 students of grade 11 and 6 English teachers at 3 upper secondary schools in Quang Ninh province are invited to participate in the study Details are provided in the following table:

24

The reasons for the choice of the participants are as follows

Firstly, the setting of the present study is at 3 top upper secondary schools, located in the 3 biggest cities in Quang Ninh using the piloted textbook in teaching and learning under the NFL 2020 project These schools have met the requirements as set

by MOET for carrying out piloted teaching and learning: students and teachers are qualified; they have been teaching and learning the material for two years; the schools have sufficient teaching and learning facilities; and more importantly, they are willing

to provide their evaluative feedbacks to the researcher

2.3 Data collection instruments

The research adopts both quantitative and qualitative research methods with a dominance of the former through the employment of questionnaires and interviews Besides, qualitative research methods are also adopted to ask the teacher for their perspectives and recommendation to improve the new textbook

2.3.1 Questionnaires

Data will be collected in this study using closed form of questionnaire as it allows proper control when attendees are doing the survey In addition, using questionnaires makes collected data easy to manipulate and categorize for analysis Among various types of questionnaire items, the researcher decided to utilize the most

frequent question type: closed-ended (or simply “closed”) questions, which the

respondents are provided with ready-made response options to choose from That is, these items do not require the respondents to produce any free writing; instead, they are

to choose one on the alternatives, regardless of whether their preferred answer is among them To be more specific, the Likert-scale, the most commonly used scaling technique will be used to design questions Respondents are asked to indicate the

34 to 36), Pronunciation (items 37 to 40), and Grammar (items 41 to 44) The 53-item questionnaire for teachers is added with two more categories: Teacher’s book (items 49

to 51) and Objectives and Curriculum Fit (items 52 and 53)

The questionnaires for students are written in Vietnamese (English version will

be translated into Vietnamese before delivering the questionnaires) to make sure that all the items will be equally understood Here’s is an example:

English version for teachers:

26

The questionnaires for students:

2.3.2 Interviews

With the aim of enhancing the reliability and the validity of the study as well as

reducing the questionnaire’s limitation, the researcher goes on using semi-structured interview instrument, which allows the interviewer to ask probing questions and ask

interviewees to clarify unclear answers and follow upon interesting one Interviews are conducted with 10 students (at 3 schools) and 4 teachers who already completed the questionnaires The researcher will ask them more for their perception of the new textbook; and what aspects do they think that the textbook and curriculum should be improved The result from interviews will contribute to giving meaningful implications after the finding discussion part

The interviews are prepared in English but because the research is conducted in Vietnamese, it is translated into Vietnamese to make sure that the teachers and students understand what is needed by the researcher Questions for interview can be seen in Appendix 4

The combination of two data collection instruments: questionnaire and interview help the researcher to have sufficient and reliable information to evaluate the quality of

Tiếng Anh 11

27

2.4 Data analysis methods

2.4.1 Statistics analysis

After the data is collected by the five-point Likert-scale questionnaire, the

researcher goes on analyzing data by using statistical analysis method A descriptive

statistical analysis of students’ responses to the survey items is used to describe the basis features of the collected data by presenting quantitative items in a manageable form (William, 2006) This type of research is quantitative research which is described

by the terms “empiricism” (Leach 1990) and “positivism (Duffy, 1985)

Descriptive statistics are used simply to describe what is going on in the collected data There are two major characteristics of a single variable including the

“mean” for central tendency and “standard deviations” for variability, which are used

to decode the collected data on order to describe the respondents’ perception of the textbook’s quality Besides, data is also converted into percentages from overall average percentage of 48 items to average percentage of each item to make comparison easier

2.4.2 Content analysis

The interview made up the qualitative approach (content analysis method) of the study as this focused on personal accounts, description and individual insights of the respondents For qualitative analysis method, it consists of identifying, coding, and categorizing patterns and themes found in the data In this study, the researcher applies this method to classify which strengths and weaknesses of the textbook has In addition, answers recorded in the interviews will be decoded in order to give recommendations after the finding discussion

28

3 Research procedure

3.1 Data collection procedure

3.1.1 Stage 1 Questionnaire

Step 1: Preparing: In this step, the researcher will prepare and design the

questionnaire by referring to the supervisor’s consultancy and other research related to the same issue, especially the evaluative criteria of some big scholars

Step 2: Piloting: Constructive comments from the piloted group of 2 colleagues

and 2 11th graders will be considered to help the researcher make adjustment on questionnaire

Step 3: Administering: Face-to-face questionnaire administration method will

be employed On average, it will take 15 minutes to administer the questionnaires During the evaluation process, the respondents could communicate directly with the researcher to receive detailed instructions to ensure that all the items will be fully and accurately evaluated In total, 166 copies of questionnaire will be given out (160 for students and 6 for teachers)

3.1.2 Stage 2 Interview

Step 1: Preparing:

- Contact with 10 students who responded to the questionnaires and 4 teachers teaching new “Tieng Anh 11” then decides time and place for interviewing

Step 2: Interviewing: There researchers carry out interviews with 10 students

and 4 teachers to investigate their perception of new textbook they are using and their desires to promote the textbook quality The interviews are audio-tapped and then transcribed to be analyzed later

29

3.2 Data analysis procedure

The results will be presented according to the 2 research questions in chapter 3 3.2.1 Stage 1 Analyzing collected data from questionnaires

Step 1: Summarizing the gathered data

Counting the number of choices for each statement (from 1 to 5)

Step 2: Calculating data

Using the support of excel to calculating the percentage, the mean and the standard deviation of teach statement with available formulas

including the “mean” for central tendency and “standard deviations” for variability,

which are used to decode the collected data on order to describe the student’s perception of the strengths and weaknesses of the new textbook

No Items Strongly agree Agree Neutral Disagree Strongly Disagree Mean SD

4.05

0.77

30

Step 3: Putting the picture together

After calculating the mean and SD of each item, the researcher comes to picture how students evaluate the new textbook, then putting them into different categories as

in the questionnaires

3.2.2 Stage 2 Analyzing collected data from interviews

Step 1: Taping records the interview and taking notes at the same time,

translating into Vietnamese In order to keep personal information not to be identified, each interviewee will be named with a sign from S1 to S10 (for students) and T1 to T4 (for teachers) in accordance with the order of interviews

Step 2: Analyzing content of interviewee’s answer basing on the interviewed

questions

Summary

In order to achieve the study’s objectives, the researcher will apply two data collection instruments, namely questionnaires and interviews Then, the gathered data will be analyzed with both statistics and content analysis The combination of different methods and instruments enables the researcher to have a deep look into the studied issue As a result, the finding of this study would be potentially significant as expected