(LUẬN văn THẠC sĩ) đánh giá giáo trình solutions upper intermediate dùng cho học sinh chuyên anh lớp 10 tại trường THPT chuyên

ABSTRACT The purpose of this study is to gauge the suitability of a coursebook titled “Solutions Upper-intermediate” for students‟ levels of English and general expectations in terms of

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

NGUYỄN THI ̣ PHƯƠNG

AN EVALUATION OF THE COURSEBOOK “SOLUTIONS INTERMEDIATE” FOR TENTH-GRADE ENGLISH MAJORS AT FOREIGN LANGUAGE SPECIALIZED SCHOOL, ULIS, VNU

UPPER-(Đánh giá giáo trình “Solutions Upper-intermediate” dùng cho học sinh chuyên Anh lớp 10 tại trường THPT Chuyên Ngoại Ngữ, Đại học Ngoại Ngữ,

Đại học Quốc Gia Hà Nội)

M.A MINOR PROGRAMME THESIS

Field: English Teaching Methodology Code: 6014.0111

HANOI – 2016

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

NGUYỄN THI ̣ PHƯƠNG

AN EVALUATION OF THE COURSEBOOK “SOLUTIONS INTERMEDIATE” FOR TENTH-GRADE ENGLISH MAJORS AT FOREIGN LANGUAGE SPECIALIZED SCHOOL, ULIS, VNU

UPPER-(Đánh giá giáo trình “Solutions Upper-intermediate” dùng cho học sinh chuyên Anh lớp 10 tại trường THPT Chuyên Ngoại Ngữ, Đại học Ngoại Ngữ,

Đại học Quốc Gia Hà Nội)

M.A MINOR PROGRAMME THESIS

Field: English Teaching Methodology Code: 6014.0111

Supervisor: Prof Nguyễn Hòa

HANOI - 2016

CANDIDATE'S STATEMENT

I, Nguyen Thi Phuong, hereby certify that the thesis entitled:

An evaluation of the coursebook “Solutions Upper-intermediate” for grade English majors at Foreign Language Specialized School, ULIS, VNU

tenth-is the results of my research for the degree of Master of Arts at University of Languages and International Studies, Vietnam National University, Hanoi and the thesis has not been submitted for any degrees at any other universities or tertiary institutions

Hanoi, 2016

Nguyễn Thị Phương

ACKNOWLEDEGEMENT

A thesis costs every researcher a lot of time to complete and mine does, too However, I could not finalize this study with personal efforts and without the assistance of some enthusiastic people who I want to send my deep thanks to

The first one, I want to sincerely thank is Prof Nguyen Hoa He is my

supervisor His constructive comments helped me realize my mistakes and have the reasonable corrections In addition, his enthusiasm made each step in my research process easier and made me feel more comfortable and be under less stress caused

by the great amount of work Nothing can totally and exactly express my gratitude

to him I also acknowledge my gratitude to Dr Tran Thi Tuyet for her useful

lectures on materials evaluation and development She also gives me valuable instructions, comments, and her kindly encouragement during the development of this thesis

Additionally, I would like to express my appreciation to many teachers,

especially Tran Thi Lan, (M.A), who is a head of English Division at FLSS and

tenth-grade English majors at FLSS, ULIS, VNU for their cooperation as well as their help in the survey Without their support, the questionnaires and interviews which orientate the suggestions – the heart of my thesis – could not be completed as expected

Lastly, I want to thank my parents who gave me the life and stand by me on each step I take in this life The completion of my research could not be achieved without their spiritual supports

ABSTRACT

The purpose of this study is to gauge the suitability of a coursebook titled

“Solutions Upper-intermediate” for students‟ levels of English and general expectations in terms of the content and language skills Following that, the teaching-listening situation can be identified and unsuitable content will be adapted Quantitative and qualitative data are obtained through questionnaires with 70 students and semi-structured interviews with 4 teachers, in which survey questions are developed based on the students‟ general expectations and MOET‟s criteria Quantitative data in the questionnaires are calculated manually into rounded percentages to answer two research questions Qualitative data from “Comments” in the questionnaires with the students and from interviewed teachers are used to contribute more information to the answers to the research questions

The results reveal that the coursebook is partly appropriate to students‟ levels and expectations Firstly, it could be affirmed that this coursebook partly suits their levels Reading passages are fully appreciated by the respondents Speaking, writing, and vocabulary relatively match students‟ levels In contrast, listening skills and grammar are unsuitable for them It is recommended to find more difficult listening passages and grammar for the students Although vocabulary is somewhat suitable for them, it is better for several lessons with many technical terms or complex words to have more guidance and activities Secondly, as regards the appropriateness for the students‟ expectations, in general, the survey results indicate that the coursebook meets their expectations in terms of the content and language skills to a certain extent The topics are various and impressive but it is advisable that more contemporary and compelling topics should be added Reading and writing are two parts which are appreciated by both the students and the teachers Listening and speaking are unable to meet the expectations of the students, which means more activities and tasks on developing these skills should be added The language content including vocabulary and grammar are generally suitable

TABLE OF CONTENTS

CANDIDATE'S STATEMENT i

ABSTRACT iii

LIST OF FIGURES AND TABLES viii

CHAPTER 1: INTRODUCTION 1

1.1 Rationale 1

1.2 Aims and objectives 2

1.3 Significance 2

1.4 Scope 2

1.5 Methodology 3

1.6 Organization of the thesis 3

CHAPTER 2: LITERATURE REVIEW 4

2.1 Definition of coursebook 4

2.2 Definition of evaluation 5

2.3 Definition of materials evaluation 6

2.4 Types of materials evaluation 7

2.4.1 Pre-use evaluation 9

2.4.2 In-use evaluation or whilst-use evaluation 9

2.4.3 Post-use evaluation 9

2.5 Purposes of materials evaluation 10

2.6 Methods of textbook evaluation 11

2.6.1 The impressionistic method 11

2.6.2 The checklist method 11

2.6.3 The in-depth method 12

2.7 Criteria for materials evaluation 13

2.8 Previous studies on materials evaluation 15

2.9 Summary 17

CHAPTER 3: METHODOLODY 18

3.1 THE CONTEXT OF THE STUDY 18

3.1.1 The course 18

3.1.2 The coursebook “Solutions Upper-intermediate” – Student‟s book 19

3.1.3 The students 21

3.1.4 The teachers 22

3.2 METHODOLOGY 22

3.2.1 Research methods 22

3.2.2 Participants 22

3.2.3 Data collection instruments 23

3.2.4 Procedure of data collection 27

3.2.5 Data analysis procedure and presentation of data 28

3.2.6 Summary 28

CHAPTER 4: RESULTS AND DISCUSSION 30

4.1 Findings for the first research question 30

4.2 Findings for the second research question 32

4.2.1 The appropriateness of general content to students‟ expectations 32

4.2.2 The appropriateness of listening skills to students‟ expectations 35

4.2.3 The appropriateness of speaking skills to students‟ expectations 36

4.2.4 The appropriateness of reading skills to students‟ expectations 37

4.2.5 The appropriateness of writing skills to students‟ expectations 38

4.2.6 The appropriateness of language content to students‟ expectations 39

4.3 Summary 40

CHAPTER 5: CONCLUSION 41

5.1 Conclusion 41

5.2 Recommendations 42

5.3 Limitations 43

5.4 Suggestions for further studies 43

REFERENCES 44 APPENDICES I

APPENDIX 1 II APPENDIX 2 VIII APPENDIX 3 XIII APPENDIX 4 XVIII APPENDIX 5 XIX

LIST OF ABBREVIATIONS

ELT: English Language Teaching

FLSS: Foreign Language Specialized School

MOET: Ministry of Education and Training

ULIS: University of Languages and International Studies

VNU: Vietnam National University

LIST OF FIGURES AND TABLES

Figure 1: Four basic components of evaluation 5 Figure 2: Macro and micro-evaluation in language teaching 8

Table 1: Adaption from MOET‟s checklist 25 Table 2: Students‟ evaluation on the appropriateness of the content and language

skills to their levels 31

Table 3: Students‟ evaluation on the appropriateness of general content to their

CHAPTER 1: INTRODUCTION

1.1 Rationale

In Vietnamese high schools, textbooks or coursebooks are an indispensable part in English teaching and learning For non-specialized high schools, they have used the standard textbooks set by Ministry of Education and Training (MOET)

Nevertheless, in addition to the use of piloted textbooks, other schools like

specialized high schools have selected coursebooks for their own students With the current numerous number of books, the decision to choose one coursebook cannot

be taken lightly and whether it is acceptable or not is one challenging question Hence, teachers need to be knowledgeable enough to opt for the most suitable ones for their students through evaluating these materials based on the guidelines and criteria Foreign Language Specialized School (FLSS) is a specific instance which has chosen the coursebooks for their own students in learning English Teachers at English Division held a meeting and came to the conclusion that they selected

“Solutions Upper-intermediate” as the core material for tenth-grade English major students, and this material has been used for three years A question that plagues the researcher is just whether this coursebook suits the students or not Nevertheless, there have been no strict criteria established or research on evaluating this coursebook carried out so far

For that reason, the current research is undertaken to gauge this coursebook and it specifically seeks to see how far the main factors - the content and language skills suit the students‟ levels of English and general expectations of the coursebook In

other words, the researcher will conduct a study titled An evaluation of the

coursebook “Solutions Upper-intermediate” for tenth-grade English majors at FLSS, ULIS, VNU This research will acquaint teachers and administrators at FLSS

with a general picture about the content and language skills of this material in the eyes of their students after collecting and analyzing data Following that, the impetus for improvements which suit students‟ levels and expectations is evident

1.2 Aims and objectives

Aims

By doing this study, the author wishes to find out students‟ and teachers‟ evaluation

on the coursebook with the focus on the suitability of the content and language skills for the students‟ language levels and general expectations From that, the learning and teaching situation will be identified and some recommendations about adaptation appropriate to the learners will be made, which aims at improving the quality of language teaching at FLSS

Objectives

The achievable goal of the study is to answer the following research questions:

“To what extent does the coursebook suit tenth-grade English majors‟ levels in terms of the content and language skills?”

“To what extent does the coursebook match tenth-grade English majors‟ general expectations of the coursebook in terms of the content and language skills?”

1.4 Scope

Firstly, this paper will highlight the suitability of the content and language skills for students‟ levels and expectations of the coursebook Due to this, other criteria to evaluate the coursebook and other factors relating to learners will not be referred to

Secondly, the study is carried out with tenth-grade English majors at FLSS of the

2014-2015 school year and teachers who are teaching them Finally, the study is limited to judging the coursebook “Solutions Upper-intermediate” – Student‟s book

so that its companying teacher‟s book and workbook will not be mentioned

1.5 Methodology

In order to achieve the objectives of this thesis, qualitative and quantitative research methods are employed Data are collected from the students through questionnaires and from the teachers through interviews The questionnaire is designed to gather data about the students‟ ideas on the intended criteria Information gained from interviews with the teachers will be taken notes, analyzed and used for providing more information

1.6 Organization of the thesis

Chapter 1 “Introduction” is an overview of all main contents of this study

Chapter 2 “Literature review” presents a review of literature concentrating on the definitions of coursebook, evaluation, and materials evaluation After that, types of and purposes materials evaluation, textbook evaluation methods, and criteria for evaluating coursebook are shown Finally, some previous studies are mentioned Chapter 3 “Methodology” has the first part about the course, the coursebook, the students and the teachers at FLSS The second part includes participants, data collection instruments and data collection process and analysis

Chapter 4 “Results and discussion” reports the result of analysing data from collected questionnaires, interviews and discuss them

Chapter 5 “Conclusion” summarizes all the main issues mentioned in the study It consists of the conclusion of the research, recommendations based on the research, limitations of the research and suggestions for further studies

CHAPTER 2: LITERATURE REVIEW

This chapter clarifies key terms which are encountered in this thesis It aims at evaluating the coursebook “Solutions Upper-intermediate”; therefore, it would be reasonable that the theories of “coursebook”, and “evaluation” emerged first before the concept “materials evaluation” After that, purposes, types and methods of materials evaluation will be referred Next, “Criteria” includes the criteria have been suggested and what will be used in this thesis The last part is “Previous studies”

2.2 Definition of evaluation

In an educational setting, evaluation is “a process of collecting, analyzing and interpreting information about teaching and learning so as to make informed decisions that enhance student achievement and the success of educational programs” (Rea-Dickins & Germaine, 1993; Genesee & Upshur, 1996; O‟Malley & Valdez-Pierce, 1996, as cited in Carter & Nunan, 2001, p 144) In order to make the definition of evaluation clearer, three following characteristics are given by Genesee (as cited in Carter & Nunan, 2001, p 144) He asserts that evaluation can concentrate on various aspects of teaching and learning What is more, evaluation can be made for different reasons and the reasons impact in substantial ways Finally, evaluation consists of four components:

Figure 1: Four basic components of evaluation

Articulate purposes for evaluation

Identify and collect relevant information

Make decisions

Analyse and interpret information

From the figure above, the purpose of the evaluation is first mentioned After that, the relevant information is identified and collected Next, the collected information

is analysed and interpreted before any decisions are made

One of examples which Genesee (as cited in Carter & Nunan, 2001) gave is as follows:

The English Language Institute at Central University, South Africa offers courses in oral and written English for business purposes to adult non-native speakers of English whose employers want to transfer them to international operations They have designed an evaluation to decide whether to continue using them in the coming year Questionnaires will

be used to collect feedback from the students, their teachers and their employers (p 144)

This evaluation is comprised of four main components, including the purpose of the evaluation to decide whether to continue using courses in oral and written English in the coming year, relevant information (feedback from students, teachers and employers), the interpretation of the feedback, and decisions made on whether the materials should be kept or rejected

In this thesis, the evaluation also consists of four parts as follows:

 The purpose of the evaluation to identify the current situation and to have suitable changes in the coursebook,

 Relevant information (feedback from students and teachers),

 The interpretation of the feedback,

 Decisions made about what can be adapted

2.3 Definition of materials evaluation

Materials evaluation has become such a crucial activity that has attracted the attention of several linguistic researchers with various definitions

“Evaluation is a matter of judging the fitness of something for a particular purpose” (Hutchinson & Waters, 1987, p 96) The authors does not consider the evaluation

as an activity to find out what is good or bad of the material, just the suitability of something for a purpose Furthermore, making materials evaluation is considered as

“a procedure that involves measuring the value (or potential value) of a set of learning material It involves making judgements about the effect of the materials

on the people using them” (Tomlinson, 2003, p 15) This process measures, for instance, the appeal of the materials to the learners, the ability of the materials to interest the learners and the teachers and so forth (Tomlinson & Masuhara, 2004) Tomlinson (2003) also indicates that there is a distinction between an evaluation and an analysis He explains that an evaluation concentrates on the users of the materials and an evaluator will make judgements about their effect on the users This process is subjective An analysis focuses on the materials only and this way is objective Littlejohn (1998, as cited in Tomlinson, 2003) also agrees with this separation

While Hutchinson, Waters and Tomlinson just mention what materials evaluation is, Brown (1995) adds the benefits of this process in his definition Textbook evaluation is the collection and analysis of all relevant information in a systematic way necessary to improve a curriculum and assess its effectiveness within a certain context

From the above definitions, this research highlights some following aspects:

 Judging the fitness of the content and language skills of the coursebook for the students‟ levels and expectations,

 Making judgements about the effect of the material on the users (tenth-grade English majors),

 Promoting the improvement of the English coursebook at FLSS,

 Evaluating within the specific context which is at FLSS

2.4 Types of materials evaluation

Two ways of categorization suggested by Ellis (as cited in Tomlinson, 1998), Tomlinson (2003), McGrath (2002) and Cunningsworth (1995) are mentioned in the existing literature The first way, according to Ellis, is based on the aspects of evaluation whereas in the second one, Tomlinson, McGrath and Cunningsworth divide the process into 3 types built on 3 stages of evaluation These two methods of classification complement each other to help the evaluator have a full view of materials evaluation

Firstly, Ellis divides materials evaluation into macro-evaluation and evaluation As shown in the article “The evaluation of communicative tasks”, they have some differences as follows:

micro-Marco-evaluation can be defined as evaluation that seeks to answer one or both of the following questions:

1 To what extent was the program/project effective and efficient in meeting its goals?

2 In what way can the program/project be improved?

(As cited in Tomlinson, 1998, p 218)

He claims that the implementation of a macro-evaluation of a program or project involves collecting various kinds of information relating to administrative matters and/or curriculum matters (materials, teachers, and learners)

Micro-evaluation, according to Ellis (as cited in Tomlinson, 1998, p 219), “is characterized by a narrow-focus on some specific aspects of the curriculum or the administration of the program.” As can be seen from the figure, timetable evaluation, task evaluation, evaluation of questioning practices, and evaluation of levels of participants are examples of this type

Program/project evaluation

Administrative matters Curricular matters

Materials evaluation

Teacher evaluation

Learner evaluation

task evaluation, etc

evaluation of questioning practices, etc

evaluation of levels of participation, etc

timetable evaluation , etc

The second way mentioned by Tomlinson (2003), McGrath (2002) and Cunningsworth (1995) shows that there are three types of materials evaluation, including pre-use, in-use or whilst-use, and post-use evaluation This means that the evaluation can take place before a coursebook is used, during its use and after its use respectively

2.4.1 Pre-use evaluation

As shown by Tomlinson (2003), evaluators predict about the potential value of language teaching materials for their users It can be context-free, context-influenced or context-dependent Pre-used evaluation is often impressionistic and a teacher can gain a quick impression of its potential value Nonetheless, in order to reduce subjectivity and make the evaluation more principled, evaluators can establish an evaluation criterion As commented by Cunningsworth (1995), this is the most challenging kind of evaluation The reason is that teachers and learners have not been working through the coursebook for some time Thus, they cannot make a good evaluation of that book

2.4.2 In-use evaluation or whilst-use evaluation

According to Tomlinson (2003, p 24), in-use evaluation means “measuring the value of materials while using them or while observing them being used”; therefore, this process is less subjective than pre-use evaluation However, it is a fact that people only can measure what is observable

Tomlinson includes what can be measured in an in-use evaluation as follows: clarity

of instructions, clarity of layout, comprehensibility of texts, credibility of tasks, achievability of tasks, achievement of performance objectives, potential for localization, practicality of the materials, teachability of the materials, flexibility of the materials, appeal of the materials, motivating power of the materials, impact of the materials, and effectiveness in facilitating short-term learning

2.4.3 Post-use evaluation

Post-use evaluation is considered by Tomlinson (2003, p 25) as “the most valuable type of evaluation as it can measure the actual effects of the materials on the users”

He points out that this process can answer some huge questions below:

 What do the learners know which they did not know before starting to use the materials?

 What do the learners still not know despite using the materials?

 What can the learners do which they could not do before starting to use the materials?

 What can the learners still not do despite using the materials?

 To what extent have the materials prepared the learners for their examinations? (p 25)

He also indicates methods of measuring the post-use effects of materials including:

 tests of what has been „taught‟ by the materials;

 tests of what the students can do;

 examinations;

 interview;

 questionnaires;

 criterion – referenced evaluations by the users;

 post – course diaries;

 post – course „shadowing‟ of the learners;

 post – course reports on the learners by employers, subject tutors, etc (p 25)

The main problem is that it takes time and expertise to measure post-use effects reliably However, as shown by Cunningsworth, this type of evaluation can be useful for identifying strengths or weaknesses of the coursebook over a period of continuous use It can be useful in helping to decide whether to use the same coursebook in the future

This study belongs to micro-evaluation and the post-use evaluation method is used

to carry out the evaluation to collect useful information about real effects of the coursebook on the students

2.5 Purposes of materials evaluation

There are various reasons for materials evaluation

The first reason is to select a new suitable material, which is the frequent and major reason for evaluation as shown by Cunningsworth (1995), Hutchinson and Waters (1987) and Ellis (1997) Secondly, thanks to the materials evaluation, teachers can

be aware of their teaching and learning situation (Hutchinson & Waters, 1987) Finally, teachers can identify strengths and weaknesses of in-use coursebook in order that strong points can be made use of and weak points can be substituted or adapted (Cunningsworth, 1995) or decide whether to continue using the material or

to replace it with a better material (Ellis, 1997)

By virtue of the materials evaluation in this current research, teachers and administrators at FLSS can understand their teaching and learning situation Furthermore, suitable and unsuitable points to the students‟ levels and expectations will be identified and inappropriate ones will be substituted or adapted

2.6 Methods of textbook evaluation

Three methods for evaluating textbooks identified by McGrath (2002) include the impressionistic method, the checklist method, and the in-depth method

2.6.1 The impressionistic method

This method is “to obtain a general impression of the material” (McGrath, 2002, p 25) Cunningsworth (1995, p 1, as cited in McGrath, 2002, p 25) uses the term

“impressionistic overview” for this method which suggests that it is “wide-ranging but relatively superficial.” He shows that in the case of global textbooks, when using this method, evaluators will skim through the publisher‟s “blurb” (i.e the brief description of the book on the back cover), the content page, organization, layout, topics, and visuals Therefore, it is inadequate if it intends for textbook evaluation and selection

2.6.2 The checklist method

A checklist consists of “list of items which is referred to for comparison, identification or verification” (Collins English Dictionary, 1992, as cited in McGrath, 2002, p 26) The use of checklists for specific evaluation purposes has at least four following benefits:

1 It is systematic, ensuring that all elements that are deemed to be important are considered

2 It is cost effective, permitting a good deal of information to be recorded in a relatively short

space of time

3 The information is recorded in a convenient format, allowing for easy comparison between

competing sets of material

4 It is explicit, and, provided the categories are well understood by all involved in the

evaluation, offers a common framework for decision-making

(McGrath, 2002, p 27)

The systematicity of the checklist method ranks the first among all the advantages

It is well brought out by Skierso (1991, p 440, as cited in McGrath, 2002):

A textbook evaluation checklist should consist of a comprehensive set of criteria based on the basic linguistic, psychological, and pedagogical principles underlying modern methods

of language learning These criteria should be exhaustive enough to insure assessment of all characteristics of the textbook And they should be discrete enough to focus attention on one characteristic at a time or on a single group of related characteristics (Tucker, 1978, p 219)

However, this method also has its limitations Firstly, the systematicity is only strength if the criteria of which a checklist is composed are relevant to the specific context in which it is used Secondly, a checklist cannot be a statistic phenomenon

as indicated by Williams (1983, as cited in McGrath 2002, p 27) This means that the categories in the checklist are “a reflection of the time at which they were conceived and of the beliefs of their designer.”

2.6.3 The in-depth method

In-depth techniques are explained by McGrath (2002, p 27-28) as follows:

“go beneath the publisher‟s and author‟s claims to look at, for instance, the kind of language description, underlying assumptions about learning or values on which the materials are based on.”

In a broader sense, this method explores whether the materials are likely to live up

to the claims being made for them

McGrath argues that while such techniques ensure that the selection process is a more considered affair, they may also have certain disadvantages:

1 Representativeness of samples: the samples (e.g exercises, lessons, units) selected

for analysis may not be representative of the book as a whole, and this may therefore distort any judgment

2 Partiality: because in-depth analysis is normally narrowly focused (being based

either on a particular section of the material or one or more threads running through it)

It gives only a partial insight into what the material offers

3 Time and expertise required: some proposals for in-depth evaluation would

involve a good deal of time; others require expert knowledge (e.g of language description) that is not available Though it can be argued that the time spent on evaluation is well spent if a potentially unsuitable textbook is rejected, there may be more economical ways of arriving at this decision (p 28)

In this thesis, the checklist method is chosen for evaluating the coursebook

“Solutions Upper-intermediate” because it is cost effective and convenient for the researcher Additionally, the advantages of this method when applied in this research paper overweigh its disadvantages Two potential limitations are not true in the case of this research This study only evaluates the coursebook used for tenth-grade English major students at FLSS in the 2014-2015 school year; therefore, the criteria in the checklist are relevant to the context in which it is used The categories

in the checklist reflect the time at which the study is undertaken, which is obvious

2.7 Criteria for materials evaluation

A set of criteria for evaluation is a basis to “reach a decision regarding what to be evaluated” (Tomlinson, 1998, p 220) This means they are the foundation on which evaluators base when making judgments Accordingly, one of the most crucial steps that evaluators should spend time on is determining evaluation criteria

According to Dudley-Evans and John (2007), criteria for materials evaluation depend on what is being evaluated and the reason why they need to be evaluated A number of experts such as Cunningsworth (1995), Hutchison and Waters (1987), Tomlinson (2003) and MOET (2015) have tried to establish practical checklists on textbook evaluation

Cunningsworth (1995) suggests a list of checklists covering seven areas: Aims and approaches; design and organization; language content; skills; topic; methodology; teachers‟ books and practical considerations

Tomlinson (2003) agrees that material evaluation is a procedure to measure some or all of the aspects such as: appeal to learners; credibility to learners, teachers and administrators; validity; reliability; ability to interest learners; ability to motivate learners; short-term and long-term learning value; learners‟ and teachers‟ perception

of the value of the material; assistance given to teachers in terms of preparation, delivery and assessment, and so on

MOET (2015) suggests a checklist (see appendix 1) for the process of evaluating materials, which is composed of 45 criteria under 4 main headings:

 Objectives, teaching principles and methodology (from item 1 to 10): focus on the aims of the coursebook corresponding with the aims of the teaching program and the methods of teaching

 Design and organization (from item 11 to 18): give an overview of the organizational foundation of the coursebook through its cover, introduction and table of contents statements, visuals, book size and weight

 Content and language skills (from item 19 to 42): concentrate on topics, content, four language skills (listening, speaking, reading, and writing), the language content (vocabulary, grammar, and pronunciation)

 Textbook user supports (from item 43 to 45): mean evaluating the supplementary materials

To evaluate the coursebook “Solutions Upper-intermediate”, the author is going to apply the criteria adapted from a tentative checklist for textbook evaluation of MOET (2015) The checklist is undeniably appreciated by the evaluator in this thesis because of its clear-cut and coherent criteria and its suitability in the Vietnamese context As shown by Tomlinson (2003), coursebook criteria are emphatically local and no one is really certain what criteria can be applied in ELT contexts worldwide In detail, this study focuses on the evaluation on general

learning-teaching content, language skills (Listening, speaking, reading and writing) and language content (grammar and vocabulary) Pronunciation is not evaluated because it is not covered in this coursebook

2.8 Previous studies on materials evaluation

The author would like to mainly mention several of works which investigate the same field with this thesis - materials evaluation

First of all, the author Isil Cakit (2006) evaluated the coursebook “New Bridge to Success 3” from the perspectives of the teachers and students This study was conducted at four high schools, three of which are Super Lycees and one of which is

an Anatolian Fine Arts in the province of Mersin, Turkey The evaluation of the textbook was conducted at macro level on the basis of eleven criteria In the review

of literature, the author mentioned roles of textbooks, approaches to materials evaluation in ELT, models for the evaluation of materials, and criteria for materials evaluation Both quantitative and qualitative data were obtained through student questionnaires administered to 336 students and interviews with 8 teachers The results revealed that both teachers and students felt negative about the most of the characteristics of the textbook The majority of the students and all the teachers mentioned that the level of the textbook was not appropriate for the particular age group It also indicated that the materials failed to consider learning style preferences of the visual, auditory, and kinesthetic learner On the other hand, one

of the strengths of the textbook was up-to-date and helpful for the students to understand the lessons

The material “Lifelines” for the first-year non-English major students was appraised

by Tran Thi Chung Oanh (2010) at Hai Phong University The author adopted the criteria of Hutchison and Waters (1987) and employed questionnaires and informal interviews as the research instruments Major findings of the study revealed that the material‟s content and methodology suited the students‟ requirements but there was some mis-match in time allocation Finally, the suggestions for the material

improvements on time proportion for each unit and time allocated for the course were made

Next, in her research, Tran Thi Thu Trang (2011) evaluated an intermediate textbook titled “English Written Proficiency” which is an instructional in-house material for freshmen in their first term at Faculty of Foreign Languages – Thainguyen University from the perspectives of teachers and students The author mainly used five criteria given by Hutchinson and Waters as criteria for materials evaluation The results revealed that both teachers and students felt positive about most of the characteristics of the textbook However, there were some gaps that need to be fulfilled in terms of the layout and organization of the book, some aspects in its content and methodology so that the book would bring the most benefits to those who were using it

Le Thi Bac (2013) conducted a study on the piloted coursebook “English 6” to find out whether the book was suitable for the requirements of the 2020 Project as well

as the Piloted English Language Curriculum for lower secondary education in the context of teaching and learning this book in some lower secondary schools in Hai Duong province A survey questionnaire was designed based on a combination of evaluation checklists and criteria suggested by different authors and informal talks with the respondents are utilized as the research instruments of the study The results and analysis of the textbook indicated that the book was suitable enough to

be used in the teaching and learning situations in some lower secondary schools in Hai Duong province In response to the research questions, it was also demonstrated that the book was compatible with the requirements of the National Foreign Language 2020 Project and the Piloted English Language Curriculum

All things considered, although there is a similarity between the above studies and this thesis about the materials evaluation, they differ in the content of literature review, the criteria and aspects for evaluation Firstly, in the review of literature, some studies do not mention the definition of evaluation and methods of textbook evaluation Secondly, all of the authors above mentioned use one single checklist or

a combination of two or more checklists of criteria, depending on their specific purposes In Post Graduate Studies of ULIS, one of the common points in these studies is that they all used Hutchinson and Waters‟ criteria (1987) to evaluate materials Nonetheless, in the present study, criteria by MOET will be modified to suit the specific situation Finally, numerous research papers are conducted to evaluate all the aspects of a coursebook while some focus on several specific aspects such as content and methodology In this research, only the content and language skills are concentrated on What is more, this study stress two factors associated with learners - levels and general expectations of the coursebook

“material”, “textbook” are alternative names of “coursebook.” Among types of materials evaluation, the post-use evaluation will be employed in this research The understanding of the teaching-learning situation and identification of suitable and unsuitable points of the coursebook to the students‟ levels and expectations in order

to have appropriate changes are two purposes of materials evaluation in this thesis

Of three methods used for materials evaluation namely the impressionistic, the checklist, and the in-depth method, the checklist method would be adopted in this study Many criteria suggested by different authors notwithstanding, the criteria in the checklist by MOET are chosen and adapted

CHAPTER 3: METHODOLODY

As materials can only be meaningfully evaluated in relation to their intended teaching situation (Richards, 2001) Thus, before discussing the methodology, the researcher mentions the situation in which the material will be used In the first part, the current situation at FLSS relating to the study will be referred to and then the research methodology will be treated Information about the course, the overview of the coursebook, the characteristics of learners and teachers is addressed in the first part The second part about the methodology employed in this study includes research methods, participants, data collection instruments, data collection procedures and data analysis procedure and presentation of data

3.1 THE CONTEXT OF THE STUDY

Before referring to information about the course for ten-grade English majors, the author would like to give an overview about FLSS This school is a reputable high school for gifted students studying foreign languages This school, which was established in 1968, is a member of ULIS, VNU

3.1.1 The course

As regards language teaching materials in this school, apart from English 10 (Advanced) provided by MOET, “Solutions” is chosen by teachers at English Division as the core coursebook for all English-major and non-English major students, which means the eleventh form students and the twelfth form students also use this book but there is a difference in the level of the coursebook

The objective of English course for tenth grade English majors is to help them reach B2 level According to Common European Framework of Reference for Languages

by Council of Europe (2001), at that time, students can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his or her field of specialisation They can interact with a degree of fluency and spontaneity that make regular interaction with native speakers quite possible without strain for either party They can produce clear-cut and detailed text on a

wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options

In order to become students of this school, they have to pass the stiff entrance exams of three subjects (Literature, Maths and English) with the mark of at least 24/30 In grade 10, there are 11 classes in total and each class has from 40 to 45 students However, there are only 5 English specialized classes (10A, 10B, 10H, 10I, 10K) When they learn English, each class is divided into two small classes to achieve the effectiveness in learning this foreign language The students have six 45-minute periods a week taught by Vietnamese teachers In their academic school year, the students have to take 10 tests (45-minute test and 90-minute test) in total and 2 end-of-term exams

3.1.2 The coursebook “Solutions Upper-intermediate” – Student’s book

Tim Falla and Paul A Davies are two authors of “Solutions Upper-intermediate”, published by Oxford University Press This coursebook belongs to one of five-level English courses for teenagers (Solutions Elementary, Pre-intermediate, Intermediate, Upper-intermediate and Advanced) This coursebook has been used for tenth-grade English majors at FLSS for 3 years It supports students through its straightforward layout and clear-cut presentation There are 10 topic-based units in this 151-page long book

Each teaching unit covers a general topic and is structured into 7 sections: A Vocabulary and Listening, B Grammar, C Culture, D Reading, E Grammar, F Speaking, G Writing analysis and Writing task

In each section, there is a specific topic For example, Unit 1 has the general topic is

“Against the odds” and the specific topic of part A is “Talking about people.”

Part A “Vocabulary and Listening” is the beginning section in this unit Although this part is about vocabulary and listening, there are also speaking activities for students This is true for other parts like “Grammar”, “Reading”, and “Writing” The course develops language and skills so that speaking and listening becomes frequent and familiar Part A introduces the topic of the unit and the main

vocabulary Students practice new words through listening and other activities Students can consolidate their words in “Vocabulary Builder” at the end of the book Following this, it is part B “Grammar” which contains the first grammar point of the unit and exercises for students to practice There are the boxes called “Learn this” and “Look out” This lesson connects to the “Grammar Builder” at the back of the book providing more practice and grammar reference

Part C “Culture” is closely related to people and cultures about Britain, the USA and other English-speaking countries However, this part is optional in the teaching program for the students at FLSS

Part D “Reading” contains the reading texts of the unit The texts link to the topic of the unit and recycle the main grammar points of the unit New words are highlighted in color and practiced in a follow-up activity and in the workbook

Part E mentions the second grammar point of the unit “Learn this!” boxes provide key information in a clear way The speaking activity in this part helps students personalize the new language

In part F “Speaking”, students have chances to listen to draw more ideas, new words, and understanding There are functional phrases for students to practice Part G “Writing analysis and Writing task” concentrates on writing and involves the typical type required for exams In this part, learners have also chances to listen to get more ideas, new words, and understanding Each unit has two writing lessons, including “Writing analysis and Writing task” Sample texts and a “Check your work” part are included in this part

“Writing analysis” provides a sample and exercises for students to do on this

“Writing task” arranges phrases for students to write

There are “Get ready for your exam” lessons after each unit which prepares students for exams These lessons include exam tasks for reading, speaking, listening and use

of English After every two units, there is a review about vocabulary and grammar, which is aimed at consolidating students‟ language knowledge called “Language Review” Students have opportunities to do many exercises about new words and

grammar points which they have learned in these two units Additionally, after every two units, there is a part namely “Skills round-up” which is a review part about 4 skills such as Listening, Speaking, Reading and Writing

At the end of this coursebook, there are a grammar builder and references which provide grammar points and exercises In addition, there is vocabulary builder section, from which students get masses of practice There are wordlist and communicative activities in which wordlist includes new words appearing in each Unit and communicative activities complement activities for speaking in some units

3.1.3 The students

Ten-grade English majors at FLSS are specialized students who are 15 and 16 year old teenagers According to the results of the first small survey, here are their expectations of the coursebook in general:

 Helping them broaden their horizons,

 Studying interesting lessons,

 Opportunities to accumulate vocabulary and enhance language skills with various activities and useful guidance

Moreover, three pieces of learners‟ information about the duration of learning English, levels and reasons for learning English are shown in the questionnaires In regard to the duration of learning English, most of the students have been learning English since they were 1st and 3rd form students, which means that they have experienced English about from 8 to 10 years There are also small numbers of students who have learned English for 5 years Secondly, their levels at the end of the course are at least B2 Lastly, they enter this school with myriad reasons and hopes in the future such as:

 To meet their interests in learning English,

 To communicate with native and non-native speakers,

 To get high marks in tests such as TOEFL, IELTS or exams for gifted students,

 To study abroad,

 To have a dream job with handsome salary

3.1.4 The teachers

As for teachers, a group of seven people who are about from 25 to 30 years old are assigned to teach English to tenth-grade English majors As the information from the interviews, they have obtained Bachelor of Art degrees from ULIS and other prestigious universities All of these teachers are young, dynamic and enthusiastic Furthermore, they have experienced teaching English for many years They often deliver lectures in both Vietnamese and English but more in English

3.2 METHODOLOGY

3.2.1 Research methods

The study will utilize both quantitative and qualitative research methods together, which is the common approach with the combination of questionnaires and interviews The result of quantitative research into students‟ evaluation in the survey questionnaire can be quantified However, the statistics produced do not tell about their reasons, their explanations or their comments, hence qualitative research method is also used in the study Qualitative data will be obtained from

“Comments” in the questionnaires with the students and from interviews with the teachers

This research belongs to survey research because according to Brown (1988, p 3),

“survey studies focus on a group‟s attitudes, opinions, and characteristics.” He also shows that “the advantage of this type of research is that substantial amount of information can be collected in a relatively short time.” However, he also indicates the disadvantage that “the number of people who respond to a questionnaire, especially when it is mailed, is often low” In this thesis, questionnaires are distributed to tenth-grade English majors at their classes, not via emails, and the enthusiasm of the participants will help to overcome the disadvantage

3.2.2 Participants

In this study, the teachers and the students (tenth-grade English majors) at FLSS are the users of this coursebook For this reason, the researcher decided to get them

involved in the evaluation process as participants In other words, there are two groups of participants in the survey:

Group one includes the participants who are tenth-grade English-major students The sample size is 70 students from 5 classes (10A, 10B, 10H, 10I and 10K), accounting for approximately 33.3 % of the whole population (about 210 students) The sampling strategy adopted here is the systematic sampling According to Cohen, Manion and Morrison (2000), in order to make sure that the sample represents the whole population, the starting point is selected randomly More specifically, at FLSS in the 2014-2015 school year, there are 5 classes for English majors, including 10A, 10B, 10H, 10I and 10K It is a population of 210, a sample

of 70, and every third person can be selected The starting point for the selection is chosen at random This sampling strategy is used because it is simple and easy to apply and design

Another group consists of 4 teachers who are teaching English to the tenth form students As mentioned in the context of the study, there are 7 teachers in charge of teaching English to tenth-grade English majors However, due to convenience and time limitation, 4 teachers can take part in the survey All the teachers are qualified

in English teaching They are interviewed for providing further information

3.2.3 Data collection instruments

Questionnaire and semi-interview are used as two instruments for collecting data There are some reasons why they are chosen in this study

Firstly, the questionnaire, which is a relatively popular means of collecting data according to Nunan (1992), is appropriate for this paper This is because it allows the study to access a significant number of participants All the close and open-ended questions are to reply to the research questions and the use of Vietnamese and term explanations will help to clarify the questions As a result, possible misunderstandings leading to the inaccuracy of the outcomes might be avoided Secondly, this paper will hold semi-structured interviews for the teachers as an instrument in researching The interviews have a structured framework but the

participants have more flexibility They can change the order of questions or add more follow-up questions As pinpointed by Dowsett (1986, p 53), the semi-structured interview is “quite extraordinary – the interactions are incredibly rich and the data indicate that you can produce extraordinary evidence about life that you do not get in structured interviews or questionnaire methodology.” Mc Donough and

Mc Donough (1997) also have the same idea that this kind of interview allows

“richer interactions and more personalized responses” (p 184) Though the researcher cannot access more subjects for the interviews, detailed discussions with the teachers that are taken notes will provide more valuable data to answer the research questions

Finally, as shown in types of materials evaluation, according to Tomlinson (2003), questionnaires and interviews are two of among methods of measuring the post-use effects of materials

In this thesis, the questionnaire for students (see appendix 2 and 3) consists of two sections as follows

 Section one called “Learner‟s information” is open-ended questions designed to “allow the respondents to feel that they can contribute more individual points of view and more detailed information” (Mc Donough &

Mc Donough, 1997, p 176) In this thesis, this kind of question is used to obtain factual information on the learners: their ages, the duration of learning English, their language level, and their reasons for learning English This section aims at providing more information about the learners in the context

Donough and Mc Donough (1997), this kind of questions is used to “elicit opinions” (p 176) Moreover, “comments” are added at the end of each part

to get more ideas, explanations, or comments on what they ticked It is noted that before creating section two, a small survey about the students‟ general expectations of the coursebook was made in order to design the criteria What is more, the checklist is referred from MOET‟s one

Although the author bases on the checklist by MOET, there are some changes made, for example, editing or adding by the researcher The checklist for evaluating

“Solutions Upper-intermediate” includes 40 criteria, in which the author adapts as follows:

Criteria Adaptation from

MOET’s criteria

Criteria Adaptation from

MOET’s criteria

1, 2 Edited from criterion 19 19-21 Copied from criterion 31-33

3, 4 Edited from criterion 20 22 Added by the researcher

5, 6 Edited from criterion 21 23 Copied from criterion 34

7, 8 Edited from criterion 23 24 Edited from criterion 35

9 Copied from criterion 24 25 Edited from criterion 36

10 Copied from criterion 26 26 Added by the researcher

11 Copied from criterion 27 27 Copied from criterion 37

12 Added by the researcher 28 Copied from criterion 38 13-15 Edited from criterion 28 29, 30 Edited from criterion 39

16 Copied from criterion 29 31, 32 Edited from criterion 40

17 Added by the researcher 33-37,

40

Added by the researcher

18 Copied from criterion 30 38, 39 Edited from criterion 41

Table 1: Adaption from MOET’s checklist

More specifically, the checklist used for this thesis comprises 40 criteria under 6 main headings as follows:

 General content (from item 1 to 10): concentrates on evaluating whether the content in general is suitable for students‟ levels and general expectations or not

 Listening (from item 11 to 17): evaluates whether listening skills are suitable for students‟ levels and general expectations or not

 Speaking (from item 18 to 22): evaluates whether speaking skills are suitable for students‟ levels and general expectations or not

 Reading (from item 23 to 26): evaluates whether reading skills are suitable for students‟ levels and general expectations or not

 Writing (from item 27 to 30): means evaluating whether writing skills are suitable for students‟ levels and general expectations or not

 Language content (from item 31 to 40): focuses on evaluating whether vocabulary and grammar are suitable for students‟ levels and general expectations or not

The interviews for the teachers (see appendix 4) are designed regarding the following issues

 Section one “Teacher‟s information” is open-ended questions designed to obtain factual information on the teachers, including their age, their qualifications, and their years of experience in teaching English This section aims at providing more information about the teachers in the context

of the study

 Section two “Teacher‟s evaluation of the coursebook” with 10 ended questions is designed to gather more information about the teachers‟ evaluation on the appropriateness of the coursebook This part aims at providing more information for answers to the research questions Another aim is to get their suggestions in improving the textbook This section consists of 10 open-ended questions based on MOET‟s criteria which are related to teaching content as follows:

open-Question 1 is to ask about the suitability of the coursebook for students‟ levels

Questions 2-9 are about evaluation the suitability of the coursebook for students‟ expectations, which are based on the checklist in the questionnaires for the students Question 2, 3, 4, 5, 6, 7, 8, and 9 are designed based on the criteria 7, 8, 11, 18, 23, 27, 34, and 39 in the questionnaire respectively Question 10 is to ask them about suggestions for any changes to improve the coursebook

3.2.4 Procedure of data collection

The procedures of pooling the data were conducted in the middle of May, 2015 that was at the end of the 2014-2015 school year as follows:

The first stage was the preparation for the data collection process As mentioned in the instruments, a small survey was undertaken to generalize the students‟ expectations of the coursebook which created the scaffold for designing the evaluation checklist After that, the questionnaire and interview schedules were designed for two main groups of participants, namely the teachers and the students Secondly, the researcher spent 3 days distributing the survey questionnaires in Vietnamese to 70 tenth-grade English majors Their personal information was ensured to be kept confidential Before completing the list of questions, the students were explained about the research topic and the purpose so as to ensure the exactitude of results In the end, the researcher thanked the respondents for spending their time It took the students about 25 minutes to complete the questionnaires All students fully answered the question, which means a response rate of 100% This result was achieved thanks to the teachers‟ help and the students‟ enthusiasm at FLSS

In the third stage, the results in the questionnaire were processed according to two research questions Once the results were obtained, it was easier to carry out the interviews to explore more information from the participants Interviews were conducted among 4 teachers

For interview of teachers, appointments were made The interviewees were contacted in advance and explained the reason for the research and the aims at interviewing them The researcher met each interviewee face-to-face at the staffroom or classroom during break time Each interview lasted about 20 minutes The language used in the interviews was English The interviews began with the introduction of the interviewer and the research topic In the interview, the information was permitted to take notes of (see appendix 5) There was the exchange of questions between the interviewee and the interviewer The interviews ended with the thanks from the researcher Those interviews were opportunities to probe the teachers‟ mind for more information

3.2.5 Data analysis procedure and presentation of data

In the first place, the results collected from the questionnaire will be categorized and analyzed according to the research questions

 Criteria 12, 16, 22, 25, 26, 30, 33, 35, 37 and 40 are about evaluating whether the content and language skills are suitable for the levels of the students which answer the first research question

 The other criteria in the checklist will answer the second research question

The data will be then transferred manually in numerical forms and displayed in tables The figures were changed into rounded percentages by an electronic calculator Secondly, information gained from “comments” in questionnaires and semi-interviews will be used together with the data from the questionnaires to give a comprehensive picture of the material Participants‟ answers will be summarized and cited in the chapter “Results and discussion” Equally important, information gained will be used for reference in making recommendations to improve the quality of the material in “Recommendations” of the last chapter “Conclusion”

3.2.6 Summary

The focus of this chapter is on the context in which the study is undertaken and the methodology used The objectives of the course, the general content of the

coursebook “Solutions Upper-intermediate”, the learners and the teachers are mentioned This study is a survey research with the combination of two instruments – questionnaires with 70 students and interviews with 4 teachers The data in checklist of the questionnaire will be then transferred manually in numerical forms and displayed in tables Information gained from questionnaires and interviews will

be added to support the interpretation