The empirical evaluation of the coursebook “english file beginner student’s book – third edition” for non english major students at a preparatory school in viet tri

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-GRADUATE STUDIES ********** VŨ THỊ HỒNG LUYẾN THE EMPIRICAL EVALUATION OF THE COURS

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

**********

VŨ THỊ HỒNG LUYẾN

THE EMPIRICAL EVALUATION OF THE COURSEBOOK

“ENGLISH FILE BEGINNER STUDENT’S BOOK

– THIRD EDITION” FOR NON-ENGLISH MAJOR STUDENTS

AT A PREPARATORY SCHOOL IN VIET TRI

(Đánh giá thực nghiệm giáo trình “English File Beginner Student’s Book

- Third Edition” dùng cho học sinh Tiếng Anh không chuyên

tại một trường dự bị đại học ở Việt Trì)

M.A MINOR PROGRAM THESIS

Field : English Teaching Methodology Code : 8140231.01

Hanoi, 2018

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

**********

VŨ THỊ HỒNG LUYẾN

THE EMPIRICAL EVALUATION OF THE COURSEBOOK

“ENGLISH FILE BEGINNER STUDENT’S BOOK

– THIRD EDITION” FOR NON-ENGLISH MAJOR STUDENTS

AT A PREPARATORY SCHOOL IN VIET TRI

(Đánh giá thực nghiệm giáo trình “English File Beginner Student’s Book

- Third Edition” dùng cho học sinh Tiếng Anh không chuyên

tại một trường dự bị đại học ở Việt Trì)

M.A MINOR PROGRAM THESIS

Field : English Teaching Methodology Code : 8140231.01

Supervisor : Assoc Prof Dr Ho Ngoc Trung

Hanoi, 2018

DECLARATION

I certify that the work contained in this thesis is the result of my own research, and this thesis has not been submitted for any degrees at any other universities or institutions

ACKNOWLEDGEMENTS

First and foremost, I would like to express my deepest gratitude to my supervisor, Assoc Prof Dr Ho Ngoc Trung for his useful comments and continuous support of my study, for his patience, encouragement, enthusiasm and immense knowledge His guidance helped me tremendously in all the time

of research and writing of this thesis

I would also like to thank my school leaders and colleagues for their support and sharing the workload so that I could wholeheartedly focus on the study and get it completed

Finally, I would like to express my profound gratitude to my parents, my spouse, and to all of my friends for providing me with unfailing support and continuous encouragement throughout my years of study and through the process of researching and writing this thesis

This accomplishment would not have been possible without all of them Thank you!

Hanoi, November 2018

Signature

Vu Thi Hong Luyen

ABSTRACT

The present study evaluates the coursebook “English File Beginner Student’s Book – Third Edition” by Christina Latham-Koenig, Clive Oxenden (Oxford University Press, 2014) for non-English major students at a preparatory school in Viet Tri in terms its suitability to the objectives of the course, students’ needs and current teaching and learning method 150 non-English major students and 8 teachers who were using this coursebook were invited to answer the questionnaires, and two groups of three or four students were chosen randomly together with 8 teachers to take part in the interviews Data were analyzed both quantitatively and qualitatively Findings revealed that the coursebook fundamentally met the course’s objectives and students’ needs, and fit in with the current teaching and learning method, though there were still minor unsuitable parts that need adaptation The result of this study would benefit both teachers and learners who work with this material for better use in the future

Key words: coursebook, textbook, material, coursebook evaluation,

empirical evaluation, English File Beginner Students’ Book, non-English major

LIST OF TABLES

Table 4.1: Teacher’s assessment of students’ English level at the

beginning of the course 30 Table 4.2: Learners’ purposes of learning English 31 Table 4.4: Students’ ability to communicate in English about the topics in

the coursebook 35 Table 4.5: Students’ and teachers’ evaluation of the language elements in

the coursebook 40 Table 4.6: Students’ and teachers’ evaluation of the skills in the

coursebook 41 Table 4.7: Students’ suggestions on the proportion of the skills 42 Table 4.8: Teachers’ suggestions on the proportion of the skills 42 Table 4.9: Effectiveness of the teaching and learning interactions used

with the coursebook 46 Table 4.10: The frequency of the teaching aids used with the coursebook 47 Table 4.11: Students’ and teachers’ evaluation on the methodology of the

coursebook 48

LIST OF CHARTS

Chart 4.1: Students’ age 28 Chart 4.2: Students’ background of learning English 29 Chart 4.3: Students’ self-assessment of their English level at the beginning

of the course 30 Chart 4.5: Students’ progress after the course 36 Chart 4.6: Students’ and teachers’ evaluation of the topics of the

coursebook 39 Chart 4.7: The appropriateness of the coursebook with students’ learning

purposes 43 Chart 4.8: Students’ and teachers’ overall rating of the coursebook 51 Chart 4.9: Students’ and teachers’ judgement on the time allocation for each

unit 52

TABLE OF CONTENTS

DECLARATION i

ACKNOWLEDGEMENTS ii

ABSTRACT iii

LIST OF TABLES iv

LIST OF CHARTS v

CHAPTER 1: INTRODUCTION 1

1.1 Rationale 1

1.2 Aims of the study 2

1.3 Scope of the study 3

1.4 Research questions 3

1.5 Significance of the study 4

1.6 Structure of the study 4

CHAPTER 2: LITERATURE REVIEW 6

2.1 Text book, coursebook and material 6

2.1.1 Definitions 6

2.1.2 Roles of materials in ELT classrooms 7

2.2 Coursebook evaluation 9

2.2.1 Definitions of coursebook evaluation 9

2.2.2 Approaches to coursebook evaluation 9

2.2.3 Criteria for coursebook evaluation 14

2.3 Needs analysis 17

2.4 Previous studies on coursebook evaluation 19

CHAPTER 3: RESEARCH METHODOLOGY 22

3.1 The setting 22

3.1.1 English teaching and learning context 22

3.1.2 Course material 23

3.1.3 Course objectives 23

3.1.4 Course assessment 24

3.2 Research methodology 24

3.2.1 Participants 24

3.2.2 Research instruments 25

3.2.3 Data collection procedure 26

3.2.4 Data analysis methods 27

CHAPTER 4: FINDINGS AND DISCUSSION 28

4.1 About the learners 28

4.1.1 Learners’ English learning background 28

4.1.2 Learners’ English level 29

4.1.3 Learners’ purposes of learning English 31

4.1.4 Learners’ favorite learning styles 32

4.2 The suitability of the coursebook to the objectives of the course 33

4.2.1 Data collected from the questionnaire 33

4.2.2 Data collected from semi-structured interviews 37

4.2.3 Discussion and conclusion 38

4.3 The suitability of the coursebook to students’ needs 38

4.3.1 Data collected from questionnaires 38

4.3.2 Data collected from semi-structured interviews 44

4.3.3 Discussion and conclusion 45

4.4 The suitability of the coursebook to the current teaching and learning methods 46 4.4.1 Data collected from questionnaires 46

4.4.2 Data collected from semi-structured interviews 49

4.4.3 Discussion and conclusion 50

4.5 General evaluation and suggestion for further use of the coursebook 50

4.5.1 General evaluation 51

4.5.2 Suggestions for further use 52

CHAPTER 5: CONCLUSION 54

5.1 Conclusion 54

5.1.1 The suitability to the objectives of the course 54

5.1.2 The suitability to students’ needs 54

5.1.3 The suitability to current teaching and learning methods 55

5.2 Limitations and suggestions for further studies 56

REFERENCES 57 APPENDIXES I APPENDIX 1: QUESTIONNAIRE FOR STUDENTS I APPENDIX 2: QUESTIONNAIRE FOR TEACHERS X APPENDIX 3: TRANSCRIPT OF TEACHER INTERVIEW XIX APPENDIX 4: TRANSCRIPT OF STUDENT INTERVIEW XXIII

CHAPTER 1: INTRODUCTION

This chapter presents a general overview of the study with specific references to the rationale, the aims of the study, research questions, scope of the study, significance of the study, and structure of the study

1.1 Rationale

Coursebooks are generally considered as the core materials and “the visible heart of any ELT program” (Sheldon, 1988) They play a lot of essential roles in ELT such as a source of learning and teaching activities, a resource of learners’ self-study, a syllabus, and a support for inexperienced teachers (Cunningsworth, 1984) However, coursebooks are usually designed and written for the general market, thus they cannot fit in all teaching and learning contexts with possibly inappropriate cultural and social contents or teaching methods That is the reason why selecting a suitable coursebook for a particular teaching course is a challenging It also takes on special importance because the chosen coursebook would determine the teaching and learning process and reflect the value of the educational institution Therefore, the need of evaluating coursebooks should also take on the same importance It helps identify the strong points and weak points of the materials so that future decisions can be made about whether or not to keep using the materials, or adapting them to better fit learners’ needs and learning objectives However, very few of such a retrospective evaluation has been undertaken in comparison with the predictive evaluation, which is carried out before using the materials to decide if they are

to be selected (Ellis, 1997) In terms of retrospective evaluation, the evaluation

of the materials that have actually been used, there are two ways of evaluating: impressionistic and empirical evaluations While the former involves evaluation based on observation of learners’ engagement and enthusiasm in

activities and contents of the book, it is more commonly conducted than the latter one, which relates to collecting data in a more systematic way

The importance and the inadequate quantity of empirical evaluations of coursebooks triggers a desire in the researcher to apply an evaluation of this type to the coursebook “English File Beginner Student’s Book – Third Edition” for non-English major students in a preparatory school in Viet Tri Since English was incorporated into the school program as a compulsory subject, there has not been an official coursebook approved by the Ministry of Education for the school It is the teaching staffs who select the book that they consider best fit the objectives of the course and corresponds to students’ needs based on predictive evaluation of the books The coursebook “English File Beginner Student’s Book – Third Edition” is currently used as the main coursebook in the school after several choices of different materials After a year of piloting the book, the researcher, also the teacher who has worked with this book, finds it quite interesting to students with a lot of authentic videos and meaningful tasks; however, it also reveals several problems which renders it difficult to meet students’ needs and achieve the ultimate goal of teaching and learning program To present, there has not been any careful and systematic evaluation of this material That is the reason why the researcher desires to perform an empirical evaluation on this book to examine the validity of the predictive evaluation of the teachers in English groups of the school and that of her personal impressionistic evaluation, as well as to evaluate the appropriateness of the book for learners, learning and teaching context and purposes

1.2 Aims of the study

This study mainly aims at evaluating the coursebook “English File Beginner Student’s Book – Third Edition” by Christina Latham-Koenig, Clive

Oxenden (Oxford University Press, 2014), which is used for non-English major students at a preparatory school in Viet Tri to determine whether it meets the requirements of the course in terms of the objectives, students’ needs and methodology

1.3 Scope of the study

In terms of book evaluation, Cunningsworth (1984) listed a number of important evaluating criteria in the quick-reference checklist for evaluation such as: aims and approaches, design and organization, language content, skills, topic, methodology, teachers’ book, practical considerations Besides, evaluation can be conducted predictively or retrospectively for different purposes (Ellis, 1997) Due to the time constraint and the length limitation a master’s thesis, the subject of the evaluation is narrowed down to one type of retrospective evaluation, the empirical evaluation of the coursebook “English File Beginner Student’s Book – Third Edition” for non-English major students

at a preparatory school in Viet Tri with main focus its suitability to the course’s objectives, students’ needs and current methodology

1.4 Research questions

The study will be conducted to answer the following three questions:

- To what extent does the coursebook “English File Beginner Student’s Book – Third Edition” meet the objectives of the course?

- To what extent does the book meet students’ needs?

- To what extent is the book suited to the current teaching and learning methods?

1.5 Significance of the study

The findings of the thesis would contribute to the school’s decision of using the coursebook “English File Beginner Student’s Book – Third Edition”

or parts of the book again, or substituting and adapting inappropriate parts of book with more suitable ones, or replacing the book with another one Thus, it would benefit both teachers and learners who work with the material Teachers would have a chance to apply the most appropriate materials and teaching methods for their students with different levels whereas students could find the materials that best fit their needs and their learning purposes The result will also be valuable for teachers and learners in other preparatory schools in the country with similar teaching and learning context

1.6 Structure of the study

The study consists of five chapters:

Chapter 1: Introduction - presents a general overview of the study with

specific references to the rationale, the aims of the study, research questions, scope of the study, significance of the study, and structure of the study

Chapter 2: Literature Review - provides theoretical basis for the study with

the detailed discussion about some relevant theories related to textbook, coursebook, material, material evaluation and need analysis

Chapter 3: Research methodology – describes the methodology employed in

the study

Chapter 4: Findings and discussion – includes a detailed description of the

findings as well as full explanation and interpretation of these findings

Chapter 5: Conclusion – gives conclusion and limitations of the study and

then provides suggestions for further study

1.7 Summary

Chapter 1 has introduced the rationale, aims and scope of the study Research questions, significance and structure of the study were also included The importance of coursebook and coursebook evaluation, together with the inadequate quantity of empirical evaluation of the coursebook “English File Beginner Student’s Book – Third Edition” used for non-English major students

at a preparatory school in Viet Tri have triggered the researcher’ desire to conduct the study The main focus of the study is on the suitability of the coursebook to the course objectives, students’ needs and current teaching and learning methods Therefore, this study aims at evaluating this coursebook to find out if it meets the course objectives, students’ needs and current teaching and learning methods The findings of the study will help the school to make decision about keep using the book or replace it or part of it by more suitable materials

CHAPTER 2: LITERATURE REVIEW

This chapter provides theoretical basis for the study with the detailed discussion about some relevant theories related to textbook, coursebook, material, material evaluation and need analysis

2.1 Text book, coursebook and material

2.1.1 Definitions

Teaching materials are an indispensable component of any language teaching curricula There are a number of ways researchers defined this concept but they shared the same viewpoint that materials can be anything used to facilitate language teaching and learning (Littlejohn 1998, Tomlinson 1998, McGrath 2002) Littlejohn (1998) and Tomlinson (2011) viewed materials in various forms which can be students’ books, workbooks, teacher’s guides, videos, CDs, DVDs, lesson plans, website activities and more Similarly, McGrath (2002) provided a broad sense of materials in which he explained that they could be “realia” (real objects) or presentations (drawings or photographs)

He also added textbooks, worksheets, computer software and recordings as sources of materials Among these forms, textbook is considered the official material and “the visible heart of any ELT programs” (Sheldom, 1988) It plays

an important role of teaching a particular subject in schools and colleges and serves a basis of study for students and a primary teaching instrument for teachers (Oxfordlearnersdictionaries.com, 2008)

As a type of teaching materials, a coursebook is defined as “a textbook that students and teachers use as the basic of a course” (Collinsdictionary.com, 2018) In other words, it is used to refer to a book “used by students when they

do a particular course of study” (Dictionary.cambridge.org, 2018) In accordance with the dictionary definition, Ur (1996) and Tomlinson (2011)

explain that coursebooks function as the core materials for a language-learning course that a teacher and each student has a copy It aims to provide as much

as possible in one book and serves as the only book which learners basically use during a course A coursebook usually includes exercises not only on language elements like grammar, vocabulary, pronunciation but also on functions and skills of reading, writing, listening and speaking

From the above view of “materials”, “textbook” and “coursebook”, it can

be pointed out that text book and coursebook are types of materials and these three terms can be used interchangeably in ELT teaching and learning context

2.1.2 Roles of materials in ELT classrooms

According to Richards and Rogers (2001) as cited in Ulaş Kayapinar (2009), coursebooks play an indispensable part in the curriculum because they specify content and define coverage for syllabus items They play multi essential roles in ELT such as a source of learning and teaching activities with systematic and standard knowledge of grammar, vocabulary, pronunciation, etc., a resource of learners’ self-study, a syllabus in accordance with determined learning objectives, and a support for inexperienced teachers (Cunningsworth, 1995) Coursebooks also help standardize instruction, frame the language contents, and provide language models and practice activities for teachers and students to follow in class (Ur, 1996) Ur adds more advantages of coursebooks For example, they provide available and appropriate texts and exercises for most learners and classes They are also the most economic and convenient ways of providing teaching and learning materials for both teachers and learners Thus, the use of coursebooks is more and more popular among universities, private schools and some state schools It is the ready-made syllabi

in the ready-made coursebooks that are preferred by both school administrations and teachers of English Besides, coursebooks provide

opportunities for learners to practice the target language in the classroom before they use it in real life situations

However, coursebooks as preplanned instruction materials have some possible disadvantages According to Richards and Renandya (2002), as cited

in Ulaş Kayapinar (2009), coursebooks fail to present real-life language models and contextualize language activities They fail to address linguistic competence as well as to teach idioms in everyday language Lack of equity in gender representation and the inadequate cultural understanding encouragement are also among the disadvantages of coursebooks Ur (1996) adds the following drawbacks of coursebooks Firstly, coursebooks are inadequate in that every class and learner has their own learning needs and a coursebook cannot supply these satisfactorily Secondly, coursebooks are irrelevant because the topics dealt within the books may not necessarily be interesting for the class Coursebooks may lead to boredom and lack of motivation on the part of the learners Also, they do not cater for variety of levels of ability and knowledge that exist in most classes Moreover, although coursebooks are considered as the magical instruction tool for language teachers, highly structured coursebooks may even lead to the de-skilling of teachers (Hutchinson & Torres 1994 as cited in Osman Dülger 2016)

In conclusion, on one hand, coursebooks prove to be very advantageous for both school administrations, language teachers and students with available syllabi and already designed texts and tasks which are suitable for a large portion of language learners On the other hand, coursebooks reveal some limitations such as inadequacy, irrelevance, inauthenticity That is the reason why any coursebooks should be evaluated to see whether or not they match the school curriculum and learners’ needs and interests

2.2 Coursebook evaluation

2.2.1 Definitions of coursebook evaluation

Coursebook evaluation is of great significance as it seeks to identify the strengths and weaknesses’ of the books and helps to make decisions about adapting the materials or adopting new ones In order to conduct the evaluation

of a coursebook, it is necessary to understand what the process of evaluation involves There are a number of definitions of evaluation provided by researchers According to Tomlinson (2011), coursebook evaluation is an attempt to measure the potential value of the coursebooks It involves making judgements about the effects of coursebooks on such agents as learners, instructors, administrators who use them through such features of the books as credibility, validity, flexibility, etc Coursebook evaluation is also defined by Rea-Dickins and Germanie (1994) as “the means by which we can gain a better understanding of what is effective, what is less effective and what appears to

be no use at all” Effective evaluation relies on asking appropriate questions and interpreting the answers to them (Cunningsworth, 1995)

In summary, coursebook evaluation is the process of collecting data, giving judgement on the effectiveness of books based on the collected data to make precise decisions of effectively using the materials or replacing them

2.2.2 Approaches to coursebook evaluation

Material evaluation is “a dynamic process which is fundamentally a subjective, rule-of-thumb activity" where "no neat formula, grid, or system will ever provide a definitive yardstick" (Sheldon, 1988) There are a variety of approaches and criteria of coursebook evaluation which vary according to the ELT contexts Administrators and instructors should choose the most suitable

criteria to evaluate coursebooks, considering the specific teaching and learning contexts

The following sets out a range of approaches to coursebook evaluation which all tend to evaluate coursebooks on four main aspects, including the internal content, the aims and approaches, the supporting sources and the physical appearance of the coursebook

Ellis’s approach (1997)

According to Ellis (1997), there are two main types of coursebook evaluation: predictive evaluation and retrospective evaluation A predictive evaluation is designed to help teachers decide what coursebooks to use whereas

a retrospective evaluation may be conducted once the book is used to find out

if the material matches the learners’ need and current teaching methodology Predictive evaluation, according to Ellis, can be carried out in two principal ways One involves teachers relying on evaluations carried out by experts specializing in coursebook evaluation Alternatively, teachers can carry out their own predictive evaluations by making use of various checklists and guidelines available in the literature Another kind of evaluation – retrospective evaluation – is carried out while a coursebook is in use (whilst-use evaluation)

or after it has been used (post-use evaluation) There are two ways of retrospective evaluations: impressionistic and empirical one While the impressionistic evaluation is based on observation of learners’ engagement and enthusiasm in activities and contents of the book, empirical one relates to

“collecting data in a more systematic manner” (Ellis, 1997) To conduct empirical evaluation, teachers use “end-of-course questionnaires to judge the effectiveness of their teaching, including the material used”; therefore, it is often more objective and reliable than other kinds of evaluation as it is based

on users’ feedback Such an evaluation provides information that can be used

to determine whether the material is worth using again In addition, Ellis (1997)

states that retrospective evaluation is a good way of testing the validity of a predictive evaluation

Tomlinson’s approach (2003)

Another approach of evaluating coursebooks is proposed by Tomlinson (2003) with a three-stage process of evaluation: pre-use evaluation, whilst-use evaluation and post-use evaluation The first stage, pre-use evaluation, according to Tomlinson (2003), is “impressionistic and consists of a teacher flicking through a book to gain a quick impression of its potential value” It involves assessing the potential value of materials for their users, including a quick scan of a book’s physical appearance and content pages to get an impression of its potential value The second stage, whilst-use evaluation, involves evaluating materials in use This can be more reliable than pre-use evaluation, as it involves deeper evaluation of the content of materials and makes use of classroom observation and feedback from the users although, according to Tomlinson (2003), this stage includes controversial issues about what exactly can be measured The criteria seem to be general and some (e.g credibility of tasks, achievement of performance objectives, motivating power

of the materials, etc.) are not easy to measure However, he also suggests that these criteria “can be estimated during an open-ended, impressionistic observation of materials in use but greater reliability can be achieved by focusing on one criterion at a time” They can be measured by different methods For example, to measure the motivating power of the materials he suggests ‘noting such features as student eye focus, proximity to the materials, time on task and facial animation’ (Tomlinson, 2003) In the last stage of evaluation in this approach, post-use evaluation, most evaluators tend to seek answers to such questions as: What is the impact of the coursebook on learners? What is its impact on teachers? And what is its impact on administrators? According to Tomlinson (2003), this stage is probably the most valuable stage

in the evaluation process as it involves measuring the effects of the materials

on users after the coursebook has been used Also, as Cunningsworth (1995) points out, post-use evaluation is useful in helping to decide whether to continue using a coursebook or not in the future

Littlejohn’s approach (1998 & 2011)

The third major approach for coursebook evaluation is suggested by Littlejohn (1998 & 2011) He suggested a three-level evaluation checklist including: (1) What is there? (2) What is required of users? and (3) What is implied? At the first level, there are questions about the physical properties of the coursebook These cover publication date, intended audience, physical aspects (number of pages, paper quality, artwork, etc.), and are similar to the ones in Tomlinson (2003) Level 2 focuses on the language learning activities and tasks in the coursebook to see what teachers and learners using the coursebooks need to do It moves slightly deeper into an analysis of what is assumed to be the most important aspect of coursebooks At the third level, there are questions on the approach, philosophy and aims of the coursebook Littlejohn points out that his levels of evaluation move from more objective criteria to less objective ones The first level is the most objective while the third level is the least

‘in-learning Features assessed at this step include the teacher’s book and how well

it relates to the student book, supporting resources (e.g cassette recordings, photocopiable worksheets, etc.) If all these are satisfied, the evaluator proceeds

to step 3 - context relevance At this stage the evaluator assesses the suitability

of the coursebook in terms of aims, syllabus, length and its suitability for learners’ proficiency level, age and background If all of these meet the requirements, the evaluator goes further to step 4 - learner appeal This step involves considering the appeal of the book to learners This is mainly related

to the appearance of the book The evaluation starts from the first sub-step If a coursebook meets all of the key criteria for each step, the process continues with the next one If it does not meet the key criteria in any sub-step, the process stops and the coursebook is rejected If all of the above-mentioned key criteria are met, the in-depth evaluation step is carried out to assess the coursebook in detail

According to McGrath (2002), the in-depth evaluation tends to address the following points:

- The aims and content of the book

- What they require learners to do

- What they require the teacher to do

- Their function as a classroom resource

- Learner needs and interests

- Learner approaches to language learning

- The teaching-learning approach in the teacher’s own classroom

McDonough and Shaw’s approach (1993 & 2003)

Another important approach for coursebook evaluation is suggested by McDonough and Shaw’s (1993 & 2003) with three stages: external evaluation, internal evaluation and overall evaluation This implies that evaluation is continuous and never static, as the criteria can be changed to suit different

teaching and learning contexts External evaluation is similar to McGrath’s (2002) ‘first-glance evaluation’, Tomlinson’s (2003) ‘pre-use evaluation’ and Littlejohn’s (1998 & 2011) ‘level 1 evaluation’ It focuses on the intended audience, the proficiency level, the context, the organization and presentation

of units, the author’s views and methodology, the learning process and the learner, and the physical appearance Issues of layout and design and local availability, supporting resources (teacher’s book, audio-visual materials, etc.) are also included in this stage The internal evaluation is concerned with the content, including grading and sequencing This stage seems to resemble McGrath’s (2002) ‘in-depth evaluation’ and Tomlinson’s (2003) ‘whilst-use evaluation’

In general, each approach to coursebook evaluation, as mentioned above, has its own strengths and weaknesses Evaluators should choose or even devise

an approach which is the most principled, systematic and suitable for their context In the researcher’s opinion, it is also important for evaluators to combine approaches, in order to offset the weaknesses of one approach with the strengths of others The present study uses a mixed approach, mainly based

on Ellis’s framework (1997), though modified, with evaluation criteria from other authors

2.2.3 Criteria for coursebook evaluation

From different approaches to the evaluation of coursebooks, there are also different sets of criteria for evaluating coursebooks

Criteria proposed by Hutchinson and Water (1987)

Hutchinson and Water (1987) proposed a checklist of material evaluation including items related to subjective and objective analysis of the book such as: audience, aim, content, methodology, price and its availability

Audience: This part of the checklist consists of some questions related to

information about the learners such as their age, sex, nationalities, study or work specialism, language background and interests

Aim: Evaluators need to consider if the aims and objectives of the

coursebook match those of the course/ school program

Content: The questions covered in this part involve the language

description, language point, macro-skill and their proportion, micro-skill, types, subject matter areas, topics, organization through the course, organization within the course unit, content sequence

Methodology: In this part, the checklist investigates methodology

including questions about the theory of learning, the attitudes or the expectation

of the learners about learning, exercises and tasks, teaching and learning techniques used, teaching aids, teaching guidance needed and the flexibility of the material in different teaching situations

Criteria suggested by Cunningsworth (1995)

Cunningsworth (1995) proposed a number of criteria to consider the materials, which was presented as a basic quick-reference checklist for evaluation and selection, including: aims and approaches, design and organization, language content, skills, topic, methodology, teacher’s books and practical considerations

Aims and approaches: Evaluators should take into consideration the

suitability of the aims of the coursebook with the aims of the teaching program and the needs of the learners as well as the suitability of the book with the learning/ teaching situation and styles

Design and organization: The components of the book package, the

organization of the content, the recycling and revision included in the coursebook are concerned in this criterion

Language content: The language components such as grammar,

vocabulary, pronunciation is considered carefully through detailed questions in Ellis’ checklist

Skills: The questions involved four skills are to be investigated in this set

of criteria with regard to the adequacy, authenticity, integrity and suitability of the skills in the coursebook with students’ level

Topic: Evaluators need to consider whether the coursebook provides

students with enough variety and range of topics of genuine interest to learners; whether the topics equally address men and women as well as other groups of various ethnic origin, occupation, disability; whether the social and cultural values are conveyed in the range of topics in the book

Methodology: Approaches to language learning presented in the

coursebook, the language presenting/ practicing techniques, the method to teach different skills and the guideline to students’ self-study are taken into account in this set of criteria

Teachers’ books: The guidance, supporting materials and the answer key

that the book provides for teachers as well as the teaching techniques, language items and cultural content that they cover are to be evaluated

Last but not least, the practical considerations which involves the price,

the duration, the appearance of the book and the equipment it requires are also considered

Criteria proposed by Wong Pak Wing Lawrence (2011)

Wong Pak Wing Lawrence (2011) proposed a self-constructed evaluation framework for evaluating curriculum fitness with the following criteria:

Use of task-based approach teaching and learning strategies: Evaluators

base on this set of criteria to check if the materials are designed with centered instruction, target-oriented English learning, integrative and creative

learner-language use, learning grammar in context, purposeful tasks and exercises as well as extended tasks and project learning

Integrated skills: In this part, the checklist examines if the four skills –

listening, speaking, reading and writing – are integrated in the book for authentic purposeful communication

The teaching of language arts: Evaluators take into consideration the

place of language arts (i.e films, songs, lyrics, movies, dramas, novels, short stories, etc.) in the English Language Curriculum and considerations for selecting language arts materials

Other criteria are to check if the book promotes independent language learning or provide information technology for interactive learning and life-wide learning

In summary, the criteria were set by both established material evaluators like Hutchinson and Water (1987), Cunningsworth (1995) and novice research scholars like Wong Pak Wing Lawrence (2011) for an in-depth and objective analysis of teaching materials They share some core factors and the criteria listed may not always be effective in a specific context Therefore, it is necessary to prepare “a new one or modify the existing ones in order to cover all aspects of evaluation” (Cunningsworth, 1995) In this regard, in this study the author used and modified ideas from these checklists to design a set of criteria for the empirical evaluation of the coursebook “English File Beginner Student’s Book – Third Edition” for non-English major students in a preparatory school in Viet Tri

2.3 Needs analysis

Needs analysis (also known as needs assessment) play a vital role in the process of designing and carrying out any language courses, and its importance has been acknowledged by several scholars and authors

The term “needs analysis”, according to Elaine Tarone and George Yule (2000), refers to the collection and evaluation of information to answer the question: “What aspects of the language does some particular group of learners need to know?” Mountford (1981) offers the definition that “needs” can be defined as what students should be able to do at the end of their language course, or “what the user institution or society at large regards as necessary or desirable to be learnt from a program of language instruction” Another definition given by Brindley (1984) is that “needs” refers to wants, desires, demands, expectations, motivations, lacks, constraints, and requirements It is evident that course designers may take students’ needs into consideration when designing a course For Johns (1991), needs analysis is the first step in course design and it provides validity and relevancy for all subsequent course design activities Different approaches to needs analysis attempt to meet the needs of the learners in the process of learning a second language Among which, a modern and comprehensive concept of needs analysis is proposed by Dudley-Evans and St John (1998) with the concept of needs analysis as follows:

 Environmental situation – information about the situation in which the course will be run (means analysis)

 Personal information about learners – factors which may affect the way they learn (wants, means, subjective needs)

 Language information about learners – what their current skills and language use are (present situation analysis)

 Learner’s lacks (The gap between the present situation and professional information about learners): learner’s needs from the course – what is wanted from the course (short-term needs); language learning needs – effective ways of learning the skills and language determined by the lacks

 Professional information about learners – the tasks and activities English learners are/will be using English for (Target Situation Analysis and objective needs)

 How to communicate in the target situation – knowledge of how language and skills are used in the target situation (register analysis, discourse analysis, genre analysis)

Different types of need analysis are presented in the below figure: Needs analysis jigsaw

As can be seen from the figure, these types of needs analyses are not exclusive but supplementary Each of them provide a piece to complete the jigsaw of needs analysis

2.4 Previous studies on coursebook evaluation

Up to now, there have been a number of studies on coursebook

evaluation In 2009, Ulaş KAYAPINAR conducted a study which revealed the

teachers’ views on the quality of foreign coursebook packages (from beginner

to intermediate level) widely used in the teaching process of English preparatory classes in twenty-five different high schools In the study, widely used (best-selling) coursebooks with the same sub-domains were chosen to be evaluated by teachers The data were gathered from the questionnaire results of ninety-four teachers and standard open-ended interview results of forty teachers who teach in English preparatory classes and use particular coursebook packages in the classroom environment (n=134) The views revealed that teachers did not have positive impressions about the coursebook packages used in general Moreover, the general conceptions of the teachers suggest that coursebooks should be developed and used to meet the needs of the learners in the national context In the same year, Azadeh Nemati carried out a study to evaluate English Pre-University textbook of Karnataka state in India with respect to general criterion as well as vocabulary teaching Firstly, a questionnaire was made with reference to some critical features extracted from different material evaluation checklists The prepared questionnaire was completed by 26 Pre-University teachers from 12 randomly selected governmental and non-governmental schools Then strong points and weaknesses of the coursebook were discussed in detail In the second phase, to analyze the vocabulary of the book systematically, some texts of the book were randomly selected, submitted to vocabulary profile to see if they are sequenced from the most frequently used words to the less used ones which according to Nation (1990) is an important factor in vocabulary teaching

In Vietnam, Tran Minh Thanh (2014) conducted a study to evaluate the book “English for Information Technology” for second-year students of a university in Thai Nguyen to see how the coursebook is relevant to the learners’ needs in terms of content, format and methodology Survey questionnaires and semi-structured interviews were employed to investigate both students and

teachers’ perception of the book The result revealed that the coursebook was relatively relevant to the needs of the learners and it should be further used by second-year students in the university with some changes and adaptation

In 2015, Tran Thi Thuy carried out an evaluation of the coursebook

“New English File - Elementary” for the first year students at a university in Hanoi The aim of the study is to evaluate the material to determine whether it matches the course’s requirements in terms of the audience, content, aims and methodology The criteria suggested by Hutchinson and Warter (1987) together with two data collection instruments: document analysis and survey questionnaires were selected The findings of the study showed that the book basically met the requirement of the Faculty of English Department of the university besides some demerits to be changed and adapted

This study is also carried out to evaluate the coursebook “English File Beginner Student’s Book – third edition”, but the evaluation is only focused on the suitability of book to the aims and methodology of the course as well as the needs

of the students The researcher does not use a single model to conduct the evaluation but criteria from different evaluators are chosen and adapted for the evaluation by means of delivering questionnaire and interviewing both teachers and students It is hoped that the study will reveal reliable and valuable results

2.5 Summary

Chapter 2 has provided theoretical framework for the thesis Knownedge of text books, coursebooks and materials has been discussed and an agreement has been reached that these three terms can be used interchangeably Roles of coursebook, definition, approaches and criteria of coursebook evaluation were also provided together with related studies on coursebook evaluation In this thesis, the researcher uses mixed approaches, mainly based on Ellis’ framework (1997) together with a selection of different criteria by different evaluators to evaluate the coursebook “English File Beginner Student’s Book – Third Edition”

CHAPTER 3: RESEARCH METHODOLOGY

This chapter presents a description of the research method employed in the study with four aspects including: the setting, participants, data collection instruments and data collection procedure

3.1 The setting

3.1.1 English teaching and learning context

The preparatory school in which this study is conducted was founded in

1975 in order to prepare the necessary knowledge and skills for students who have just graduated from high schools to enter university All of the students between the age of eighteen and twenty are from ethnic groups, mainly in northern areas of the country Most of them have studied English for about seven or ten years but the non-English major students are almost at beginning level while those whose major is English are at elementary or pre-intermediate level They only study at the school for one year with two semesters, each lasting three months

The school year 2017-2018 is the second year the coursebook “English File Beginner Student’s Book – Third Edition” officially used for non-English major students at the school At first, the coursebook “English Know-how Opener” was applied for non-English major students of blocks A, B, C, then it was substituted by a new, state-of-the-art coursebook namely “New English File Beginner”, and finally replaced by a new more updated version called

“English File Beginner Student’s Book – Third Edition”, which is currently used as the main coursebook in the school The teaching and learning of English File Beginner series follow communicative language teaching approach

3.1.2 Course material

English File Beginner was designed by Christina Latham-Koenig, Clive Oxenden in 2014 It is the third edition of the original book “New English File Beginner Student’s Book” The package of book “English File Beginner Student’s Book – Third Edition” includes a Student’s Book, a Teacher’s Book, one Workbook without keys for students and one with keys for teachers, English File Beginner iTools and MultiPACK, Class Audio CDs and DVD and the e-book version of the Student’s Book, Teacher’s Book and Workbook The English File Beginner Student’s Book consists of twelve units/ files divided into two teaching sections Each file presents and practises grammar, vocabulary and pronunciation with a balance of reading and listening activities and lots of opportunities for speaking At the end of every two units, there is one “Practical English” section, which teaches useful, high-frequency, social English and everyday English with real-world In-The-Street interviews There are also Practical English drama and documentary videos that motivate and engage students with English language and culture The Revise and Check pages after every two units help students to consolidate and measure their progress Besides, at the back of the book includes additional resources such as Grammar Bank, Vocabulary Bank and Sound Bank which offer additional support for students

3.1.3 Course objectives

The English course provided by the preparatory school aims to review and reinforce basic English knowledge that students have learnt at high school as well as prepare for students with essential knowledge and skills to enter universities/ colleges After the course, the students are expected to extend their vocabulary of familiar topics such as our life, community, recreation and so on

Also, they are expected to be able to exercise and improve four skills: listening, speaking, reading and writing for basic communication Moreover, after finishing the course, students can develop skills of working individually and collaboratively in pairs or groups

3.1.4 Course assessment

Because the coursebook “English File Beginner Student’s Book – Third edition” is applied for non-English major students, there is only one kind of summative assessment which includes two forty-five-minute end-of-term tests

3.2 Research methodology

3.2.1 Participants

The informants of the study were 150 non-English major students out of

356 students of the academic year 2017-2018 and 08 Vietnamese teachers of English who were using the coursebook “English File Beginner Student’s Book – Third edition” Of nine classes, four classes including 56 female students and

94 male students from eighteen to twenty years old were randomly chosen by the researcher They all had just finished high school and most of them learned English for seven years or more However, they have different levels of English proficiency because they come from different places in Vietnam with different background knowledge, though most of them are only at beginning level Regarding English teachers, most of them have more than two years in teaching English File at the school One teacher is taking the M.A course and three of them are M.A degree holders in teaching methodology or linguistics Moreover, all the teachers are willing and enthusiastic toward their teaching and they always pay attention to the improvement of the course Therefore, they show a lot of interest in this study

3.2.2 Research instruments

To evaluate if the coursebook meets the requirement of the course in terms of methodology, objectives and needs of the students, two instruments for collecting data were used, which were questionnaires and semi-structured interviews

Questionnaires

In this study, both questionnaires were used for both teachers and students

to collect their attitude and evaluation of the currently used coursebook

“English File Beginner Student’s Book – Third Edition” in terms of its suitability to the objectives of the course, students’ needs and current teaching and learning method Each questionnaire is divided into five sections with student version being written in Vietnamese and teacher version in English

Section 1 consists of four questions in questionnaires for students and one

question in questionnaires for teachers While the students’ questionnaires ask about students’ English learning background and their motivation, the teachers’ version aims at finding out learners’ level of English

Section 2 is to identify the suitability of the coursebook to the aims of the

course Three questions in both questionnaires for teachers and students aim at investigating students’ progress after the course as well as the extent to which the coursebook “English File Beginner Student’s Book – Third edition” meets the aims of the course

Section 3 includes five questions in both questionnaires for students and

for teachers which are to identify the suitability of the course to students’ needs Questions about the topics of the book, the language elements and the skills taught in the book are asked to collect data about its fitness to students’ needs

Section 4 is designed with three questions in both students’ and teachers’

questionnaires to measure the suitability of the coursebook to the current teaching and learning methods through questions about the teaching and learning techniques used in the book and the teaching aids required by the book

Section 5 is teachers’ and students’ general evaluation of the coursebook

as well their suggestions and expectations for the book adjustment if necessary

Semi-structured interviews

In order to get supplementary ideas besides those collected from the questionnaires, the semi-structured interviews are designed for both teachers and students For teachers, there are four questions to clarify the strengths and weaknesses of the book, its suitability to the course aim, students’ needs and the current teaching and learning methods, the difficulties teachers and students have when using the material and their suggestions for better use For students, there are also four questions about what students like or dislike about the coursebook, their progress after the course, the suitability of the book to their level, learning purposes and learning method, the difficulties they have using this material and their suggestion for future use

3.2.3 Data collection procedure

Data collection was carried out through questionnaires and semi-structured interviews for both teachers and students

- First, the researcher designed the questionnaires based on the criteria that she had carefully chosen and adapted from different evaluators

- Next, the questionnaires were distributed to the students and the teachers who are participants of the study They were returned on the same day

- Then, the semi-structured interviews were conducted All of eight teachers from English group of the school were interviewed and coded as teacher

1, 2, 3, 4, 5, 6, 7 and 8 Two groups of students were selected randomly from two classes, with one group including four students coded as student A, B, C, D and another group including three students coded as student E, F, G These interviews were carried out after synthesizing the data collected from the questionnaire Their answers were collected and analyzed to support the questionnaire results about the suitability of the coursebook to the course’s objectives, students’ needs and current teaching and learning method

- Finally, the data were processed manually and the results of the study would be revealed and discussed in the next chapter

3.2.4 Data analysis methods

The data were analyzed quantitatively and qualitatively For analysis of the questionnaires, the Likert scale was used and necessary calculations were carried out For the analysis of the semi-structured interviews and the open-ended questions in the questionnaires, qualitative techniques were used to categorize and analyze the data

3.3 Summary

Chapter 3 has presented the setting and methodology of the study English teaching and learning context at the preparatory school where the study was conducted, course objectives and course assessment applied in the school were discribed There were 150 students and 8 teachers answered the questionnaires 7 of those 150 students and all of 8 teachers were interviewed after the data collected from questionnaires were synthesized Finally, data were processed and analyzed both quatitatively and qualitatively to reveal the findings of the study

CHAPTER 4: FINDINGS AND DISCUSSION

In this chapter, the researcher presents and discusses the results from the questionnaires and interviews to find out whether the coursebook “English File Beginner Student’s Book – third edition” is suitable to the course in terms of the course aim, students’ needs and current teaching and learning methods

4.1 About the learners

Background information about learners such as their age, years of learning English as well as their current English levels or their purposes of learning is very important for educational institutions to build up the appropriate curriculum and decide on the suitable coursebook for students to study The charts and tables below summarize some major information of the learners at the preparatory school

4.1.1 Learners’ English learning background

The two pie charts below present students’ age and their background of learning English as the result collected from students’ questionnaires

Chart 4.1: Students’ age

8%

92%

20 years old 19 years old

As can be seen from Chart 4.1, a large majority of the students (92%) were nine-teen years old at the time of survey, which means that they entered the preparatory school right after they graduated from high school The rest (8%) who were twenty years old joined the school one year after high school graduation

Chart 4.2: Students’ background of learning English

It is obvious from chart 4.2 that all students have been learning English for a long period of time Almost half of them (57%) have spent 7 years studying English while the other half have studied the subject for 10 years (28%) or more than ten years (15%) It means half of them started learning English at primary school and the other half at secondary school

4.1.2 Learners’ English level

While chart 4.3 presents students’ self-assessment of their English level at the beginning of the course, table 4.1 below shows teachers’ assessment of their

students’ level It can be seen that 74% of the students were at “poor” level,

which is consistent with teachers’ assessment based on the result of the placement test at the beginning of the school year All teachers assessed that

from 70% to 90% of the students had “poor” result in English Nearly a quarter

of the students (21%) supposed that they were at “average” level and very few (5%) admitted that they were “good” at English This result is similar to

teachers’ assessment that from 10% to 30% of the students were at average level and from only 1% to 10% of them were good Both teachers and students did not highly appreciate students’ ability in learning English when none of

them assessed the students were “excellent”

Table 4.1: Teacher’s assessment of students’ English level

at the beginning of the course

Percentage

Level

1%- 10%

10%

- 20%

20%- 30%

30%

- 40%

40%

- 50%

50%

60%

-60%

- 70%

70%

80%

-80%

90%

- 100%

Chart 4.3: Students’ self-assessment of their English level

at the beginning of the course

From the above analysis, it can be concluded that a vast majority of students had poor level of English

74%

21%

5% 0%

Poor Average Good Excellent