An evaluation of the ESP material english for international tourism for students of tourism department at hanoi university of industry

LIST OF FIGURES AND TABLESFigure 1: Types of ESP Kennedy and Bolitho, 1984Figure 2: The materials evaluation model proposed by McDonough & Shaw1993:75 Figure 3: The materials evaluation

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OFLANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-

GRADUATE STUDIES -

NGUYỄN THỊ THÙY LINH

AN EVALUATION OF THE ESP MATERIAL “ENGLISH FORINTERNATIONAL TOURISM” FOR STUDENTS OF TOURISMDEPARTMENT AT HANOI UNIVERSITY OF INDUSTRY

ĐÁNH GIÁ TÀI LIỆU TIẾNG ANH CHUYÊN NGÀNH “ENGLISH FORINTERNATIONAL TOURISM” DÀNH CHO SINH VIÊN KHOA DU LỊCH

TRƯỜNG ĐẠI HỌC CÔNG NGHIỆP HÀ NỘI

M.A MINOR THESIS

FIELD: ENGLISH TEACHING METHODOLOGYCODE: 60 14 01 11

Hanoi, 2015

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OFLANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-

GRADUATE STUDIES -

NGUYỄN THỊ THÙY LINH

AN EVALUATION OF THE ESP MATERIAL “ENGLISH FORINTERNATIONAL TOURISM” FOR STUDENTS OF TOURISMDEPARTMENT AT HANOI UNIVERSITY OF INDUSTRY

ĐÁNH GIÁ TÀI LIỆU TIẾNG ANH CHUYÊN NGÀNH “ENGLISH FORINTERNATIONAL TOURISM” DÀNH CHO SINH VIÊN KHOA DU LỊCH

TRƯỜNG ĐẠI HỌC CÔNG NGHIỆP HÀ NỘI

M.A MINOR THESIS

I hereby, certify that the minor thesis entitled “An evaluation of the ESP material ‘English for International Tourism’ for students of Tourism Department at Hanoi University of Industry” is the result of my own research; and that it has not

been submitted for a higher degree to any other universities or institutions I agreethat the origin of my paper deposited in the library can be accessible for the purposes

of study and research

Hanoi, 2015

Nguyễn Thị Thùy Linh

I am, first of all, greatly indebted to my thesis supervisor, Dr Ngô Tự Lập, forencouraging me to complete this work Without his valuable instructions, comments,criticisms and corrections this thesis would be impossible Special thanks go to mylecturers of Post Graduate Department for their useful lectures

My appreciation and gratitude are also extended to all my colleagues andstudents who have stimulated and guided my thinking during the time I did thisresearch

Last but not least, I would like to express my deep gratitude to my parentswhose love and encouragement have been equally important to my educationalendeavors, and especially to my husband and my son who has helped me to ease theburden of the work

ABSTRACTMaterials evaluation is very important for language teaching and learningsince no textbook or set of materials is perfect This thesis was carried out to evaluatethe ESP material which is currently in use for the students in Department of Tourism

at HaUI The research findings are expected to give suggestions for the adaptation ofthe material in the near future with an ambition to improve the effectiveness of theteaching and the learning English at HaUI

The data collection instruments in this study were questionnaires, interviewand the document analysis The material in use is analyzed according to the criterialisted in Hutchinson and Waters’ objective analysis process A survey on the teachers'and students' opinions about the extent to which the material meets the requirements

of the course in terms of the aims, content and methodology is conducted Finally, thefindings from the analyses and the surveys are discussed

The result of the thesis revealed some strength and weaknesses of the material

It was found that the material basically met the course objectives in terms of aims,content and methodology

Based on the findings, the thesis suggests that adaptation should be made toovercome its weaknesses so that the students can benefit in the future course

LIST OF FIGURES AND TABLESFigure 1: Types of ESP (Kennedy and Bolitho, 1984)

Figure 2: The materials evaluation model proposed by McDonough & Shaw(1993:75)

Figure 3: The materials evaluation model proposed by Littlejohn (1998:3)Figure 4: The materials evaluation model proposed by Hutchinson & Waters(1987:98)

Table 1: The students' opinions on the aims of the material

Table 2: Students' opinions on the difficulty of the material

Table 3: Students' opinions on the usefulness of the material

Table 4: Tourism teachers' opinions on the material

Table 5: English teachers' opinions on methodology

Table 6: Students' opinions on the activities

LIST OF ABBREVIATIONSESP: English for Specific Purposes

GE: General English

ELT: English Language Teaching

HaUI: Hanoi University of Industry

EOP: English for Occupational Purposes

EAP: English for Academic Purposes

EST: English for Science and Technology

TABLE OF CONTENT

DECLARATION 1

ACKNOWLEDGEMENTS ii

ABSTRACT iii

LIST OF FIGURES AND TABLES iv

LIST OF ABBREVIATIONS v

PART A: INTRODUCTION 1

1 Rationale 1

2 Practical background of the research 1

3 Aims of the study 2

4 Research question 2

5 Scope of the study 2

6 Methods of the study 3

7 Organization 3

PART B: DEVELOPMENT 4

CHAPTER 1: LITERATURE REVIEW 4

1.1 An overview of ESP 4

1.1.1 Definitions of ESP 4

1.1.2 Types of ESP 5

1.2 Materials evaluation 6

1.2.1 Definitions of materials evaluation 6

1.2.2 Reasons for materials evaluation 6

1.2.3 Types of materials evaluation 7

1.2.4 Approaches to materials evaluation 8

1.2.4.1 Cunningsworth’s approach 8

1.2.4.2 McDonough and Shaw’s approach 8

1.2.4.3 Littlejohn’s approach 9

1.2.4.4 Hutchinson and Waters’s approach 10

1.2.5 Summary 12

1.2.6 Criteria for materials evaluation 13

1.3 Previous studies 14

1.4 Summary 16

CHAPTER 2: METHODOLOGY 18

Chapter 2 begins with the description of the research method It then specifies the data collection tools, the subject, and the data collection procedures 18

2.1 Data collection instruments 18

2.1.1 Document analysis 18

2.1.2 Questionnaires 18

2.1.3 Interview 19

2.1.3.1 Interview questions for English language teachers 19

2.1.3.2 Interview questions for teachers in department of Tourism 19

2.2 Subjects 19

2.3 Data collection process 20

2.4 Summary 21

CHAPTER 3: DATA ANALYSIS AND DISCUSSION 22

3.1 The appropriateness of the material to the aims of the course 22

3.1.1 Aims of the course 22

3.1.2 English teachers’ interview results 22

3.1.3 Students’ questionnaire result 22

3.2 The appropriateness of the material to the content requirements of the course 24

3.2.1 Material analysis results 24

3.2.2 The content requirements of the course 25

3.2.3 Teachers’ interview results 25

3.2.4 Students’ questionnaire results 26

3.2.4.1 The difficulty 26

3.2.4.2 The usefulness 27

3.2.5 Tourism teachers’ questionnaire results 29

3.3 The appropriateness of the material to the methodology requirements of the course 29

3.3.1 Methodology implied in the syllabus 29

3.3.2 Methodology implied in the material 30

3.3.3 Teachers’ interview result 30

3.3.4 Students’ questionnaire result 31

3.4 Suggestions for the material improvement 32

3.4.1 Teachers’ interview result 32

3.4.2 Students’ questionnaire result 33

3.5 Summary 33

3.5.1 Strengths 33

3.5.1.1 Good selection of authentic texts of different topics 33

3.5.1.2 Grammatical structures and grammar 34

3.5.1.3 Writing skills 34

3.5.2 Weaknesses 34

3.5.2.1 Language skills 34

3.5.2.2 Supporting materials 35

3.5.2.3 Methodology implied in types of tasks/exercises: 35

PART C: CONCLUSION 36

1 Summary of the study 36

2 Recommendations 37

2.1 On the ESP materials 37

2.1.1 Areas for adaptation 37

2.1.2 Techniques for adaptation 37

2.2 On the requirements of the course 38

3 Limitations and suggestions for further study 38

REFERENCES 40APPENDICES IAPPENDIX 1 IIAPPENDIX 2 VIIIAPPENDIX 3 XIAPPENDIX 4 XIIAPPENDIX 5 XIVAPPENDIX 6 XX

PART A: INTRODUCTION

1 Rationale

Today, English is used in a wide variety of fields such as technology,medicine, education, science, economics, etc, so it is becoming more and moreimportant In Vietnam, English has been taught as a subject in schools and there arealso a large number of English materials being used In recent years, Vietnam isapplying the student-centered approach in language teaching As a matter of fact,curriculum, materials, teaching methods, and evaluation should all be designed forlearners and their needs As a result, English for Specific Purposes (ESP) materialsrelating to different fields become more and more popular

Materials determine both learning and teaching, so choosing an appropriatematerial is very important However, the more various the ESP materials are, themore difficult it is to choose an appropriate material for the teaching and learning.Different materials have different strengths and weaknesses, so evaluation is anecessary process to help teachers also learners select a good material to follow.Therefore, evaluation plays as a key role in educational research and development As

a matter of fact, English materials used to teach the non- English major students atHaUI have never been evaluated Although at the beginning of each new school year,there are always some activities for clarifying aims of the textbooks and the targetresults of teaching and learning, many parts in the lessons do not match with HaUI’steaching and learning condition Many students complaint that while they expect toimprove their four English skills - listening, speaking, reading and writing, Englishmaterials used as ESP material for the ESP course of Tourism department at HaUI “

English for International Tourism” focuses much on speaking and listening In

addition, the last year students are required to take TOEIC test (which consists ofreading and listening only) and get 450 TOEIC score to graduate from the university.These concerns arising from the researcher’s teaching experience, which encourages

the researcher to carry out the following research: “To what extent does the “English for International Tourism (pre-intermediate)” satisfy the requirements of an English language course at HaUI in terms of aims, content and methodology?”

2 Practical background of the research

The English programs of Tourism department at Hanoi University of Industry (HaUI)

The English courses for the non-major students at HaUI are divided into fourstages: In the first year (the first two semesters), they have to complete 120 forty-

five minute periods of general English with the course book “New elementary” written by Liz and John Soars In the second year, they have other 120 forty-five minute periods of general English with “New Headway-pre-intermediate”

Headway-written by Liz and John Soars In the third year, they had 120 forty-five minute

periods with the books “Real listening and speaking 2” and “Understanding English pronunciation” For the last year, the students need to finish 75 forty-five minute periods with the book “English for International Tourism- Pre Intermediate” written

by Dubicka and O’Keeffe (2003)

The English teaching material

For the last year students of tourism department at HAUI, “English forInternational Tourism Pre- Intermediate” written by Dubicka and O’Keeffe (2003)has been used as the principal material since 2012 It contains 15 units; each unit isorganized by a set pattern of components: language focus, vocabulary, andprofessional practice The first component, language focus, introduces some usefulgrammatical structures In the vocabulary section, a number of words on differenttopics are given in reading, listening and speaking tasks In addition, the students alsohave to practice pronunciation with sounds, word stress, ending sounds or intonation

In the last component, professional practice, the students are offered opportunities topractice in situations related to tourism

3 Aims of the study

The main purpose of this study is to evaluate the ESP material “English for International Tourism Pre- Intermediate” to determine whether it satisfies the

requirements of the ESP course at HaUI in terms of aims, contents and methodology,which implies what changes to be made to improve the effectiveness of the materialfor future use

4 Research question

To achieve the aims of the study, the answers to the following question were

sought: To what extent does the book “English for International Tourism” satisfy the

requirements of an English language course at HaUI in terms of aims, contents andmethodology?

5 Scope of the study

Evaluating a material relies on a number of criteria such as audience, aims,content and so on Given that the target learners and the pedagogical conditions atHaUI are clearly specified, for this study, the researcher focuses on the followingcriteria:

- Aims of the book

- Methodology (in terms of tasks and teaching - learning technique)

Due to the limited scope of a minor thesis, the researcher will invite 60 lastyear students to fulfill the survey questionnaires, 7 teachers (4 from the Englishdepartment and 3 from the Tourism department) for interviewing (See more in theChapter 2 - Methodology)

6 Methods of the study

The study applies both qualitative and quantitative methods In order toachieve the research’s aims, document analysis, questionnaires for the students andinterview to teachers are also exploited

7 Organization

This thesis includes three parts: Introduction, Development, and Conclusion.Part A: The introduction provides the rationale, the practical background, the aims,the research question, the scope, the methods and organization of the thesis Part B:The development consists of three chapters:

Chapter one: Literature Review begins with an overview of ESP in language

teaching, and then presents major issues in materials evaluation, including definition

of materials evaluation, reasons for materials evaluation, types of materials evaluationand approaches to materials evaluation Research instruments employed in materialsevaluation are also discussed The chapter ends with a picture of previous works, bothabroad and in Vietnam, on the topic

Chapter two: Methodology - describes the choice of methodology adopted in

the study, the data collection instruments, the subjects, as well as the data collectionprocedures

Chapter three: Data Analysis and Discussion - discusses the findings of the

study, pointing out the strengths and weaknesses of the material

Part C: Recommendations and Conclusion - suggests solutions to improve thematerial, as well as directions for further research and concludes the study

PART B: DEVELOPMENTCHAPTER 1: LITERATURE REVIEWThe first part of this chapter is an overview of the ESP, theories concerning theevaluation of the ESP material In the first part, an overview of the ESP course isintroduced The next part defines materials evaluation and discusses reasons formaterials evaluation, types of materials evaluation, as well as approaches to materialsevaluation The chapter ends with the review of some previous evaluation projects.

1.1 An overview of ESP

The growing demand for teaching and learning of English for SpecificPurposes (ESP) often comes from groups of learners who have already completed ageneral course and want to learn English for particular reasons connected to theirstudies and their jobs However, there is still confusion between teaching English forSpecific Purposes (ESP) and teaching specialist subjects due to the lack ofunderstanding what ESP means and what characterizes the ESP For this reason, someconcepts related to ESP should be made clear before the designing of an ESP course

1.1.1 Definitions of ESP

There are various definitions of ESP made by researchers with focuses on theelements that characterize ESP Focusing on the primary needs in ESP related to thelearners, the language required and the learning context, Hutchinson & Waters (1987)see ESP as an approach, but not as a product While discussing what is criteria toESP, Robinson (1991, p.3) points out that an ESP course is based on a needs analysis,which aims to specify as closely as possible what exactly it is that students have to dothrough the medium of English Actually, learners’ aim of learning ESP is not onlyfor a general examination but also for their own study The learners of ESP usuallystudy to perform a role In other words, they are considered to be successful in ESP ifthey can perform convincingly a role and works well in their job

All in all, the authors above agree that ESP can be seen as an approach toteaching which specifies what the learners need and take the learner’s needs intoconsideration Therefore, in my opinion, an ESP course also should be designed insuch a way that the teachers have to take much concern on the learners’ needs so thatthe learners are able to study English purposefully and efficiently

1.1.2 Types of ESP

There are many types of ESP A major distinction is often drawn between EOP(English for Occupational Purposes), EAP (English for Academic Purposes), and EST(English for Science and Technology)

In discussing EOP, Kennedy and Bolitho (1984) point out:

English for Occupational Purposes (EOP) is taught in a situation in which

learners need to use English as a part of their work or profession Instances ofEOP students would be doctors in casualty or technicians servicing equipment.They need English in the first case, to talk and respond to patients and otherstaff, and in the second, to read technical manuals (p.5)

English for Academic Purposes (EAP) is taught generally within educational

institutions to students in need of English for their studies The language taught may

be based on particular disciplines at high levels of education when students arespecializing (in - study) or intends to specialize (pre- study) in a particular subject.For example, an overseas student studying for a high degree at a British Universitywill need to master language skills such as listening to lectures, taking notes, writingreports, and reading text books so that these skills will help him understand thesubject matter easily and precisely

English for Science and Technology (EST) is an important aspect of ESP program.

The term EST presupposes a stock of vocabulary items, grammatical forms, andfunctions which are common to the study of Science and Technology Below is a

classification of ESP quoted by Kennedy from Strevens (1977)

To sum up, a large number of various types have painted an overall picture ofthe main groups of learners who certainly take much concern of teachers Thediagram shows the first step for the teachers to analyze and design an ESP course.1.2 Materials evaluation

1.2.1 Definitions of materials evaluation

Materials evaluation is one of the key components of the process of curriculumdevelopment So far, a number of definitions of materials evaluation have been made

by different researchers They all have tried to provide a more comprehensive andinclusive definition

Materials evaluation is a process with "attempts to measure the value ofmaterials" or "the systematic appraisal of the value of materials in relation to theirobjectives and to the objectives of the learners using them" (Tomlinson, 1998:3) Inother words, materials evaluation can be considered as a principle process ofproviding useful information about the targeted materials in order to select anddevelop them in a reliable and valid way

Material evaluation is a complete process "which begins with determiningwhat information to gather and ends with bringing about change in current activities

or influencing future one" (Dudley-Evans and St John, 1998:128)

In general, materials evaluation involves the determination of what needs to beevaluated, the objectives and requirements of the materials and the judgments of thevalue of the materials being evaluated in relation to the objectives and requirementsdetermined

1.2.2 Reasons for materials evaluation

There may be various reasons why materials should be carefully evaluated.According to Cunningsworth (op cit.), for example, there are two reasons forevaluation: (1) the intention to adopt new coursebooks; (2) identifying the points ofstrengths and weaknesses

According to Ellis (1997), firstly, there may be a need to select among theresources available the most suitable materials to use for particular group of students

in a particular context Secondly, there can be a need to determine whether theselected or designed material works well to fulfill the intended purposes after beingused for a period of time

In short, the purposes of materials evaluation are to identify the strengths andweaknesses of the materials to select or develop the materials Because materials are

considered as a key component of the language-teaching program, it is obviouslynecessary that materials be evaluated periodically so that they do not become stalewith the particular program involved In this research, materials evaluation aims atpointing out the strength, weaknesses of the materials and also the appropriateness ofusing the material in teaching at HaUI.

1.2.3 Types of materials evaluation

Based on the discussion of various scholars, Robinson (1991:59) classifiesmaterials evaluation into three types: preliminary, summative and formative The firsttwo are more likely to focus on published or at least completed materials and the thirdfocuses on in-house materials

According to Robinson (1991), preliminary evaluation takes place before anESP course begins and involves selecting the most appropriate from the publicationsthat are available It is obviously useful in this case to have some sort of checklist offeatures which one wants the textbook to have and to rank these in order of priority

In contrast to preliminary evaluation, summative (performance evaluation)takes place at the end of the course and addresses the question of whether thematerials have been effective Other methods include use of a checklist and use ofquestionnaires

Formative or revision evaluation is conducted while the course is on-going, sothat modification can be made to the materials Revision evaluation of languageteaching materials has received very little attention in the past, and relatively little isknown about the usefulness of certain techniques This still seems to be the case Themain techniques used with performance evaluation are tests and questionnaires, plusinterviews

Ellis (1997) introduces two new terms related to types of materials evaluation:predictive and retrospective Predictive evaluation is design to make a decisionregarding what materials to use while retrospective evaluation examines the materialsthat have actually been used

Recently, Tomlinson (1998:3) has referred to three notions; pre-use, while-useand post-use when classifying types of materials evaluation Pre-use evaluationpredicts whether or not materials will work While-use evaluation examines howmaterials really work in the classroom and post-use evaluation aims to find out howactually effective they are

All in all, although different terms are used to indicate the types of materialsevaluation, they are basically similar Of these evaluation types, the formative andsummative materials evaluation can only be meaningful determined after a period ofclassroom use In this study, the researcher uses formative or revision evaluation toevaluate the book “English for International Tourism” while it is on-going at HaUI.1.2.4 Approaches to materials evaluation

Because of the importance and necessity of materials evaluation, an urgentquestion has been raised: How to evaluate materials systematically and accurately?

So far there have been different approaches of materials evaluation proposed bydifferent authors Each one has its own strengths and weaknesses

According to Cunningsworth, evaluating the material’s language contentmeans finding out what aspects of language system (form, function, etc), languageskills, and language styles are taught in the material as well as the social context

Checking the presentation and practice of new language items means findingout how the new grammatical and lexical items are presented and practiced in thematerial as well as the comments on these presentation and practice

As for developing language skills and communicative abilities, it is necessary

to find out the comments made on the material about the nature of reading texts orlistening material with accompanying exercises or natural of writing exercisesincluded in the material Moreover, activities carried out during the lesson such asactivities for free production of spoken English or integrating language skills mustalso be taken into account

1.2.4.2 McDonough and Shaw’s approach

McDonough and Shaw (1993) suggest that materials evaluation can be carriedout in two complementary stages called external and internal stages The purpose ofthe first stage - external evaluation - is “to gain an overview of the organizationprinciples involved The evaluating criteria for this first stage are intended audience,proficiency level, language context, presentation of language

items, role and availability of teachers” books, table of content, inclusion ofvocabulary list, use of visuals and presentation, cultural specificity, and so on.

After the first stage, the authors suggest moving on to a detailed internalevaluation which examines such criteria as: the treatment and presentation of theskills, sequencing and grading of the materials, types of macro skills (reading,listening, etc) contained in the materials appropriateness of tests and exercises, self-study provision, and teacher - learner “balance” in use of the materials

The purpose of the second stage is to see how far the materials in questionmatch up to what the author claims and to the aims and objectives of a given teachingprogram Based on the analysis of the claims made on the above criteria by concernedpeople like the author, the publisher, teachers, students, sponsors, etc, teachers’conclusions regarding the suitability of the materials for specific groups orindividuals can then be reached

From figure 2, it can be seen that if the external evaluation shows that thematerials are potentially appropriate, the internal stage will be carried out, which istime-saving If the external stage show that the materials are inappropriate, it is notnecessary to conduct the next stage However, when the claims made by the authorare not detailed enough, an internal evaluation also needs to be carried out though theexternal stage shows that the materials are not appropriate

aspects of design and aspects of publication The next stage is matching andevaluation which helps the evaluator to figure out how appropriate the aspects ofdesign and aspects of publication to the target situation of use are In the action stage,what can be done to improve the materials such as materials adoption, reject,adaptation, supplementation, etc will be carried out.

Analysis of the target situation of use Materials analysis

The course (proposed aims, content, * What is required of users?methodology and means of evaluation) * What is implied by their use?

Aspects of publication

Match and evaluationHow appropriate are the aspects of design andaspects of publication to the target situation ofuse?

ActionAdopt the materialsReject the materialsAdapt the materialsSupplement the materialsMake the materials a critical object

Figure 3: The materials evaluation model proposed by Littlejohn (1998:3)

1.2.4.4 Hutchinson and Waters’s approach

Hutchinson and Waters (1987) divide the evaluation process into four majorsteps: Defining criteria; Subjective analysis; Objective analysis; Matching

In the first step, the criteria for evaluation are determined In other words, it ismaking clear from which perspectives the material will be looked at Obviously,different evaluators with different evaluative purposes will need to use different sets

of criteria as discussed in the previous part Thus, the selection of criteria would

focus much on these features so as to make the evaluation more accurate andpertinent to the given context.

The second step - subjective analysis - is the analysis of the course in terms ofmaterials requirements under the defined criteria As suggested by Hutchinson andWaters (1987) the course can be analyzed in terms of its learners (age, English level,educational background, interests, etc) and its objectives Then from the results therequirements of the material are specified in terms of content and methodology Withsuch types of learners and to meet such goals, the material should follow what kind oflanguage description, cover what language points and skills, in what proportion, inwhat sequence, etc Also from the analysis of the course’s learners and objectives, therequirements of the material’s methodology are made clear, for example, the kinds oftasks and exercises needed, kinds of teaching techniques to be used, aspects oflearners’ attitudes to and expectation about learning to be taken into consideration,etc In other words, the subjective analysis brings a detailed description of the idealmaterial for the learners and the course

The next step - objective analysis - is the analysis of the materials beingevaluated under the defined criteria Once again the questions about the targetlearners, aims, content and methodology are answered not for the sake of courseanalysis but for the sake of the material analysis (What exactly are there in thematerial? What are its actual aims? What are there in its content? What teaching

methodology is used in the material) From the answers to these questions, therecomes a detailed description of the material.

In the last step of the process the two analyses are put into comparison to findout whether they match up to each other and if not, to what extents they do not match

up Based on the results, good aspects as well as problematic parts of the material can

be identified, which may suggest necessary modifications of the material to make it asclose to the ideal material as possible Figure 4 can illustrate the process:

1.2.5 Summary

The four approaches presented above suggest four options to carry outmaterials evaluation Certainly, which model to choose depends upon the evaluators'purposes

McDonough and Shaw’s procedure comprises the external and internal

evaluation, which urges evaluators to use their model to select appropriate teachingmaterials through the external and internal evaluation Whereas the externalevaluation provides an overview of the materials organization, the internal evaluationperforms an in-depth investigation in the materials to analyze the extent to which thematerials match the aims of the materials or the objectives of a given teachingprogram

Littlejohn’s approach requires evaluators to both examine different sections of

the materials and move through three different levels of analysis His model proposes

an open procedure in which the evaluators can investigate the internal character of thematerials and the situation in which they will be used, making their own personaljudgment and act accordingly Consequently, the model helps administrators; teachersand learners take more control over the materials

Hutchinson and Waters (1987) suggest the model for materials evaluation that

is composed of subjective analysis and objective analysis The most distinguishingpoint in the model is the clear-cut description of the stages and criteria which arepresented more systematically than McDonough and Shaw's working model and moresimply than Littlejohn's framework It can help evaluators to visualize what must bedone to analyze the materials in comparison with the course requirements Inaddition, the guidelines offer a straightforward analytical matching process to thosewho are going to do evaluation research In the ESP

teaching and learning context, this model proves to be more appropriate than theothers.

For this reason, the approach of Hutchinson and Waters (1987) will be applied

in this study However, since the target material was designed for students ofTourism, its audience is rather clearly defined The evaluation, therefore, put morepriority on material’s aims, content and methodology

1.2.6 Criteria for materials evaluation

Evaluation means making judgment, so there must be checklists forcomparative purpose Checklists for materials evaluation determine whichperspectives of the materials should be taken into consideration Up to now differentchecklists have been developed in order to make the task of evaluating materials moreefficient and accurate Brown (1995) emphasizes that whether materials aredeveloped, adapted or adopted, thorough study is very important and necessary todetermine the suitability of the materials for a course Thus he shows a very helpfulchecklist for evaluating textbooks used in a course The most crucial criteria listed inthe checklist are the fitness of the textbook to the curriculum of the program includingapproach, syllabus, general language needs, goal and objectives, and languagecontent, its logistical characteristics containing price, usability, and availability, andits teachability including teachers’ editions, review, and acceptability among learners.The evaluators should begin considering the extent to which the materials agree withthe overall approach and syllabus of the curriculum Next, s/he can focus on thedegree to which the materials match the language needs of students in a general way.After that, the specific language needs and situation needs of the students should beconsidered After considering the most important criteria of the textbook, theevaluator can go on to examining their logistical characteristic and teachability

Another checklist for evaluation and selection of course books also given byCunningsworth (1995) He suggests examining such aspects provided in the textbook

as language content, topics, skills, methodology Examination of the language contentshould focus on the appropriateness of the main grammar structures and vocabularyprovided in the materials to the students’ needs As for examination of skillspresented in the materials, there should be answers to the questions about whetherlanguage skills are adequately covered and whether sub-skills are suitable to thecourse aims and the students’ level and so on According to

him, the topics of the materials should be interesting, various, able to help expandstudents’ awareness and enrich their experience, but not too sophisticated andunfamiliar within the students’ language level Except for examination of languagecontent, skills and topics, questions about methodology of the materials should befound out whether they are appropriate to the learning and teaching situation of theprogram, what techniques, aids are used for presenting new language items, howdifferent skills and kinds of exercises are taught.

The similar checklist for materials evaluation, which the researcher will apply

to this study, is given by Hutchinson and Waters (1987) These are more specific and

in more detail According to Hutchinson and Waters (1987) there are four maincriteria for materials evaluation which any evaluator should concern: The audience,the aims, the content, the methodology

The audience of the materials (the evaluator should check different

information about learners to find out whether the materials are suitable to thestudents’ age, knowledge of English, interest and so on

Aims of the materials (the evaluator has to check if the materials match the

aims and objectives of the course)

Content of the materials (the evaluator has to check whether the materials’

language points, macro-skills and micro-skills, topics suit the learners’ needs)

Methodology of the materials (the evaluator has to find out if techniques, aids,

guidance provided in the materials satisfy the learners and the teachers of the course

1.3 Previous studies

Until now, there have been a certain number of studies carried out byresearchers on evaluating teaching materials, many of them focus on evaluating ESPmaterials, such as those by Saeid Najafi Sarem, Hadi Hamidi, Rezvan Mahmoudie(2013), Vo Thi Anh Dao (2003), Nguyen Thi Bich Lien (2004) and Tran Thi ThuyNga (2005)

The most remarkable study on materials evaluation is that of Saeid NajafiSarem, JHadi Hamidi, Rezvan Mahmoudie (2013) In this research, the authors use

case study method to evaluate the book “ English for International Tourism Pre Intermediate” written by Dubicka and O’Keeffe (2003) for pre-intermediate students

majoring in Tourism The authors based on the evaluation checklist constructed byDaoud and Celce-Murcia (1979) to evaluate the book The checklist

analyzes and evaluates the book in five sections: subject matter, vocabulary andstructures, exercises, illustrations and physical make-up Basing on this checklist, theauthors evaluated the book in details with both the pros and cons The authorsconsulted the book with some other qualified teachers of English and piloted someactivities and exercises of the book with some students, at the same level of languageproficiency with that the book has determined, in Sharif Language Institute inAsadabad, Hamedan as well The results from the study lead the authors to thedecision of keep using the book for the students of Tourism Department in SharifLanguage Institute in the future In order to get these results, the authors worked hard

to evaluate the book in every aspects and details The strength of the study is that theauthors chose a suitable checklist for evaluation However, the authors use qualitativedescriptive analyses only in evaluating the book; or in other words, the evaluation ofthe book primarily bases on the checklist Although the authors claim that theyconsult the book with some teachers and also pilot the exercises with the students ofpre- intermediate level, there is no interview or questionnaires employed in the study

so the results of the study are not very persuasive and practical

Vo Thi Anh Dao (2003) chose the criteria proposed by Hutchinson and Waters

to evaluate ESP materials for students of Husbandry and Veterinary Science at TayNguyen University in terms of language level, aims, content and methodology Thesubjects of the study were forty second-year students and four teachers The methodsused in the study were document analysis, questionnaire for students and interviewswith the teachers The study was well-presented, evaluated the suitability of thematerials in use to the course aims, content and methodology After collectinginformation from the document analysis and the survey, she concluded that thematerials, when compared with the course requirements, were difficult for thestudents in terms of grammatical structures and lexical items However, her thesisrevealed some weaknesses Her document analysis was rather simple The interviewquestions seemed to follow the format of a questionnaire And it would be muchbetter if she had paid much more attention to examine the methodology used at theuniversity

Another study that also based on criteria proposed by Hutchinson and Waterswas carried out by Nguyen Thi Bich Lien (2004) The purpose of the study was tofind out whether the course book Lifelines Pre-intermediate was appropriate to the

program and the students' needs in terms of aims, content and methodology Thesubjects of the study were 100 third-year students and seven teachers of ForeignLanguages Department in Hanoi Junior Teachers' Training College The authorrealized that after matching, actions of adopting, rejecting, adapting, supplementingare important for a complete evaluation process So a combination model ofLittlejohn and Hutchinson was the most suitable one for her study The methods used

in the study were document analysis, questionnaire for the students and interviewswith the teachers

Tran Thi Thuy Nga (2005), in her own thesis, recommended criteria for theadaptation of the English material for the first and second-year marine students atNhaTrang navy academy after carrying out document analysis and conducting aquestionnaire survey for teachers and students, using Hutchinson and Waters'sframework for materials evaluation She concluded that the materials generallysatisfied the requirements of the course However, the materials still revealed severalweaknesses that need modification The biggest mismatch of the material in relation

to the course requirements is the inappropriate treatment of the reading component Anumber of reading texts followed by easy comprehension questions do not help thestudents to develop their reading skills The reading tasks do not offer the students theopportunity to make use of their personal knowledge, ideas and opinions to tackle thetexts The next mismatch of the material falls into the treatment of pronunciationelements including sounds, stress and intonation

The strength of Tran Thi Thuy Nga's study is that her research methodologyhelps ensure the reliability of the evaluative results through using document analysisand questionnaires

From the above-mentioned studies, it can be concluded that the major purpose

of conducting an evaluation of teaching materials is to evaluate the effectiveness ofthe material in use or its suitability to the course aims It is also found that the criteriafor evaluation may differ to some extent from situation to situation but generally theycan be grouped into the five criteria defined by Hutchinson and Waters: Audience,Aims, Content, Methodology and Other criteria It is also found out that Documentanalysis, Questionnaire and Interviews seem to be popular methods to collect the datafor materials evaluation

1.4 Summary

This chapter has dealt with different theoretical issues, which should be takeninto consideration in the work of evaluating ESP materials The instructionalapproaches to materials evaluation are the focus in this chapter Issues related to

material evaluation which work out the role of material in ELT, the definition andtypes of evaluation, reasons for evaluating material are discussed It is also madeclear in this part the evaluative criteria and process to carry out an evaluation of ateaching material It is certain that this review can help the author of this study to do

an evaluation research on materials evaluation

CHAPTER 2: METHODOLOGYChapter 2 begins with the description of the research method It then specifiesthe data collection tools, the subject, and the data collection procedures.

2.1 Data collection instruments

To evaluate the book “English for International Tourism”, the employment of

document analysis, questionnaires and interview is used

2.1.1 Document analysis

The ESP material being used by students of Tourism department at HaUI,

“English for International Tourism”, will be analyzed based on the defined criteria to

give a description of what are there in the material The material includes 15 units and

is used for the last year students of Tourism department at HaUI The analysis isbased on the criteria for materials evaluation established by Hutchinson and Waters(1987) In the first step, the author analyses the aims set in the course book as well asthe content of the book Another examination point is methodology focusing on theactivities set in the book and teaching/learning techniques suggested by the author ofthe book

The questionnaire explores students’ opinions on the Aims of the material,Content, Methodology and material improvement It employs likert scales (1=strongly agree, 2 = agree, 3 = not sure, 4 = disagree, 5 = strongly disagree) in most ofthe key questions The first six questions investigate the aims of the material Thenext part aims to find out what the students think about the difficulty level and theusefulness of the material In the third part, questions related to the teaching/learningtechniques are presented and the last part is about suggestions for the materialimprovement The language used in the questionnaires is English

because the respondents are the last year students in department of Tourism so theirEnglish is good enough to understand the questionnaires However, to avoidunexpected misunderstanding, before delivering the questionnaires to therespondents, the author went through the questions with the respondents and gavehelp when necessary.

2.1.3 Interview

Interview is employed to get information from teachers of Tourism faculty andEnglish language teachers All questions for interviewing teachers of Tourism andEnglish department are in English

2.1.3.1 Interview questions for English language teachers

The interview questions for English language teachers are composed of fourparts (Appendix 3) Part one investigates the aims of the materials in relation to thecourse aims Part two is concerned with their opinions about the content of thematerial Part three searches the information on the methodology implied in thematerials While part four asks for suggestions for the content and methodologyimprovement

2.1.3.2 Interview questions for teachers in department of Tourism

There are two questions for the interviews with Tourism teachers (Appendix4) The first questions are concerned with the teachers' opinions about the relevance

of the topics in the material while the next one asks the subjects to evaluate how to-date the content of the texts is

up-2.2 Subjects

The subjects/evaluators of the research include three groups, (1) the studentsresponding the questionnaire, (2) English teachers, (3) teachers in department ofTourism attending the interviews

The first group:

English language teachers: HaUI has a staff of 120 English language

teachers, aged from 23 to more than 40 but only four of them teach this ESP material

The four teachers have taught English for International Tourism for two years They

have already had several academic meetings discussing the issues related to the ESPsyllabus and the material; therefore they are a very reliable source of data

The second group:

The teachers in the department of Tourism: One male and two female teachers

of Tourism department, who are forty, thirty six and thirty respectively,

take part in this study The youngest teacher has three years of teaching experiencewhile the others have nine years All of them have Master degrees and have muchexperience because one also works for travel agents as a part- time job and the othersruns travel companies.

The third group:

The students: The third group of the participants consisted of 60 last year

students of Tourism department These students are from two classes which theresearcher teaches, their ages range from twenty one to twenty four and most of themare females In the first year (the first two semesters) they completed 120 forty-five

minute periods of general English with the course book “New Headway-elementary” written by Liz and John Soars In the second year, they had other 120 forty-five minute periods of general English with “New Headway-pre-intermediate” written by Liz and John Soars In the third year, they had 120 class hours with the books “Real listening and speaking 2” and “Understanding English pronunciation” By the time

of this study, the students are going to finish 75 class hours with the book “English for International Tourism- Pre Intermediate”.

2.3 Data collection process

At first, some copies of the questionnaires were distributed to pilot subjects tocheck their suitability with students and with the research purposes As all the pilotsubjects understood the questions and gave complete answers, copies of thequestionnaires then were distributed to the 60 students in class Tourism 1.1 andTourism 1.2 The questionnaires were delivered to the students at the beginning of theclass hour on May 20th, 2014 They were asked to return the questionnaire in the nextlesson on May 22nd, 2014 After getting the questionnaires, the students wererequired to read through all the items and identify any points causing comprehensionproblems and encouraged to raise questions, then the researcher provided explanation

On the same day (May 20th, 2014), the four English teachers were interviewed

to give their opinions on the material The interviews were taken place at the Englishdepartment’s office after class hours The interviews were also carried out separatelywith two teachers of Tourism department They were invited to “Café Việt” near

HaUI where the researcher provided them with “English for International Tourism” and a short introduction of the content of the book This was intended to prepare them

better for the coming interviews The data from the

interviews together with the data collected from the material analysis and thequestionnaires was then analyzed and discussed to give a full and detailed evaluation

of the material

The researcher decided to use Vietnamese as the language of communicationwith the teachers in department of Tourism so the interviewees would feel morecomfortable when expressing their opinions about the material During the interviews,the interviewer took notes the answers of the interviewees The interviews were notrecorded because recording devices may cause hesitation among participants

2.4 Summary

This chapter has given a detailed description of the subjects of the study,means of data collection and procedure of data collection on formative evaluation.The study involves four groups of subjects: (1) the last year students of Tourismdepartment who has used the material; (2) teachers of English, who have been usingthe material, (3) the teachers of Tourism department and (4) the material itself Thequestionnaires, interviews and analysis are all based on the criteria defined in thechecklist, which consists material’s aims, material’s content and material’s

CHAPTER 3: DATA ANALYSIS AND DISCUSSION

This chapter presents and discusses the results from the document analysis,teachers' interviews and students' questionnaires The data is classified, analyzed andthen discussed in terms of the Aims, Content, and Methodology

3.1 The appropriateness of the material to the aims of the course

3.1.1 Aims of the course

The aims of the ESP course for the students of Tourism department at HaUI asstated in the curriculum is to provide students with knowledge to apply incommunication situations of Tourism such as providing guests with informationabout tours, answering phones, writing responds to the guests, arranging tours, etc.3.1.2 English teachers’ interview results

When asked about the aims of the material, all the four English languageteachers at HaUI shared the opinion that the material aimes at providing the students

with the opportunity to practice basic grammatical structures (present simple, present continuous, model verbs, etc) in context and vocabulary related to Tourism (fly-drive holiday, cruise ship,…) All of four English teachers agreed that listening and speaking are the focus of the material as it is said in the Introduction of the book

“English for International Tourism” However, two out of four teachers thought thatthe book has not helped the students improve their listening and speaking skillsbecause of long listening and difficult speaking tasks The other two teachers had theidea that the students have chances to practice listening and speaking tasks with life-like situations

In short, the teachers agreed that aims of the material, to some extent, matchesthe aims of the course, which focuses on helping the students improve their speakingand listening but relevant modifications are needed Reading tasks should be paidmore attention so that the students can improve their reading skills

3.1.3 Students’ questionnaire result

The items in the questionnaires require the students to indicate the extent towhich they agreed or disagreed with the effectiveness of the material in relation to thecourse aims The number of copies delivered was 60; the number of copies receivedwas 58 Two copies were not collected because the students were absent on the daythe researcher collected the questionnaires

The aims of the Strongly

1 The material offers

practice reading skills

5 The material offers

material was successful in providing the opportunity to learn grammar in context.Moreover, 100% of the respondents agreed that they could enrich their vocabularyrelated to tourism However, a large number of students disagreed that they

23

improved their listening and speaking through the tasks in the book, only 8,6% of thestudents thought that they could improve their listening and only 5,2% of them agreedfor speaking However, nearly 100% of the students strongly agreed and agreed thatthe book offered them opportunity to practise life-like situations related to tourism Avery high percentage of the students (86,2%) thought that they did not haveopportunity to practise reading skills because in their opinions, the reading tasks arenot diverse.

3.2 The appropriateness of the material to the content requirements of the course

3.2.1 Material analysis results

The material analysis will be presented under Organization, Skills and Topics

The content is organized around topics of tourism with fifteen units such as all

in a day’s work, fly- drive holiday, table for two, city tours, water cities, cruise ships… In each unit there is language focus which includes many grammar structures,

vocabulary which provides words in exercises and reading texts, professional practicewith functional language which provides frameworks for key customer serviceencounters Moreover, reading skills, speaking skills and writing skills are integrated

in three main sections above After each five units, there is a consolidation whichreviews language, specific vocabulary and also offers extra skill practice At the end

of the book, there is a writing bank which offers detail instructions of writing somewriting types and sample writings Besides, there are pair work files with role playtasks, grammar reference with structures and grammar rules However, in every unit,sections and tasks do not follow a fixed order For example, in the first lesson wecome across with Speaking, Reading and Vocabulary while in the second lesson wehave Speaking, Listening and Reading

“English for International Tourism” is specially used for the students who are

interested in tourism; therefore the author of the book focuses on developingcommunicative competence of the students rather than grammatical competence

However, in each unit, there is a section named language focus presenting

grammatical structures and grammar is always presented in context and extensivelypracticed speaking Reading and writing skills are paid little attention while listeningand speaking skills are the focuses of the book In each unit, there are at least twolistening and speaking tasks Moreover, each unit consists of professional

practice with authentic tasks, pronunciation with sounds of English and vocabularysections also help much for speaking and listening skills.

The treatment of topics in the material is appropriate in comparison with the

course requirement As the material “English for International Tourism” is especially

designed for students specialized in tourism, topics are all related to tourism such as

Fly-drive holiday in unit 2, City tours in unit 4 or Winter holiday in unit 12 The

topics deal with specialist knowledge relevant to the students’ subject matter To take

Unit 4 as an example, this unit presents exactly the subject matter areas needs for

students of Tourism in which students have to acquire such skills as give directions,give a Guided Tour and Describe a Building Thus, it can be seen that the treatment ofESP topics meets the requirement of the course

3.2.2 The content requirements of the course

According to the syllabus designer, the ESP English course at HaUI aims toequip the students with grammatical structures and vocabulary which are often used

in Tourism Moreover, throughout the book, the students can develop listening,speaking, reading, writing skills and also interpreting skills After the course, thestudents can use their knowledge in situations in Tourism such as providinginformation of tours, answering telephones, arranging tours, etc However, thecontent requirements in the syllabus are taken totally from the table of content in thebook There is no specific syllabus given to the teachers so the content requirements

in the syllabus are not very clear In short, at a glance, the book “English for International Tourism” seems to be perfect for the teaching and learning English at

department of Tourism at HaUI but there are some mismatches inside

3.2.3 Teachers’ interview results

This section presents the teachers' opinions on the material content categorizedinto Organization, Skills and Topics

Giving opinion about the organization of the materials, all the teachers agreedthat the book is sensibly organized around the subject matter Each unit shows clearfocus on listening and speaking skills with at least two listening and speaking taskswith communicative practice and vocabulary which facilitate the listening andspeaking tasks effectively

As far as skills are concerned, all the four teachers expressed dissatisfactionwith the reading skills provided because of the absence of important reading skillssuch as understanding author’s attitudes or differencing All of the subjects felt that

it is difficult for the students to develop speaking skills as well As the teachers said,there are just some guides for expression in the book while there are a lot of things to

do before the students can do the speaking tasks For example, there should be somemodel expressions in dealing with the guest’s complaints, taking the booking tours,answering the phones, etc Two teachers interviewed had the idea that more listeningwith the content related to the speaking tasks should be included in each unit In otherwords, more skill-integrated work needs to be taken into consideration As for thewriting sections, all teachers agreed that the writing skill were not paid muchattention in this book; however, the writing tasks were interesting with practical taskssuch as writing a CV, writing a fax, writing a letter of apology

As for the topics, all the teachers thought that most topics in the book met thelearners’ interests and proficiency level Moreover, according to the teachers, thesubject matter was presented topically with communicative practice such as hiring acar, planning a holiday, dealing with complaints, etc, which students most probablyencounter and need to know in everyday world of tourism However, three out of four(3/4) teachers thought that some topics were not very suitable with Vietnamese

situations Take unit 12 as an example, this unit is about Winter holiday providing the

students with ski equipment, ski resorts which are not popular in Vietnam

3.2.4 Students’ questionnaire results

so they complained that they hardly caught all the key ideas to complete the tasks.Many of the students (43,1%) found the speaking tasks complex while 25,9% thoughtthe speaking tasks were not complex at all and the

other students (31%) were not sure whether the tasks complex or not Those 31% ofthe students commented that some speaking tasks with clear instructions and familiartopics were simple but the others were not Some other students claimed that they didnot have enough time and vocabulary to practice the speaking as required.

The difficulty of the Strongly

1 The grammatical

27