An evaluation of the effectiveness of the materials on english for tour guides for the 3rd year students of foreign languages and tourism department at sao do industrial college and suggestions for adaptation

UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES DEPARTMENT OF POST-GRADUATE STUDIES  NGO THI MY BINH AN EVALUATION OF THE EFFECTIVENESS OF THE MATERIALS ON “ENGLISH FOR TOUR GUID

UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

DEPARTMENT OF POST-GRADUATE STUDIES



NGO THI MY BINH

AN EVALUATION OF THE EFFECTIVENESS OF THE MATERIALS ON

“ENGLISH FOR TOUR GUIDES” FOR THE 3 RD YEAR STUDENTS OF FOREIGN LANGUAGES AND TOURISM DEPARTMENT AT SAO DO INDUSTRIAL COLLEGE AND SUGGESTIONS FOR ADAPTATION

Đánh giá hiệu quả của giáo trình Tiếng Anh chuyên ngành H-ớng dẫn

Du lịch cho sinh viên năm thứ 3 khoa Du Lịch – Ngoại ngữ tr-ờng Cao

đẳng Công nghiệp Sao Đỏ và gợi ý cho việc hiệu chỉnh

M.A Minor Thesis

UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

DEPARTMENT OF POST-GRADUATE STUDIES



NGO THI MY BINH

AN EVALUATION OF THE EFFECTIVENESS OF THE MATERIALS ON

“ENGLISH FOR TOUR GUIDES” FOR THE 3 RD YEAR STUDENTS OF FOREIGN LANGUAGES AND TOURISM DEPARTMENT AT SAO DO INDUSTRIAL COLLEGE AND SUGGESTIONS FOR ADAPTATION

Đánh giá hiệu quả của giáo trình Tiếng Anh chuyên ngành H-ớng dẫn

Du lịch cho sinh viên năm thứ 3 khoa Du Lịch – Ngoại ngữ tr-ờng Cao

đẳng Công nghiệp Sao Đỏ và gợi ý cho việc hiệu chỉnh

M.A Minor Thesis

CHAPTER II: EVALUATION ON THE EFFECTIVENESS OF THE

2.1 Method of data collection and procedures 19

2.1.1.2 Description of the survey questionnaires 20 2.1.2 Formal interview to teachers, students and class observations 20

2.2.1.1 The evaluation of the teachers in terms of audience, aims, content,

2.2.1.2 The evaluation of the teachers in terms of speaking lessons 24 2.2.1.3 The suggestions of the teachers in terms of speaking lessons 24

2.2.2.1 The suitability of the course to the audience 25 2.2.2.2 The suitability of the objective’s course in terms of skills 27 2.2.2.3 The suitability of the course’s content in terms of the speaking lessons 28 2.2.2.4 The suitability of the speaking lessons in terms of methodology 29 2.2.2.5 Students’ suggestion for the speaking lessons 30

3.1.1 Suggestions for the adaptation of speaking lesson in Unit 2: Hai Duong’s

3.1.5 Suggestions for the adaptation of speaking lesson in Unit 10: Ha Long Bay 39

LIST OF FIGURES AND TABLES

Figure 1 The Materials Evaluation Model of Hutchinson and Waters (1993:98) 13

Table 2.1 Students' results of learning tourism English in the previous term 25

Table 2.2 Students' attitudes towards an “English for Tour guides” course 26

Table 2.3 Students' expectations from an “English for Tour guides” course 26

Table 2.4 Students’ opinion on skill’s importance 27 Table 2.5 Students’ opinions on their skills need to improve in the course 27

Table 2.6 Students’ opinion on the suitability of the speaking lessons 28

Table 2.7 Students’ opinion on the methodology’s suitability of the speaking

Table 2.8 Students’ suggestion for the speaking lessons 30

LIST OF ABBREVIATIONS

SIC Sao Do Industrial College

FLTD Foreign Languages and Tourism Department ETP English for Tourism Purpose

EFT English for Tour guides

ESP English for specific purposes

ELT English language teaching

PART A: INTRODUCTION

I The rationale and aims of the study

There are three reasons why I choose to carry out the research called “An Evaluation of the Effectiveness of the Materials on “English for Tour guides” for the 3 rd year students of Foreign Languages and Tourism Department at Sao Do Industrial College and suggestions for adaptation”

The first reason is in the importance of a material or a text book in the process of learning and teaching foreign languages

ELT materials in general and textbooks in particular serve as tool and tutor, guidebook and gauge The American Association for Supervision and Curriculum Development asserted that teachers throughout the world use texts to guide their instruction, so textbooks greatly influence how content is delivered Schmidt, McKnight, and Raizen (1997) identified textbooks as playing an important role in making the leap from intensions and plans to classroom activities, by making content available, organizing it, and setting out learning tasks in a form designed to be appealing to students Therefore, they may be considered

“the visible heart of any ELT program” Sheldon (1988:237) Especially in Vietnamese context, textbooks are regarded as a staple in almost every EFL class

With the growth of computer-assisted language learning, the role of this material may be changing, but it is unlikely that it will ever disappear Therefore, it is necessary for teachers

to be well-equipped with the assessment tools to evaluate materials to ensure that students are using the highest quality texts possible and that their language learning experience is enhanced, not hindered, by the book used in their classrooms However, there has been surprisingly little investigation done in terms of how and why materials are selected by teachers Moreover, most of them have not been adequately trained in the selection process and may be unaware of how to judge which texts best suit their situation That research gap has propelled me to choose materials evaluation as the theme for this master thesis

The second reason for this option lies in the fact that in recent years there have emerged diverse opinions of both the learners and teachers on the actual effectiveness of the materials “English for Tour guides”- an in-house materials used in teaching ESP for the third year students at FLTD at SIC in Hai Duong Arguments have encompassed both the potentials and the limitations of the materials for guiding students through the learning

process as well as the needs and preferences of the teachers and learners who are using it Other issues that have lately arisen include the textbook content, the methodology validity, the role of textbooks in innovation, the authenticity of materials in terms of their representation of language and the appropriateness of some speaking lessons It is essential, therefore, that we should establish and apply a wide variety of relevant and contextually appropriate criteria for the evaluation of the textbook to assess its appropriateness against the learning purposes

The last but not least, being a teacher of foreign languages and having taught the course of

“English for Tour guides” at SIC more than two years, I have found out that there are some problems with the materials of the course which are needed to be adapted With the hope

of improving the quality and minus the limitation of the materials to benefit both the learners and teachers, I conduct this research to discuss and describe the evaluation process that is to be undertaken at the FLTD at SIC of the textbook “English for Tour guides” used

in this particular learning environment and thus lead to further thought on improving the quality of the materials by suggestions for the textbook’s adaptation especially in speaking lessons These are also the aims of the study

II Scope of the study

The study is about the evaluation of effectiveness of the in-house materials “English for Tour guides” - Part IV – Tourism destinations in Vietnam Its curriculum consists of 120 periods for the third year Tour guide students of FLTD at SIC In the scope of this study, the evaluation of the effectiveness of the materials only focuses on evaluation of the suitability of the materials’ content and methodology to the course’s requirement and students’ need It bases on the evaluation of teachers and students at the school From these, I give out suggestions for adaptation especially in speaking lessons to improve effectiveness and learning motivation for Tour guide students and better serve for teaching

job this subject at SIC

III Methods of the study

To achieve the above aim, the study employs qualitative and quantitative research method The research methods employed were document analysis, survey questionnaire, informal interviews and class observation Of these methods, document analysis was one of the main instruments with a thorough objective analysis of the textbook under evaluation

Survey questionnaire also plays an important role in the study According to James H Mc Millan & Sally Schumacher (1993) “in survey research, the investigator selects a sample of subjects and administers a questionnaire or conducts interviews to collect data Surveys are used frequently in educational research to describe attitudes, beliefs, opinions and other types of information” In this study, survey questionnaire was used to collect the data that relate to the teachers' and students' perspectives and attitudes towards the textbook they have just finished Informal interviews were employed to clarify information collected in the survey Lastly, class observation was used to get student’s responses to the speaking lessons after adaptation These are effective way of collecting data for this study

The purposes of this evaluation research were to evaluate the effectiveness of the in-house materials “English for Tour guides” in terms of its objectives, content and methodology especially the content of the speaking lessons in the materials This summative evaluation was conducted right after the teachers and students have just finished the textbook to determine whether the material has worked well The researcher employed a method which was a combination of document analysis, survey questionnaires, informal interviews and class observation The researcher's belief was that a combination of different methods to collect data could provide more reliable and valid information for the evaluation

IV Research questions

The study is to find out the answers to the following research questions:

1 How do the materials “English for Tour guides” used at SIC meet the requirements of the course and meet the demands of students in terms of objectives, content, and methodology especially the content of the speaking lessons?

2 What suggestions should be made to contribute to the improvement of the speaking lessons in the materials?

To answer these research questions, the researchers employed the following evaluation procedures:

 The involvement of the researcher as the evaluator who analyzed the materials under evaluation by conducting survey questionnaires for teachers and students to get their opinions about the textbook they have used and carrying out informal interviews with teachers and students to clarify issues which are raised in the surveys

 The findings are expected to provide convincing evidences for the contribution to the improvement of the textbooks for future use

V Statement of the problem

SIC is a famous college in Hai Duong province located in Chi Linh District The school is

40 years of foundation It has more than 500 teachers and 15.000 students It is known all over country with electrical and mechanical field FLTD is a young one- only 5 years old The number of students is only about 300 students The students related to this study are 80 students of the third year The program of English for Tourism consists of 300 periods, divided into 4 different parts In the first Year, they learn knowledge related to hotel’s work In the second year they learnt about the tourists on holiday and common knowledge related to Vietnam Helping Students to become good tour guides in the future forms the basis of the 3rd year course, with students examining topics as tourism destinations in Vietnam from the North to the South and so on

The program of part IV composes of 4 main parts: Reading is to provide students with knowledge of famous tourism destinations in Vietnam; Speaking is to provide students with sentence forms, situations happening when dealing with foreign visitors and train student’s ability of making presentation on these destinations Listening part is to train student’s listening skill; writing part focuses on guiding and training students to write about topics related to tourism Language focus helps students to pronounce the words and intonation correctly and revise some important languages This course is a basic part providing necessary knowledge for tourism students and at the same time practice effectively 4 skills listening, speaking, reading, writing, develop students’ communicative ability completely

VI Design of the study

The study consists of 3 parts: Part I: Introduction composes of rationale and aims, scope, method of the study, research questions, statement of the problem and design of the study Part II: Main content consists of 3 chapters: Chapter I: Literature review deals with Textbook, course book, and materials, Materials Evaluation, Materials adaptation; Chapter II: is about evaluation on the materials’ effectiveness and Chapter III: Suggestions for the materials adaptation

Part III: Conclusion presents a review of the study, implication, limitation and the

recommendations for the improvement of the materials

PART B: DEVELOPMENT CHAPTER I: LITERATURE REVIEW

This chapter aims to provide a theoretical base to develop an operational framework for

materials evaluation The first issue addressed in this chapter relates to teaching materials

with regards to their role in language teaching and types of materials The second part

presents the review of literature concerning major issues in materials evaluation: reasons

for materials evaluation, definitions of materials evaluation, who carries out materials evaluation, who provides data for materials evaluation, models for evaluation, and criteria

for evaluation The third part discusses some of the previous research on materials

evaluation in different contexts that the author referred This chapter also includes some

theoretical discussions about materials adaptation to help forming the recommendations

for materials improvement

1.1 Teaching Materials in Language Teaching and Learning

Materials are used in all language teaching and play an important role in the process of language teaching and learning The roles of teaching materials in language teaching will

be discussed in more details in the next part of this section

1.1.1 The Roles of Teaching Materials

Crucial roles that teaching materials play in language teaching are indicated by different authors in the literature Richards and Rodgers (cited in Nunan, 1991) view instructional materials as detailed specifications of content, and guidance to teachers on both the intensity

of coverage and the amount of attention demanded by particular content or pedagogical tasks Richards and Rodgers' perspective is supported and clarified by Richards (2001), who points out that teaching materials can serve as the basis for much of the language input that the learners receive and as the source for much of the language practice that occurs in the classroom Richards further explains that materials provide basis for the content of the lesson, the balance of skills taught, and the kind of language practice students take part in In addition, good teaching materials are of great help to inexperienced teachers or poorly trained teachers (Nunan, 1991; Richards, 2001) They can serve as "a form of teacher training" (Richard, 2001: 251) and teachers can get ideas on how to plan and teach the lesson from the materials

Obviously, teaching materials are a key and crucial component in any language teaching

contexts, the teacher needs to take account of the roles as well as the requirements of teaching materials when designing or selecting materials for his/ her own teaching situation

to facilitate the most learning from the students

1.1.2 Types of Materials

The teaching materials that a teacher uses can be a textbook, institutionally prepared materials or his / her own prepared materials (Richards, 2001) The former is referred to as published textbooks and the latter in-house materials by Robinson (1991) Which one to choose, published textbooks or in-house materials is what specialists in the field of English language teaching (ELT) weigh up arguments? Robinson (1991: 58) suggests that no other medium is as easy to use as a textbook More important, "a textbook is complete not just in the physical sense but in the sense that the whole term's or year's course is available to the students at once" However, Ewer & Boys (cited in Robinson, 1991) present a strong attack on published textbooks that some really fundamental factors of textbooks such as the validity of the linguistic contents, the accuracy of the explanations and examples given, and the number and coverage of the exercises provided have been ignored because the writers just focus on approach and methodology Moreover, as Swales (cited in Robinson, 1991: 57) indicates published textbooks are "less self-sufficient in practice materials and in coverage of skill areas" so these textbooks need to be supplemented by some other materials

Similarly, there are also arguments for and against the use of in-house materials On the one hand, as Robinson (1991) suggests, there are three advantages of in-house materials Firstly, in-house materials are likely to be more specific and appropriate than published materials and

to have greater face validity in terms of the language dealt with and the contexts it is presented in Secondly, in-house materials may be more flexible than published textbooks Finally, the writers of in-house materials can make sure of the suitability of methodology for the intended learners On the other hand, Robinson (1991) points out making in-house materials are time-consuming and expensive What is worse is that many locally produced materials "show a striking resemblance to the published materials that have been rejected" (Swales, cited in Robinson, 1991: 58)

As stated above, there are both advantages and disadvantages in using published materials

as well as in-house materials, so which one to use must depend on the purposes of the

course as well as the facilities and constraints of a particular situation Based on the literature definitions of types of materials, it can be said that the textbook under evaluation

in this research is necessary to judge the appropriateness to the intended students

1.2 Materials Evaluation

Decision on which textbook to use or whether the materials in use are suitable or not is obviously of great importance in the process of learning and teaching of language And this decision can be achieved only by the means of a comprehensive evaluation

The next part of this chapter is the review of reasons for materials evaluation, definitions, types

of materials evaluation, materials evaluators, source of the data, models for materials evaluation and criteria for materials evaluation

1.2.1 Reasons for Materials Evaluation

There are two main reasons for carrying out materials evaluation Firstly, there may be a need to choose among the materials available the most suitable ones to use for a particular situation Secondly, there can be a need for materials evaluation to determine whether the materials which has been chosen works for that situation after it has been used for a period

of time (Ellis, 1997)

An evaluation of teaching materials helps to identify particular strengths and weaknesses of the materials in use Thus it will help to determine whether the materials are effective and efficient As a result, the decision whether to use the materials again or not and what should

be changed to the material can be made for better use in the future

In short, the findings of materials evaluation will provide input for responsible people to evaluate the effectiveness of the materials

1.2.2 Definitions of Materials Evaluation

There are many ways of defining evaluation According to Brown (1995) evaluation is defined as "the systematic collection and analysis of all relevant information necessary to promote the improvement of a curriculum and assess its effectiveness within the context of the particular institution involved" This definition requires that information be gathered and analyzed in a systematic manner and that only relevant information should be included, that is other information can be ignored

Nunan (1988) suggests that evaluation is "a process not a final product" that means it takes place at any time of the material design The first and foremost emphasis of evaluation is to determine whether the goals and objectives of a language program are being attained According to Hutchinson & Waters (1993) evaluation is really a matter of judging the fitness

of something for a particular purpose "Given a certain need, and in the light of the resources available, which out of number of possibilities can represent the best solution? There is no absolute good or bad - only degrees of fitness for the required purpose"

From the above definitions, it can be inferred that materials evaluation involves the determination of what needs to be evaluated, the objectives and requirements for the materials, and the judgments of the value of the materials being evaluated in relation to the objectives and requirements determined

1.2.3 Types of Materials Evaluation

There are three types of evaluation according to the reasons for materials evaluation: preliminary evaluation, formative evaluation and summative evaluation

Preliminary Evaluation: Preliminary evaluation takes place before a course commences with

the aims of choosing the materials that will be relevant and appropriate for a particular group

of learners This also has the purpose of identifying which aspects of the published materials need to be adapted to suit the purposes of the evaluators (Robinson, 1991; Ellis, cited in Tomlinson, 1993) This can be done by ''determining a set of criteria which are used to reach a decision regarding which book to adopt and how it needs to be adapted" (Ellis, cited in

Tomlinson, 1993: 220)

Formative Evaluation: Formative evaluation is normally carried out while the course is on-going

and the findings are used to make modifications to the materials when they are being used

(Dudley - Evans, 1998)

Summative Evaluation

Summative evaluation is referred to as evaluation as retrospective evaluation (Ellis, 1997)

It takes place at the end or sometimes after the course has been implemented and very valuable for "durable course" (Dudley - Evans, 1998) According to Richards (2001), a summative materials evaluation is conducted to determine whether the materials have

worked well The findings will normally lead to the decision whether to repeat or use the materials again or not: which tasks and activities are suitable and which are not; and which aspects of the materials need to be changed and how to modify them to make the materials more effective and efficient for future use (Ellis, 1997) In other words, a summative evaluation is carried out to determine the worth and value of teaching materials after they have been used This kind of evaluation therefore can serve as a means of "testing" the validity of a before - program evaluation and it can point out "ways in which the predictive instruments can be improved for future use" (Ellis, 1997: 37)

In short, whatever the types of evaluation, preliminary, formative, or summative are, it is very important that the evaluator identifies clearly the reasons for their evaluation so that it can be beneficial for the use of the materials (Robinson, 1991)

1.2.4 Materials Evaluators

A great deal of published work on materials evaluation (Dudley - Evans & St John, 1998; Tomlinson, 1998; Robinson, 1991; Richards, 2001) reveals that materials evaluation can be carried out by outsiders (those who are not involved in the program, for example, consultants, inspectors, and administrators) or insiders (e.g teachers, students, course or materials designers); each method has its own advantage as well as disadvantage

1.2.4.1 Materials Evaluation by Outsiders

According to a number of researchers, outsiders will bring "fresh" and "objective" perspectives into the evaluation, which is the advantage of evaluation carried out by outsiders However, there may be some disadvantages: outsiders as evaluators will have their own views on language teaching methods They may not understand the teaching and learning situation in which the evaluation is being carried out So, as Robinson (1991) suggests, they may be unsympathetic and it may take more time for them to get to know the local situation such as learners' needs, facilities as well as constraints Consequently, it may be hard for them to make exact judgments and realistic recommendations; or may be what they find has already been found (Dudley - Evans & St John, 1998)

1.2.4.2 Materials Evaluation by Insiders

Materials Evaluation by insiders has some advantages especially if the overall purpose of evaluation is for "development" and thus for better use in the future As they have already

involved in the program, they will understand the teaching and learning situation well (Tomlinson, 1998) Tomlinson’s view is strongly supported by Holliday (cited in Dudley - Evans &St John, 1998) and Richards (2001) According to Holliday (1998:296), the evaluator's understanding of "cultural and political factors" of the institution in which the evaluation takes place would be very important for the evaluator to make exact judgments and realistic recommendations Richards (2001) holds the view that the involvement of the insider plays a very important part in the success of evaluation because "as a consequence, they will have a greater degree of commitment to acting on its results" Richards also points out insiders, especially teachers, can 'monitor' when the materials are being used, they therefore can determine if and to what extent the materials 'work' for their purposes and thus they can make modifications to improve the effectiveness of the materials However, the insiders "may be too close and involved" (Dudley - Evans and St John, 1998: 131) so the evaluation may be influenced by their teaching experience and their subjective points of views

In short, who will carry out the evaluation outsiders or insiders should be determined according to the purposes of the evaluation and to the facilities as well as the constraints of the situation in which the evaluation takes place

1.2.5 Sources of the Data

Another important issue in materials evaluation is to determine the sources from which the data for evaluation will be collected According to Lynch, (1996) evaluation data can come from a variety of sources such as students, instructors, administrators, evaluators, and others who interact with the program Among those sources, the learners are considered the key one, especially in the summative evaluation (Richards, 2001) According to Richards, the learners can provide 'evidence' of what they have learned from the course and the materials, and whether it is relevant to their needs They can also provide data for evaluation on the way the program or materials was taught

Another important source of data for evaluation is the teachers who are much interested in the suitability of the materials (Richards, 2001), they will report their experiences with the materials and what they think about them Furthermore, they can be helpful in providing practical suggestions for improvement (Robinson, 1991)

In addition to the student and teacher sources, records and documents used can also provide useful information for evaluation This source includes planning documents, statements of needs, aims and objectives of the course, teachers' lesson records, teachers and students' journal and so on (Robinson, 1991) The program documents can supply evaluators with basic information about the program They can also help suggest evaluation questions

to be pursued and later, be used to match against "subsequent developments" (Robinson, 1991: 71) Lynch (1996) points out that students and teachers' journals can be used as a rich source of information from the participants' points of view So they should be encouraged to keep daily or weekly journals of their "experiences and perceptions", which are then used as a source of information for the evaluation

As stated above, data for evaluation can come from a variety of sources However, to have reliable evaluation, it is advisable that information about the same point should be obtained from different sources (Robinson, 1991) Especially, also according to Robinson, the teachers and the learners should be involved in the evaluation because they have taken part

in designing or selecting and using the materials This idea taken from his own judgment that learners can give information of how effective the materials are for their work whereas teachers can be helpful in providing practical suggestions for improvement However, Robinson (1991) also warns that the teachers are likely to feel that the objective of the evaluation is to "put up" them, and very often it's assumed that "any resultant evaluation" document will be negatively critical" (Robinson, 1991: 69) He then advises that it's necessary for the evaluator to inform the purposes and methodology plans of the evaluation

to people who will provide information for the evaluation, especially the teachers who have used the materials

1.2.6 Models for Materials Evaluation

There are different models and criteria for materials evaluation suggested by different authors in the literature The popular models are the ones suggested by Hutchinson and Waters (1993), McDonough and Shaw (1993) and Ellis (1997)

1.2.6.1 Evaluation Model by Ellis (A Micro - Evaluation)

A micro - evaluation is the evaluation of one particular teaching task which the evaluator has a special interest in (Ellis, 1997) In this model Ellis suggests the practice of a detailed

empirical evaluation and focuses on evaluation at the task level with reference to its actual teaching-learning context The aim in this model is to identify the match between task planned and task in use He also suggests that dimensions focused on macro - evaluation (e.g approach, purpose, focus, scope, evaluators, timing, and types of information) can also

be applicable to micro evaluation This type of evaluation can only be conducted when the materials are being used in the classroom The steps involved in the evaluation model suggested by Ellis (1997) are: Choosing a task to follow; Describing the task with specification of input, procedures, language activities, and outcome; Planning the evaluation with reference to the dimensions above; Collecting information before, while and after the task was used, and what and how the task was performed; Analysis of the information collected; Findings of the evaluation and making recommendations for future teaching and Writing the report

It could be said that a micro evaluation may be more manageable than a macro-evaluation, but it is time-consuming

1.2.6.2 Evaluation Model by McDonough and Shaw (A Combination of Macro- and Micro-Evaluation)

McDonough and Shaw (1993) suggest a three-stage evaluation model, namely external evaluation, internal evaluation and overall evaluation:

- External evaluation (Macro-Evaluation) is the examination of the claims made for the whole materials by the author or publisher They can be in the forms of claims made on the cover of the book, the instructions, and the table of content The evaluation at this stage helps

to identify whether the material is potentially appropriate; if the external evaluation shows that the materials are potentially appropriate then the internal stage starts; if the findings show that the materials are inappropriate, the evaluation will be finished at the external stage

- The internal stage (Micro-Evaluation) requires an in-depth look at two or more units to examine whether the claims made by the author are the one found in the internal evaluation

- The overall evaluation of the materials is to determine the suitability of the materials by considering a number of factors such as the usability factor, the generalizability factor, adaptability factor and flexibility factor

1.2.6.3 Evaluation Model by Hutchinson and Waters

Hutchinson and Water (1997)' model is a macro - evaluation which as defined by Ellis (1997: 37) as the "overall assessment of whether an entire set of materials has worked" for

a particular situation In their model, Hutchinson and Waters (1993: 96) define material evaluation is a "matter of judging the fitness of something for a particular purpose" In this view, the evaluation is divided into 4 steps (see Figure 1)

Figure 1: The Materials Evaluation Model of Hutchinson and Waters (1993:98)

As can be seen in Figure 1, this model presents a logical procedure for material evaluation

To examine whether the materials are suitable for a certain group of learners or not, there

must be certain criteria against which our judgments or evaluation are based It is also a

must to determine the objectives or the requirements for the materials because we cannot measure the success of a particular activity or a whole set of materials if there is no clear objective for it

In summary, although these three models vary in the processes and the nature of the purposes, they all serve to evaluate whether the set of the materials is appropriate to a certain situation or group of learners An evaluator must be based on the purposes of the evaluation, time available, facilities as well as" constraints of the context in which the

evaluation takes place to decide which model to follow In the case of this study,

Evaluation Model by Hutchinson and Waters is considered to be the most suitable one and will be chosen by the writer

Define criteria

On what bases will you judge materials?

Which criteria will be more important?

Matching How far does the material match your needs?

Subjective analysis

What realizations of the criteria do you

want in your course?

Objective analysis How does the material being evaluated

realize the criteria?

1.2.7 Criteria for Materials Evaluation

Criteria are what evaluators use to "reach a decision regarding what needs to be evaluated" (Tomlinson, 1998: 220) which are the bases upon which evaluators depend on when making judgments Defining criteria for evaluation is one of the crucial issues evaluators must take into account before any evaluation can take place Criteria for materials evaluation depend what is being evaluated and why they need to be evaluated (Dudley - Evans and St John, 1998) Researchers (Sheldon, 1988; Wallace, 1998; Hutchinson & Waters, 1993; Ellis & Johnson, 1994) have suggested certain sets of criteria of which the two most popular are the set suggested by Sheldon (1988) and the one suggested by Hutchinson and Waters (1993)

1.2.7.1 Criteria Defined by Sheldon

Sheldon (1988) suggests a wide range of criteria that can be used to evaluate almost all aspects of materials The criteria suggested by Sheldon include rationale, availability, user definition, layout/graphic, accessibility, linkage, selection/ grading, physical characteristics, appropriateness, authenticity, sufficiency, cultural bias, educational validity, stimulus/ practice/ revision, flexibility, guidance, and overall value for money

12.7.2 Criteria Defined by Hutchinson and Waters

Hutchinson and Waters (1993) in their checklist for materials evaluation suggested 5 criteria for evaluation as follows:

Audience: The first criterion in materials evaluation in the audience of the materials with

regards to different information such as ages; sex; study or professional fields; status with respect to professional fields; knowledge of English, of areas of work or study and of the

world; educational background, interest and so on

Aims: Like any language course, which has its own aims and objectives, the evaluation of the

course needs to be carried out to find out if the material can satisfy those aims and

objectives of the course

Content: Content is one of the most important elements of the materials Therefore it is

necessary to measure if the content of the materials is suitable to the content prescribed in terms of language description, language points, macro - skills and their proportion, and

micro -skills The evaluation also needs to find out if the text - types of the materials are suitable to the requirements of the course In general it is very important to take into consideration the learners' subject - matters in terms of level of knowledge, types of topics, and how these topics are presented Finally, the evaluation needs to find out if the way the

content is organized and sequenced within a unit and throughout the course is appropriate Methodology: Many aspects in terms of materials methodology need to be evaluated First of

all, the evaluation has to indicate whether the theories of learning on which the course is based are in line with the ones of the materials under evaluation Secondly, evaluators need to measure whether the learners' expectations about learning are the ones the materials are intended to Thirdly, it is necessary to determine whether the kinds of exercises/ tasks included in the materials are suited to course requirements Another aspect to be evaluated

is the teaching and learning techniques: Whether the teaching - learning techniques that can be used with the materials are those required by the course and suitable to the learners Furthermore, the evaluation also needs to discover if teaching aids available for use are those required by the materials In addition, the evaluation needs to find out whether the materials supply the teachers with necessary guidance and support for teaching the course Finally, it is necessary to determine if the materials are flexible so that they can be used in

different orders to suit the teaching contexts

Other Criteria: In addition to the criteria mentioned above, some other criteria are also suggested such as the price and availability of the materials

From the two sets of criteria by Sheldon (1988) and Hutchinson and Waters (1993) presented above, it is remarkable that almost all aspects of teaching materials can be evaluated when using the set of criteria suggested by Sheldon (1988) This, however, would require a lot of time and effort, and even hard to apply because to evaluate everything at the same time is unrealistic (Dudley - Evans & St John, 1998) In reality, criteria defined by Hutchinson and Water appear to be more manageable and suitable to condition in Vietnam That is also the reason why Criteria Defined by Hutchinson and Waters are chosen for this study

1.3 Previous Research on Materials Evaluation

There have been a certain number of studies carried out by researchers to evaluate published

or in-house teaching materials, each of which may make use of different approaches to materials evaluation, they all, however, demonstrate how materials evaluation provides feedback for improvement and modification The major purpose of these studies is to evaluate the effectiveness of the materials in use or its suitability to the course aims However, there are no evaluation related to tourism field, especially there have not been any studies mention

to evaluation of the in-house materials “English for Tour guides” used for the third years students at Foreign Languages and Tourism Department at SIC

1.4 Materials Adaptation

1.4.1 Reasons for Adapting

Once the materials have been evaluated, potential problem areas can be identified: What the materials offer can not be exactly what our learners' need; The materials methodology may not match our own; Our general aims may not match the aims of the materials; The aims of

a particular lesson/unit in the materials may not match our lesson - by - lesson aims We will have to prioritize and select We may need to supplement the materials

There are 5 main ways of modifying materials: Adding, including expanding and extending; Deleting, including subtracting and abridging; Modifying, including restructuring and rewriting; Simplifying and Reordering

1.4.2 Adapting Techniques

1.4.2.1 Adding

The notion of addition is that materials are supplemented by putting more into them, while taking into account the practical effect on time allocation First, we can certainly add in this quantitative way by the technique of extending "This means that the techniques are being applied within the methodological framework of the original materials: in other words, the model is not itself changed" (McDonough and Shaw, 1993:89) We can do this in the following situation: A second reading passage parallel to the one provided is helpful in reinforcing the key linguistic features - tenses, sentence structures, vocabulary, cohesive devices - of the first text Second, more far-reaching perspective on addition of materials can be termed expanding This kind of addition is not just extension of an existing aspect of content They go further than this by bringing about a

qualitative as well as a quantitative change This can be thought of as a change in the overall system' (McDonough and Shaw, 1993:90)

1.4.2.2 Deleting or Omitting

Deletion is clearly the opposite process to that of addition As we saw in the previous section that materials can be added both quantitatively (extending) and qualitatively (expanding), the same point applies when a decision is taken to omit materials The most straightforward aspect of reducing the length of materials is subtracting

Addition and deletion often work together Material may be taken out and then replaced with something else The methodological change is greater when, for example, grammar practice

is substituted after the omission of an inappropriate communicative function, or when a reading text is replaced by a listening passage

1.4.2.3 Modifying

Modifying can be sub-divided under two related headings The first of these is rewriting, when some of the linguistic content needs modification, the second is restructuring, which applies to classroom management

- Rewriting may relate activities more closely to learners' own background and interest, introduce model of authentic language, or set most purposeful problem-solving tasks where the answers are not always known before the teacher asks the question

- Restructuring: For many teachers who are required to follow a course book rather strictly, changes in the structuring of the class are sometimes the only kind of adaptation that is realistically possible For example, the materials may contain role-play activities for groups of certain size The logistic of managing a large class (especially if they all have same LI) are complex from many points of view, and it will probably be necessary

to assign one role to a number of pupils at the same time Obviously the converse - where the class is too small for the total number of roles available - is also possible if perhaps less likely

1.4.2.4 Simplifying

The technique of simplification is a type of modification, namely a "rewriting" activity The elements of a language which can be simplified are: the instructions and explanations that

accompany exercises and activities, and even the visual layout of materials so that it becomes easier to see how different part fit together However, texts, most often reading passages are applied this technique Usually, the emphasis has been on changing various sentences - bound elements to match the text more closely to the proficiency level of a particular group of learners Simplification could be in the following forms: Sentence structure; Lexical content, and grammatical structures

Simplification has a number of further implications Firstly, once linguistic items are changed, stylistics will be affected, and therefore the meaning of intention of the original text is changed Secondly, simplification of content is required when the complexity of the subject-matter is regarded as being too advanced Thirdly, simplification can refer to the ways in which the content is presented: we may decide not to make any changes to the original text, but instead to lead the learners through it in a number of graded stages

CHAPTER II: EVALUATION ON THE EFFECTIVENESS OF THE MATERIALS

“ENGLISH FOR TOUR GUIDES”- PART IV

In this chapter, I wish to show the attitudes and the expectations of the teachers and third year students in Sao Do Industrial College towards the effectiveness of the materials “English for Tour guides” part IV- Tourism destinations in Vietnam The first section in the chapter reveals the method of data collection and procedures The second deals with the finding and discussion of the data collections, primarily with the survey questionnaires and interview The evaluation of the materials will be based on the evaluation model and criteria of Hutchinson and Waters And the main purpose of this chapter is to find out the answer for the first research question whether the material “English for Tour guides” used at SIC meet the requirements of the course and meet the demands of students in terms of objectives, content, and methodology especially the content of the speaking lessons

2.1 METHOD OF DATA COLLECTION AND PROCEDURES

Survey questionnaires for the third year students and for teachers of English in my institution are considered to play an important part in the data collection process of my study Equally important, informal interview to both teachers and students and class observations are at the same time involved in the process

2.1.1 Survey questionnaires

2.1.1.1 Participants of the survey

Giving our 6 survey questionnaires for teachers and 9 ones for students were thoroughly completed by 4 teachers of EFT and by 80 third year students in SIC right after the course finished Necessarily, some characteristics of the Tourism English course, of the surveyed students and the teachers should be briefly introduced

As a matter of fact, in SIC, students are scheduled to study English in six terms, two of which are for Basic English with the in-house Basic English materials and the last four terms spent

on EFT with 4 materials: English for Tour guides part I, II, III and IV

The surveyed students are of class 03 HDDL 2 and 03 HDDL 2 There are 80 students Those students are of pre-experience (low-experience) at the very early of their career and the course for them here is English for Tour guides focusing on tourism destinations in Vietnam There is a strong possibility that they are basically different from those of job-experiences in view of the expectations As for the surveyed teachers, they have not been

trained to be Tourism English teachers In such a condition, they are confronted with many difficulties They do not feel easy to comprehend the daily job of a tour guide, situations happen when dealing with visitors and how to make a tourism presentation successfully In fact, their teaching has been based purely on what they have obtained from researching tourism materials and observation of lectures by tourism specialist teachers

2.1.1.2 Description of the survey questionnaires

As set out above, the survey questionnaires are designed in two forms The focus of attention put on those is the information of the Tourism English teachers and the third year students' ideas and expectations on the materials “English for Tour guides”, their opinions on the materials’ effectiveness so that reasonable adaptation for the materials can

be conducted

Questionnaire 1 (for the students) aims at getting to know the students' English background, their own opinion on the “English for Tour guides” course for students and especially their expectations to the course It respectively deals with the materials' effectiveness, the appropriateness of the contents of the course and things need to be improving in speaking lessons, students’ expectations for an effective speaking lesson

Questionnaire 2 (for the Tourism English teachers) is evaluation of the teachers on the

materials effectiveness in the light of materials evaluation methods in terms of audience, the aims, content, and methodology The surveyed teachers, aside from that, are expected to supply more information about the adaptation of the materials especially in some speaking lessons

2.1.2 Formal interview to teachers and students

In a view to having more information for any confirmation of the evaluation conclusion, the author herself finds it really necessary to carry out formal interviews to teachers, students and class observations implemented in the class 03 HDDL 1 and 03 HDDL 2 It is undoubted that the results from the survey will have put great influence on the successful completion of

my study

By applying two methods of data collection survey questionnaires and interview from both the audience who learnt the materials and experienced teachers who taught the materials, the evaluator can get an overview of the materials evaluation and the study becomes more reliable

2.2 FINDINGS AND DISCUSSION

As said previously, the target of the study is to evaluate the materials’ effectiveness in criteria such as objectives, contents, methodology For such a reason, the results of the data collection and their analysis are to be viewed basing on those criteria In this part, the study will analyze the evaluation of the teachers and students basing on the results of the survey questionnaires and interview

2.2.1 Teacher’s evaluation on the effectiveness of the materials

In the case of the teachers, there are 17 teachers of English in the Foreign Languages &

Tourism Department at Sao Do Industrial College Four of them taught the materials

“English for Tour guides” Most of these are quite young with from 3 to 7 years teaching English and from 1 to 3 years teaching English for tourism There are 3 female teachers and one male teacher All of them have learnt higher education in methodology and have had chances to learn how to evaluate a material Therefore, they have enough ability to evaluate a material, especially the materials they have taught The writer of this thesis is one of them

Objectively, according to the regulation of SIC, the entire materials must be designed by the teachers who teach them, so that they are in-house materials, and every three years these materials must be adapted to suitable with the learners at that time “English for Tour guides” is no exception

Subjectively, in the process of teaching, and with the experiences of 2 years teaching this materials, the writer of this thesis recognized the advantages and disadvantages of this one and took part in the process of evaluation and adaptation this materials

The teaching staff held a meeting with the participation of all the teachers who taught the materials “English for Tour guides” Part IV- Tourism destinations in Vietnam and discussed on the evaluation and adaptation of this materials And the writer has major responsibility in this evaluation and adaptation of this materials The survey questionnaires and interview questions as well as discussions were used in evaluation this materials

2.2.1.1 The evaluation of the teachers in terms of audience, aims, content, methodology

In discussion, the teaching staff based on criteria of Hutchinson (1987) in Materials

evaluation And these are detailed analysis

1 Audience

The materials is intended for 2nd year students of tourism major They are of the age from

18 to 21 years old They are of both genders: male and female They are tourism majored students, in the future they will work as a tour guide so English especially English for tour guides is very important for their work It is their work to introduce places to tourists A material which presents and provides information about places in Viet Nam is really useful for them Moreover, most of students are from countryside so their English language knowledge is not good and perfect They may be good at grammar but their speaking and listening skill are weak They rarely have chance to communicate with foreigners while studying The materials have met the demand of the students at SIC

2 Aims

The aims of the course are:

- To provide students knowledge related to tourism destinations in Vietnam,

- To train and practice students’ speaking, listening, reading and writing about famous places in Vietnam

- To revise and reinforce Students basic grammar

- To help student make a good presentation about interesting places

- To widen students’ vocabulary and train students’ pronunciation

- To provide a reference book about tourism

The materials, in general, are suitable with most of the given objectives of the course However, the providing knowledge related to tourism destination in some speaking lesson

is not enough Therefore student’s practice in speaking is not very effectiveness

3 Content

About kind of language description, Language in the materials is mainly structural based

related to tourism knowledge and functional discourse based for example: writing a job application, writing about a festival, giving direction, or reading about destinations in

Vietnam

Concerning Language points, languages points are covered in the materials are particular structures, vocabulary related to destinations and functions necessary for a tour guide Mentioning to proportion of work on each lesson: There are lessons for four skills in each unit but the proportion of reading is higher In some parts we can integrate skills to have

the best result Text type includes reading text, letter, dialogue, visual text (picture, charts, and table), listening text and some other kinds

Subject matter areas are required: tourism destinations in Vietnam for third year students

of Sao Do Industrial College Topics related to tourism are needed The topics are given about factual destinations and to raise human interest about visiting these places

The content in the materials is organized by topic as I mention above Organizing by this way makes students get information easily and systematically Students approach tourist destinations in detail and vocabulary source is arranged in topic and this is a suitable way

to organize materials for students of tourism major

The content in the materials is sequenced in cycle, from topic to topic No topic is more difficult than others so it doesn’t make students feel embarrassed when choosing which topic study first Within the lesson, the content is sequenced from comprehension to production, from accuracy to fluency It follows 3 steps: presentation, practice, and production

The last but not least, the sequence of the content in each unit, it is from guided to free, from comprehension to production, from accuracy to fluency

The materials are based on book “Viet Nam – the destination of the new millennium” – a book for tourist guides and foreign tourists published by tourist Agency

The purpose of the materials is to enhance students to discover Vietnam and to master their speaking skill after finishing the course

4 Methodology

The book attracts students by giving suitable kinds of tasks and exercises: from guided to free, comprehension to production, from one right answer to many possible right answer and with various actions: work in pair, work in group, individual, role play, games, drama In order for students to increase their language competence, a great amount of time is devoted to students’ presentation action Teacher organizes students into group and points a leader This leader guides the group, synthesis ideas and then present before class This teaching technique helps student be confident and it applies CLT (communicative language teaching) in teaching and learning

To use this materials effectively, there must be some aids such as: cassette recorders, overhead projectors, wall charts, video, pictures These aids are necessary and they make the lessons more interesting and accessible