An evaluation of the coursebook english for pharmaceutical course for the second year students at văn hiến medical and pharmaceutical school

LIST OF FIGURES AND TABLES FiguresFigure 1: Materials Evaluation Process of Hutchinson and Waters 1987Figure 2: Materials Evaluation Model of McDonough and Shaw 1993 Tables Table 1: Type

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF

POST- GRADUATE STUDIES

SCHOOL

(Đánh giá giáo trình “ English for pharmaceutical course ” dành cho sinh

viên năm thứ hai Trường Trung cấp Y, Dược Văn Hiến)

M.A PROGRAM PROGRAMME THESIS

FIELD: ENGLISH LANGUAGE TEACHING METHODOLOGY CODE: 60 14 10

HA NOI- 2013

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF

POST - GRADUATE STUDIES

SCHOOL

(Đánh giá giáo trình “ English for pharmaceutical course” dành cho sinh

viên năm thứ hai Trường Trung cấp Y, Dược Văn Hiến)

M.A PROGRAM PROGRAMME THESIS

FIELD: ENGLISH LANGUAGE TEACHING METHODOLOGY CODE: 60 14 10

HA NOI - 2013

BA - Bachelor of Arts

ELT - English Language Teaching

ESP - English for Specific Purposes

HUT - Hanoi University of Technology

LIST OF FIGURES AND TABLES Figures

Figure 1: Materials Evaluation Process of Hutchinson and Waters (1987)Figure 2: Materials Evaluation Model of McDonough and Shaw (1993)

Tables

Table 1: Types of information in the teachers and students‟ questionnairesTable 2A: The teachers‟ opinions about the aims of the materials ( Q1 - 6)Table 2B: The students‟ opinions about the aims of the materials ( Q1 - 6)Table 3A: Students‟ opinions on the skills of the material ( Q7 - 11)

Table 3B: Teachers‟ opinions on the skills of the material ( Q7 - 11)

Table 4A: Students‟ opinions on the language points and text types of the

material ( Q12 - 16)Table 4B: Teachers‟ opinions on the language points and text types of the

material ( Q12 - 16)Table 5A: Students‟ opinions on the topics of the material ( Q17 - 20)

Table 5B: Teachers‟ opinions on the topics of the material ( Q17 - 20)

Table 6A: Students‟ opinions on the tasks/exercises of the material (Q20 - 25)Table 6B: Teachers‟ opinions on the tasks/exercises of the material (Q20 - 25)Table 7A: Students‟ opinions on the teaching - learning techniques of the

material ( Q25 - 30)Table 7B: Students‟ opinions on the teaching - learning techniques of the

material ( Q25 - 30)

TABLE OF CONTENTS

Page

PART A: INTRODUCTION 1

1.Background and Rationale of the study 1

2 Aims of the study 2

3 Significance of the study 3

4 Scope of the study 3

5 Organization of the study 3

PART B: DEVELOPMENT……….5

CHAPTER 1: LITERATURE REVIEW AND THEREOTICAL BACKGROUND 5

1.1 Previous studies……… …5

1.2 Materials in Language Teaching and Learning 6

1.2.1 Roles of Materials 6

1.2 2 Types of Materials 7

1.2.2.1 Textbooks ……… 8

1.2.2.2 In - house materials………8

1.2.2.3 ESP materials……….9

1.3 Materials Evaluation …….11

1.3.1 Definitions of Materials Evaluation 11

1.3.2 Reasons of Materials Evaluation 12

1.3.3 Types of Materials Evaluation 13

1.3.4 Principles of Materials Evaluation 14

1.3.5 Models of Materials Evaluation 14

1.3.5.1 Evaluation by Hutchinson and Waters (1987) ……… .14

1.3.5.2 Evaluation by McDonough and Shaw (1993)……… 16

1.3.5.3 Evaluation by Ellis ( 1997)……… ……… …17

1.3.6 Criteria for Materials Evaluation…… …….18

1.4 Research instruments in materials evaluation……… …… … 19

1.4.1 Document analysis……… … 19

1.4.2 Questionnaires……… … ….19

1.4.3 Interviews……….… …20

1.4.4 Records……… ……… …….….20

1.4.5 Assessment……… ……….….21

1.4.6 Checklists……….……… 21

1.5 Summary …21

CHAPTER 2: METHODOLOGY 22

2.1 Research questions restated……….……….………… ……… .22

2.2 Data collection instruments……….……… ….22

2.2.1 Document analysis……….…… 23

2.2.2 Questionnaires ……… … …24

2.3 Subjects……… ……… .25

2.3.1 The teachers……….25

2.3.2 The students……… 26

2.4 Data collection procedures……… 26

2.5 Summary……….………… .27

CHAPTER 3: DATA ANALYSIS AND DISCUSSION 28

3.1 The results from the evaluation of aims……… …28

3.1.1 Aims of the course……… … …28

3.1.2 Results from the analysis of material aims and questionnaires…… …28

3.2 The results from the evaluation of the content……….….30

3.2.1 ESP material analysis……… ….30

3.2.2 Teachers and students‟ opinion on the content of the material…… ….32

3.2.2.1 Skills……… … 32

3.2.2.2 Language points and text types……… ….33

3.2.2.3 Topics……… ….35

3.3 Results from the evaluation of the methodology……… ………… 36

3.3.1 Methodology implied in the material in terms of tasks/ exercises ……36

3.3.2 Methodology implied in the material in terms of teaching - learning techniques ……….……….38

3.4 Summary……… ……….…….39

CHAPTER 4: RECOMMENDATIONS AND CONCLUSION….……… ….40

4.1 Recommendations for the material improvement……… 40

4.2 Limitations of the study and suggestions for further study……… 42

4.3 Conclusions of the study……….42 References ………I

Appendixes …… V

Appendix 1: Hutchinson and Waters‟s criteria checklist for materials

evaluation………V

Appendix 2: Questionnaire for students ….XIV Appendix 3: Questionnaire for teachers XX Appendix 4: A sample unit XXVII

PART A INTRODUCTION

In part A, I will introduce rationale, aims, significance, scope andorganization of the study

1 Rationale of the study.

English has been the main language of international communication, whichhas become an integral part of most modern professions Along with this trend,branch of English Language Teaching (ELT) – English for Specific Purposes (ESP)has been focused on more Harding (2007: 6) puts it: “in ESP – English for SpecificPurposes – the purpose for learning the language is paramount and relates directly

to what the learner needs to do in their vocation or job”

ESP material for pharmaceutical students at Van Hien School is designed tohelp students get effective knowledge related to their job in the future This materialhas been introduced to students since 2009 It has been officially used as the corematerial for the students of Pharmacy at Van Hien School English is a compulsorysubject at this school English course for pharmaceutical students is divided into twostages The first stage consists 60 class hours of general English with the

coursebook Headway Elementary written by Liz and John Soars (2000) And in the

second part (the third semester), they have 30 class hours of ESP with English forPharmacy It was designed by two first teachers but they moved to other school.This material contains 25 units Each unit includes one reading text andpronunciation of new words The reading text introduces the theme of unit related topharmacy The second one is the list of new words with pronunciation

Teachers and students feel dissatisfactions with this material because ofmany reasons Until now, no research on the course book evaluation has beencarried out to examine how well the material being in use match the courserequirements in terms of aims, content and methodology And based on those

reasons, the researcher decides to evaluate this material named “English for pharmaceutical course” with the hope that it will help to improve the material‟s

quality and get better learning efficiency as well as stimulate students‟ interest inlanguage learning to meet the students‟ needs and the course requirements.

2 Aims of the study

This study is to evaluate the ESP material “English for pharmaceutical course” to determine how the material satisfies the requirements in terms of aims,

content and methodology and what the material effects on students From theevaluation, some suggestions for using the material will be shown to change andimprove the effectiveness of the material for the better use

This study aims at answering the following questions:

1. What effects does the material “English for pharmaceutical course” have on

students in terms of aims, content and methodology?

2. What are students and teachers‟ attitudes towards this material?

3 Significance of the study

The findings of the study are expected to be useful not only to the researcher,the English teachers who are teaching English but also to the second yearpharmaceutical students at Van Hien Intermediate School Besides, the evaluationresults will show the suitability and unsuitability of the ESP material in use, so thatthe researcher will make some suggestions to improve the material to be better use

in the nearest time

4 Scope of the study

In textbook and evaluation, many criteria need to be examined like layout,cultural bias, audience, aims, content, methodology, and so on However, because oflimited time, the researcher bases on the Hutchinson and Waters‟ (1987) criteria for

evaluation with focus on the three following criteria: aims of the material, content

of the material (in term of language points covered, skills, text types, and topics), and methodology ( in terms of types of tasks and exercises, and teaching - learning

techniques) And the researcher only investigates the attitude of the second yearpharmaceutical students and English teachers at Van Hien Intermediate Schoolbased on their feedback

5 Organization of the study

The study is organized into two parts as follows:

Part A discusses about rationale, aims, significance, scope and organization

in materials evaluation such as: definitions, reasons, types, principles, models aswell as criteria of materials evaluation The chapter ends with research instruments

in materials evaluation

Chapter 2 describes the choices of methodology adopted in the study, the

data collection instruments, the subjects as well as the data collection procedures

Chapter 3 presents the results of the study and the discussion of those results,

data analysis and the findings of the study, pointing out the strengths andweaknesses of the material

Chapter 4 shows recommendations of the study, the limitations of the study,

suggestions for further study and conclusion of the study

PART B DEVELOPMENT CHAPTER 1 LITERATURE REVIEW AND THEORETICAL BACKGROUND

Chapter 1 aims to provide the background information about materials andmaterials evaluation The first part reviews previous studies In the second one,some main points of materials in ELT will be presented The third part is about thespecific points about materials evaluation such as: definitions, reasons, types,principles, models, criteria of materials evaluation The chapter ends with researchinstruments in materials evaluation

1.1 Previous studies

Until now, there have been many studies carried out to evaluate materials ingeneral and ESP material in particular, such as Litz (2001), Akin and Guceri( 2001), Nguyễn Thị Phương Dung ( 2002), Võ Thị Anh Đào ( 2003), Nguyễn ThịBích Liên ( 2004), Trần Thị Thúy Nga ( 2005)

The first example is the study of Nguyễn Thị Phương Dung (2002) In herown study, she recommends criteria for the adaptation of the reading material forthe fourth - year students of Electronics and Telecommunications at HanoiUniversity of Technology after conducting a questionnaire survey for teachers andstudents, using Hutchinson and Waters‟ (1987) framework for materials evaluation.Four criteria are focused on that study: the audience, the aims, the content and themethodology The subjects of the study are 80 students and 11 teachers of HUT.The questionnaires are well - organized and able to help her to collect necessaryinformation She concludes that the existing material basically meet therequirements of the course However, the material has some weaknesses whichshould be modified and added

Võ Thị Anh Đào ( 2003) also chooses the criteria proposed by Hutchinsonand Waters ( 1987) to evaluate ESP materials for students of Husbandry andveterinary Science at Tay Nguyen University in terms of language level, aims,content and methodology The subjects of the study are 40 second year students andfour teachers The methods used in the study are document analysis, questionnairesfor students and interview for teachers The study is well presented After collectinginformation from the document analysis and the survey questionnaires, sheconcludes that the material is difficult for students in terms of grammatical structureand lexical items However her thesis reveals some weaknesses And it should beimplemented some parts.

From the above mentioned studies, it can be concluded that the majorpurpose of conducting an evaluation of materials is to evaluate the effectiveness ofthe material in use and its suitability to the course requirements It is also found thatthe criteria proposed by Hutchinson and Waters (1987) are chosen in almost studies.Document analysis, questionnaire and interview seem to be popular methods tocollect data for materials evaluation

1.2 Materials in English Language Teaching and Learning

Roles of materials, Types of materials will be shown in this section

It means that materials are “basis for the content of the lesson, balance of skillstaught, and the kind of language practice students take part in” (p 251) Brown(1995:139) defines materials as “any systematic description of the techniques andexercises to be used in classroom teaching” It emphasizes that teaching materials

are crucial component in the language classroom and there is hardly any classwithout teaching materials.

Dudley - Evans and St John (1998) point out four functions of teachingmaterials: a source of language, a learning support, a source of motivation and

stimulation, and a source for reference As a source of language, materials need to

present in real language as it is used in real situations and they must be suitable to a

wide range of needs of the students As a learning support, materials need to be

reliable, that is to say, the materials need “to work, to be consistent and to have

some recognizable patterns” (p 171) As a source of stimulation and motivation,

materials need to be “challenging yet achievable; to offer new ideas and information

whilst being grounded in the learners‟ experience and knowledge (p 172) As a source of reference, materials help the learners to make “efficient use of the

resources in order to facilitate self- discovery (Tomlinson, 1998:11) For thisfunction, Dudley - Evans and St John (1998:172) shows that materials need to be

“completed, well laid out and self - explanatory”

The roles of ESP materials are shown more clearly According to Hutchinson(1987:107), the language learning process is facilitated by “providing a path throughcomplex mass of the language to be learnt” As Prabhu (1994: 94) emphasize: “Thefact that materials need to be used as sources rather than as pre-constructed coursesshould not be regarded as a weakness of task-based teaching; it can in fact be astrength for any form of teaching”

ESP materials are used in almost Vietnamese Universities and Colleges InVan Hien School, ESP material is the main component which provides the content

of the lesson, the skills and the activities in the class Because they believe thatmaterials “represent the visible heart of any ELT program” (Sheldon, 1988:238)

1.2.2 Types of materials

In the broad sense of the concept, „materials‟ as defined by Tomlinson(1998) is “anything which is used to help to teach language learners.” It can be inthe form of a textbook, a workbook, a cassette, a CD-Rom, a video, a photocopied

handout, a newspaper, a paragraph written on a white board: anything that presents

or informs about the language being learned Materials of these kinds can obviously

be exploited effectively for language learning Because of the limited scope of thestudy, only three types of materials concerned with materials evaluation, especially

in ESP are presented They are textbooks, in-house materials, ESP materials

1.2.2.1 Textbooks

Tomlinson (1998:9) states that:

A textbook provides the core material for a course It aims to provide asmuch as possible in one book and is designed so that it could serve as theonly book, which the learners necessarily use during a course Such a bookusually includes work on grammar, vocabulary, pronunciation, functions andthe skills of reading, writing, listening and speaking

Textbooks are a key component in most language programs In somesituations they serve as the basis for much of the language input learners receive andthe language practice that occurs in the classroom They may provide the basis forthe content of the lessons, the balance of skills taught and the kinds of languagepractice the students take part in

In other situations, the textbook may serve primarily to supplement theteacher's instruction For learners, the textbook may provide the major source ofcontact they have with the language apart from input provided by the teacher In thecase of inexperienced teachers, textbooks may also serve as a form of teachertraining - they provide ideas on how to plan and teach lessons as well as formatsthat teachers can use

Much of the language teaching that occurs throughout the world today couldnot take place without the extensive use of commercial textbooks Learning how touse and adapt textbooks is hence an important part of a teacher's professionalknowledge

1.2.2.2 In - house materials

In-house materials will provide students with activities that suit the specificneeds of their future or current jobs or, as Sheerin (1989: 25) points out: “In-houseproduced material is extremely valuable as it is inevitably more precisely geared tothe needs of students than published material” When determining what tasks toinclude in ESP materials, teachers should thus above all strive to select and/ordesign tasks that will simulate the learner‟s real-life business situations as closely aspossible.

When designing in- house materials, teachers can rely on some generalconsiderations that can also be used when deciding on the appropriate textbook.According to Haycraft (1987: 127), some of the most important considerations are

as follows:

- The length of the course and the target audience of the course;

- The appropriate structural grading: students should be taught what they need

to know “in the right order with the right priorities”;

- The vocabulary should be useful and in current use;

- The appropriate idiomatic English;

- The materials should be “visually alive” and “well presented”

1.2.2.3 ESP materials

Materials selection, adaptation, or writing is an important area in ESPteaching, representing a practical result of effective course development andproviding students with materials that will equip them with the knowledge they willneed in their future business life

One of the most important issues regarding ESP materials selection and/orwriting is whether the materials selected should be solely or primarily subjectspecific and what the most appropriate ratio of general materials to subject-specificmaterials is General materials focus on one‟s general ability to communicate moreeffectively, while subject-specific materials focus on a particular job or industry(Ellis and Johnson, 1994)

According to Prabhu (1994: 94), another important issue regarding materials

is that they should be used as sources: “The fact that materials need to be used assources rather than as pre-constructed courses should not be regarded as a weakness

of task-based teaching; it can in fact be a strength for any form of teaching”

To sum up, after analyzing learner needs and setting objectives for thecourse, the ESP teacher has to select materials that will help the students achievethe course objectives (Ellis and Johnson, 1994) These materials should also relateclosely to the learners‟ specific skills and content needs, which is an importantprecondition for full exploitation of the materials as well as the learners‟ motivation

1.3 Materials Evaluation

Materials evaluation is an important part of materials selection as well as thematerials development process In both cases, evaluation is primarily “concernedwith relative merit There is no absolute good or bad – only degrees of fitness forthe required purpose” (Hutchinson and Waters, 1994: 96) An evaluation of printedESP materials will thus above all serve to locate the materials that will best suit thelearners‟ needs with regard to their future or current work area When no suitableprinted materials are found, the evaluation of existing materials can serve as aspringboard for development of in-house produced materials In-house materialsthemselves should also be evaluated in order to provide the basis for their revisionwith a view to improving their quality and their suitability to the target learners‟needs

1.3.1 Definition of Material Evaluation

Many definitions of evaluation exist Most definitions refer to programevaluation; some refer to project or policy evaluation Some definitions use oneterm to refer to all types of evaluation, for example including policy and programevaluation under the umbrella title of policy evaluation This section provides anumber of examples of evaluation definitions

Evaluation is often defined as an activity that judges worth For example,evaluation is:

 …the determination of merit, worth, or significance… (Scriven 2007)

 a course of action used to assess the value or worth of a program (Farell et

al 2002)

Some definitions include the notion of improvement For example:

 [Evaluation is] a set of research questions and methods geared to reviewingprocesses, activities and strategies for the purpose of improving them in order to achievebetter results (Kahan & Goodstadt 2005)

 The real purpose of an evaluation is not just to find out what happened, but

to use the information to make the project better (Community Tool Box undated)

From those definitions of evaluation, it can be inferred that materialsevaluation involves the activity of the worth judge, the notion of improvement

1.3.2 Reasons of materials evaluation

This section will discuss about some reasons of materials evaluation.Sheldon (1988) has suggested several reasons for textbook evaluation He states thatthe selection of a textbook is indicator of an educational decision in which there isconsiderable professional, financial, and even political investment Throughevaluation, teachers will become familiar with the content of available textbooksand recognize the weaknesses and strengths of each

Tomlinson (1999a: 11) lists the following reasons for evaluating materials:

1 To select a textbook for a course

2 To select materials to supplement a course book

3 To select materials from different sources in an eclectic manner

4 As a basis for adaptation of materials in order to make them more suitable for

a particular course

5 As a basis for improving materials (trialling or piloting materials)

6 In order to edit materials produced by others

7 In order to review proposed materials for a publisher

8 In order to review published materials for a journal

9 In order to help teachers or trainee teachers develop their understanding of methodology and/or materials writing.

Evaluation is considered as the confirmation of practice if the practice isgood In the other case when there is any problem with the practice Evaluation isfor the consideration of ways to innovate or change the practice for a better one

The material at Văn Hiến Intermediate School, “English for pharmaceutical course”, is used every school year It has both advantages and disadvantages So it

is necessary to evaluate this material, which becomes the main purpose for carryingout this research of the author

different terms are used to indicate types of materials evaluation, these terms arebasically similar:

 Pre-use (preliminary) evaluation: The material is evaluated before it is

taught, and it is the most common type of evaluation But even a meticulous pre-useevaluation can be unreliable, as there may not be adequate information about the impact

of the material in the classroom

 In-use (formative) evaluation: This is done while using the material in the

classroom but it is seldom done in a methodical way If the whilst-evaluation is donewith planning and focus it can generate useful data

 Post-use (summative) evaluation: Post - use evaluation takes place at the end

of the course or when the course is finished Therefore, it does not affect the course Thepurpose of Post-use evaluation is to assess a mature project‟s

success in reaching its stated goals and makes improvement in subsequent versions of the course or materials.

Each type has its own characteristic So choosing a suitable type depends on theaims of the study This study is carried out after finishing the course of the second

year pharmaceutical students so it belongs to post - use evaluation 1.3.4 Principles for materials evaluation

Cunningworths (1995) indicates four general principles based on good language

teaching practice to help teacher to evaluate materials The first principle is that

teaching materials should match the aims and objectives of the language learning

program The second one is that teaching materials should correspond to students‟ needs Next, teaching materials should be selected in order to help students to use language effectively for their own purposes And the last principle is that teaching

materials should mediate between the target language and the learners They shouldhave a clear role as a support for learning

1.3.5 Models of materials evaluation

There are a lot of models for materials evaluation However, the mostcommonly adopted models are suggested by Hutchinson & Waters (1987),McDonough & Shaw (1993), and Ellis (1997)

1.3.5.1 Evaluation by Hutchinson and Waters (1987)

Hutchinson and Waters (1987) divides the evaluation process into 4 stages

(Figure 1): defining criteria, subjective analysis, objective analysis, and matching

They further add that to make the best choice, different parts involved in the course have to be considered: teachers, students, and sponsors This is a perfect macro-evaluation

What realization of the criteria

do you want in your course?

OBJECTIVE ANALYSISHow does the material beingevaluated realize the criteria?

MATCHINGHow far does the materialmatch your needs?

Figure 1: Materials Evaluation Process of Hutchinson and Waters (1987:98)

 The first step is to determine criteria for evaluation In this step, the different

evaluators have different purposes for evaluation so the selection of evaluation has tobase on what the materials will be judged and which criteria the evaluators choose fortheir evaluation

 The second step, the subjective analysis, is the analysis of the course in terms of

materials requirements under the defined criteria so the evaluator should describe indetail for the learners and the course According to them, subjective analysis should not

be considered as a fixed set of requirements The evaluator should use

the materials evaluation process as a means of questioning and developing his own ideas as to what is required.

 The third step is to determine the objective analysis, that is, analyses the materials

in terms of the selected criteria

 The last one is the matching process, which finds out how far the material

matches the course requirements.

Hutchinson and Waters (1987) also suggest their own checklist ( appendix 1)

of criteria for objective and subjective analysis The purpose of the checklist is forevaluating teaching material by focusing on big issues like audience, the aims, thecontent, and the methodology

1.3.5.2 Evaluation by McDonough and Shaw (1993)

McDonough and Shaw suggests a model for textbook evaluation whichinvolves three stages: external evaluation, internal evaluation, and overallevaluation (figure 2) This is a combination of macro- and micro- evaluation

EXIT

Figure 2: Materials Evaluation Model of McDonough and Shaw (1993:75)

 Firstly, external stage (macro - evaluation) that examines the organization of

materials stated by the author or the publisher including claims made on the cover page andinformation in introduction and table of contents This kind of evaluation gives informationabout the intended audience, the proficiency level, the context of use, presentation andorganization of materials, and authors' opinion about

language and methodology, use of audio-visual materials, vocabulary list and index, cultural aspects, tests and exercises included in the book.

 Secondly, internal stage ( Micro - evaluation) in which the following factors are

examined: a) the presentation of the skills, b) the grading and sequence of the materials,c) authenticity or artificiality of the listening materials, d)

authenticity or artificiality of the speaking materials, e) appropriateness of testsand materials, and f) appropriateness of the materials for different learning stylesand claims made by the authors for self-study

 The last stage is overall evaluation in which usability, adaptability, and flexibility

factors are examined

1 Choosing a task to follow;

2 Describing the task with specification of input, procedures, language activities, and outcomes;

3 Planning the evaluation with reference to the dimensions above;

4 Collecting information before, while and after the task was used, and whatand how the task was performed;

5 Analyzing the information collected;

6 Reaching conclusions relating to what has been discovered, and making recommendations for the future teaching;

7 Writing the report

In summary, these models are very useful in the process and nature of thepurpose So the evaluators have to base on the purposes of evaluation, timeavailable, facilitates and the context in which the evaluation takes place to decide

which model will follow In this study, a material evaluation is carried out to

identify the suitability of the ESP material, English for pharmaceutical course, used

for Pharmaceutical students to the course requirements The model designed byHutchinson and Waters (1987) is used in this study because it is the clearest and themost suitable to the specific context at Văn Hiến Intermediate School

1.3.6 Criteria for materials evaluation

According to Tomlinson (1998:220), criteria are what evaluators use to

“reach a decision regarding what need to be evaluated” Criteria for materialsevaluation depend on what is being evaluated and why they need to be evaluated.There are numerous criteria proposed by different authors

William (1983) suggests seven criteria, each of which has the following aspects:

 General criteria: give introductory guidance on the presentation of language

items and skills

 Speech criteria: suggest aids for the teaching of pronunciation: e.g phonetic

system

 Grammar criteria: offer meaningful situations and a variety of techniques for

teaching structural units

 Vocabulary criteria: distinguish the different purposes and skills involved in

the teaching of vocabulary

 Reading criteria: provide guidance on the initial presentation of passages for

reading comprehension

 Writing criteria: demonstrate the various devices for controlling and guiding

content and expression in composition exercises

 Technical criteria: contain appropriate pictures, diagrams, tables, etc…

According to Hutchinson and Waters (1987), there are four main criteria formaterials evaluation any evaluator should concern: the audience, the aims, thecontent, and the methodology

 The audience of the materials: the evaluator should obtain information about

and from learners to find out whether the materials are suitable to the students‟ age,knowledge of English, interest and so on

 Aims of the materials: the evaluator has to check if the materials match the

aims and objectives of the course

 Content of the materials: the evaluator has to check whether the materials‟

language points, macro-skills/ micro-skills, and topics suit the learners‟ needs

 Methodology of the materials: the evaluator has to find out if techniques,

aids, guidance provided in the materials satisfy the learners and the teachers of thecourse

In conclusion, from these criteria presented above, I decide to choose the criteriasuggested by Hutchinson and Waters (1987) to this research because they are the most suitable with specific context at Văn Hiến Intermediate School These criteria

do not only help the evaluator with a logical insight of the material but also create the flexibility in the conduction of a detailed material analysis

1.4 Research instruments in materials evaluation

Brown (1995), Hutchinson and Waters (1994), Robinson (1991) claim thatmost materials are evaluated using one or more of the following instruments:document analysis, questionnaires, interview, records, assessment and checklists

1.4.1 Document analysis

Document analysis is an important source which the data are collected.Document analysis is the use of content analysis and other techniques to analyzeand summarize printed materials and existing information

Nunan (1993) states that this instrument is used to collect qualitative datathat concerned with qualities and non-numerical characteristics In evaluation, theanalysis of existing documents can provide useful information One of the biggestadvantages of document analysis is that in some situations, a document represents areflection of reality, so it is useful for determining value, interest or public attitudes

1.4.2 Questionnaires

Questionnaire is one of the most common tools to collect data Brown (1989)indicates that “Questionnaires are any written instruments that present respondentswith a series of questions or statements to which they are to react either by writingout their answers or selecting from among existing answers” The best advantage ofthis instrument is that it provides an easy quick form of data collection Therefore,data can be collected on a large scale and reach a geographically dispersed sample

of a population

Moreover, it is economical because it involves only duplication or mailedexpenses (Creswell, 2005) Besides advantages, questionnaire also hasdisadvantages Creswell (2005:361) claims that “individuals may lack any personalinvestment in the study and decide not to return the instrument” Besides, theresearchers do not have a means for explaining questions so respondents maymisinterpret items in the questionnaires ( Creswell, 2005)

Questionnaires items can be relatively open ended and closed ended Anopen item is one in which the respondent can decide what to write and how to say(Nunan, 1992) Open ended questions include items where the actual question is notfollowed by response option for the respondent to choose from but rather by someblank space to fill A closed item is one in which the range of possible responses isdetermined by the researcher Closed ended question is one in which the respondent

is provided ready made response options to choose from by encircling or ticking or

by putting an x in the appropriate slot/box

1.4.3 Interviews

According to Creswell (2005:360), interview is “a form, on which theresearcher asks questions from an interview guide, listens for answers or observesbehavior, and records responds on the survey”

Depending on the evaluation goals and the specific evaluation context, theevaluators can make use of the following three types of qualitative interview format:the structured, semi-structured, and unstructured (Patton cited in Lynch, 1996).Interview may be face - to - face, voice - to - voice, or screen - to - screen But it istime - consuming

1.4.4 Records

According to Robinson (1991:71), “records form an essential part of the datafor the evaluation exercise” In the materials evaluation, records include teacher‟slesson records, teachers, and students‟ diaries for their experience on the materials.This instrument allows the perspectives of different writers to be compared.

1.4.5 Assessment

Assessment includes tests and examinations By using this instrument, theevaluators can collect a wealth of information to assess knowledge, skill, andperformance However, designing a test with high quality is not easy ( Brown,1995)

1.4.6 Checklists

Williams (1983:252) has noted a checklist cannot be a static phenomenon,evidenced by a wide range of checklists over the years The categories in allmaterials evaluation instrument or observation schedule are a much reflection of thetime at which they were conceived and of the belief of their designers as arepublished materials themselves Thus, an “off-the-shelf” checklist is likely to needtailoring to suit a particular context

In short, each type of instrument has its own advantages and disadvantages

to collect data in the process of materials evaluation Based on the specific situation

in Van Hien Intermediate School and the aims of this study, I decide to use materialanalysis, questionnaires for my study

1.5 Summary

This chapter has been stressed the important information of materialsevaluation It must be mentioned that materials evaluation plays a significant role inlanguage teaching and learning, as an appropriate material contributes a great deal

to the success of any course Previous studies are presented clearly The main issuesrelated to materials evaluation such as: definitions, reasons, types, principles,models, criteria of materials evaluation are discussed In this part, evaluative criteriaand process of materials evaluation are chosen Research instruments in materialsevaluation are selected to collect data

CHAPTER 2 METHODOLOGY

This is survey research and this chapter will describe the choices ofmethodology adopted in the study, the data collection instruments, the subjects aswell as the data collection procedures

2.1 Research questions restated

This study is conducted to find the answer to the following questions:

1. What effects does the material “English for pharmaceutical course” have on

students in terms of aims, content and methodology?

2. What are students and teachers‟ attitudes towards this material?

 Do the aims of the material meet the aims of the course?

 Is the material appropriate to the content requirements of the course?

 Is the material appropriate to the methodology of the course?

As mention in 1.3.5.1, the model suggested by Hutchinson and Waters(1987) presents the most logical and clearest procedure and the most suitable withthe specific context at Văn Hiến School Therefore, this model is chosen in thestudy However, regarding the audience a detailed study on student profiles withdetails such as educational backgrounds, knowledge of specialism, status/role withrespect to specialism, sex is impossible Another reason to exclude these details isthat they are not given in the book itself or any supporting material So, theresearcher only chooses three suitable criteria: aims, content and methodology tothis study

2.2 Data collection instruments

In 1.4, there are six types of instruments to collect data in the process of

materials evaluation To evaluate the material “English for Pharmaceutical course”, the researcher decides to mix 2 methods: document analysis and

questionnaire to collect and analyze information This section discusses what eachtool is used for, why it is chosen, and how it is designed.

2.2.1 Document analysis

Document analysis is an effective method to collect the data for the study AsRobinson (1991: 57) said it can provide useful information and “form an essentialpart of the data for an evaluation exercise” Therefore, this instrument was used in

this study with the ESP material, English for pharmaceutical course.

The material analysis is conducted after the curriculum analysis is finished Thematerial itself is analyzed objectively under the same three criteria used incurriculum analysis above Then twenty five units are analyzed to give a detaileddescription of the material‟s contents so that comparison between the material andthe requirements of the course could be made

Material analysis is broken into sub- criteria as follows:

As mention in 1.3.6, I decide to choose the criteria defined by Hutchinsonand Waters (1987) to evaluate this material in this study It consists of the aims,content and methodology In the document analysis, I will focus on the followingpoints: the suitability of the material in term of aims; the suitability of content interms of language points and text types, skills and topics; the suitability ofmethodology in terms of types of tasks/ exercises and teaching - learning techniques

used Therefore, the statements of the aims for teaching ESP for the second yearstudents of Van Hien Intermediate School were investigated.

English for pharmaceutical course they has studied in terms of aims, content and

methodology The questions in the questionnaires in this study are mostly closed ended questions which are designed on a 5 - point Likert scale (in which 1 =strongly agree, 2 = agree, 3 = not sure, 4 = disagree, 5 = strongly disagree) Theyare asked to express their agreement on each required item by choosing theappropriate number of rating scale The combination of closed - ended questionsand open - ended questions could gather both the respondents‟ general assessmentand their personal opinions about the material

-Two questionnaires are conducted, one for pharmaceutical students and otherfor English teachers at Văn Hiến Intermediate School who have taught this ESPmaterial All questions in two questionnaires are translated into Vietnamese to helpstudents and teachers get ideas easily and help the researcher collect data clearly

The questions for the students consist of 30 closed - ended questions anddivide into three parts as follows:

Part Types of information Question

I The aims of the material in relation to the aims of the course Q1-6

Table 1: Types of information in the teachers and students’ questionnaires

Like the students‟ questionnaire, the teachers‟ questionnaire is designed toinvestigate the teachers‟ opinions and attitudes with this material in terms of aims,content and methodology It also consists of three parts like students‟ questionnairebut it is added one more part The questions in the last part are designed with open -ended questions They aim to ask English teachers about their experiences whenteaching this ESP material and their suggestions to improve this material

or four years They have BA in English They are two men and two women whoseage ranges from 30 to 35 years old They have been dealing with this material atleast one year (one teacher) and four years (three teachers) as this material has beenused since 2009 They have used this material to teach the second year students ofdifferent classes Therefore, their judgments about this material are reliable

The researcher of this study is also an English teacher at Văn Hiến Schoolwith two years teaching experience and has also used this material to teach thesecond year students.

2.3.2 The students

60 second year pharmaceutical students from class 4A, 4B take part in thisstudy They are both men and women and their ages range from 18 to 25 In the firstyear, they complete 60 class hours of general English with the course bookHeadway Elementary written by Liz and John Soars (2000) In the first semester of

the second year, they have 30 class hours of ESP with English for pharmaceutical course.

By the time they were asked to complete the questionnaire, they have justfinished the ESP program

2.4 Data collection procedures

In this study, Hutchinson and Waters‟ (1987) model is employed to analyzethe data collected by using two instruments: document analysis and questionnaires

Firstly, the criteria were defined (appendix 1).

Secondly, the in - house material “English for pharmaceutical course” was

analyzed to collect information for the evaluation in terms of aims, content andmethodology The material analyzed in the study has been used since 2009 at vănHiến School This material has been applied to the English teaching and learning forthe third semester of the course It includes 25 units

Thirdly, the researcher develops survey questionnaires for teachers and

students The researcher chooses two periods, which lasts an hour and a half to givequestionnaire for students The researcher explains about the purpose of the studyand the meaning of each question to help students answer and clarify them Thequestions are then collected immediately, checked for completeness by theresearcher The questionnaire for teachers is given to teachers after that andcollected after three days because it had one more part to ask about teachers‟

experience and their suggestion to improve the material All questions are translatedinto Vietnamese to avoid misunderstanding and to be easy to collect data.

Finally, the information collected from the two different sources was

summarized for the analysis From the respondents, the researcher analyzes thestrong and weak points of the material and gives some suggestions to improve it forthe better use in the future

2.5 Summary

This chapter has given a detail description of the subjects of the study, means

of data collection and procedure of the study The study involves two groups ofsubjects: K4 students of pharmaceutical course who have learnt the material;English teachers, who have been using the material since the first day ofpublication The questionnaires for students and teachers and the analysis are allbased on the criteria defined by Hutchinson and Waters ( 1987) in the checklist,they are: material‟s aims, material‟s content and material‟s methodology

CHAPTER 3 DATA ANALYSIS AND DISCUSSION

In this chapter, the results of the study and the discussion of those results arepresented The judgment is categorized into the analysis and discussion of the aims,content and methodology And the last part is the summary of this chapter

3.1 The results from the evaluation of aims

3.1.1 Aims of the course

The ESP course at Van Hien School is to provide students with necessaryEnglish reading skill and strategies so that later they can use them to obtaininformation from their specialist materials; to equip students with specializedvocabulary as well as structures commonly used in pharmacy context The coursealso helps students develop writing skill by giving opportunities to write meaningfulsentences and a summary of the text, to write a report, application letter, etc.Besides, the course aims to provide students the opportunity to develop speakingskills by giving group discussion, describing the charts and diagram in the field ofpharmacy The material also helps students to develop their listening skills bylistening to get specific and main idea related to pharmacy

The material, English for pharmaceutical course, is designed especially for

this specific group of students, claims to embrace these aims: develop all four skillsfor students, grammar and pronunciation

3.1.2 Results from the analysis of material aims and questionnaires

(1 strongly agree; 2 agree; 3 not sure; 4 disagree; 5 strongly disagree) From the table 2A, 2B, it is clear that 100% students and students share thesame opinions about listening skill, speaking skill and writing skill They strongly

disagreed about these items Because in this material (English for pharmaceutical course), it does not have any tasks and exercises related to these skills They

suggested that the material should contain enough skills‟ activities and strategieswhich help students develop their English level

The material supports your students an opportunity 1 2 3 4 5

to practice:

2 Speaking activities to develop speaking skills 0% 0% 0% 0% 100%related to students‟ career

3 Reading skills and strategies that students need. 0% 0% 0% 50% 50%

Table 2A: The teachers’ opinions about the aims of the materials

The same opinions with the grammatical points are shown Because the units in this

material do not connect together with grammatical structures such as: verb tenses,

passive voice, relative pronouns and so on These points are not found in the text

and it does not exist any exercises and task about grammar

The material supports you an opportunity to

2 Speaking activities to develop speaking skills 0% 0% 0% 0% 100%related to students‟ career

3 Reading skills and strategies that students need. 0% 0% 27% 62% 11%

Table 2B: The students’ opinions about the aims of the materials

Items 3 of Table 2A and Table 2B show the teachers and students‟

assessment about the reading skill 27% students keep neutral ideas in this aspect

Two teachers (50%) and 62% students think that the reading skills are not relevant

to the students‟ needs The left numbers strongly disagree with these items Thisdocument has not any exercises related to reading skill Each unit gives the mainidea after learning the lesson which helps students summarize the main point in thelesson And in the reading text, it does not have any activities and strategiesconcerned reading skills like: pre - reading, whole - reading, post - reading, etc.Students say that they have a lot of difficulties to understand the reading text.

As seen in the table 2A, 2B, 23% students and the same with students agreethat the material is successful in providing students with a considerable amount of

vocabulary of pharmacy such as: bacteria, solution, fever, capsule, typhoid, etc The

word list translation in Vietnamese at the end of each unit is very convenient forstudents to look up and understand more about what they learn to apply in theircareer 18% students are not sure with this item The left number disagree andstrongly disagree about vocabulary presented There are no examples for furtherexplanation or illustration of the use of each new word Vocabulary exercises for arealso missing in this material New vocabulary is just presented to students withoutgiving them the chance to practice

To sum up, the first criterion chosen from Hutchinson and Waters‟ model(1987) is used to investigate the question of how well the materials meet the aims ofthe course for the second - year pharmaceutical students at Van Hien IntermediateSchool From what has been discussed above, it can be concluded that the materialwas not appropriate to the course aims The unsuitable aspects are that the materialdoes not offer the revision grammatical structures in each unit and have anyexercise/tasks, activities, strategies in terms of developing the students‟ listening,speaking, reading and writing skills

3.2 The results from the evaluation of the content

3.2.1 ESP material analysis

The material analysis will be presented under Language content, Text - types,Skills and Topics