An evaluation of american headway 1 for first year junior college students in electric power university submitted in partial fulfilment of requirements for the degree of master of arts in tesol

The study attempts to find out how much the textbook fits with the course aims and the students' needs on the content and methodology.. Research questions According to Cunningsworth 1995

M IN IS T R Y OF ED UC ATIO N A N D TR A IN IN G

HANOI UNIVERSITY

N G U YEN T H I BICH NGOC

AN EVALUATION OF AMERICAN HEADWAY 1 FOR FIRST YEAR JUNIOR COLLEGE STUDENTS

IN ELECTRIC POWER UNIVERSITY

TIMJNG И М

Т4ГѴ

SUBMITTED IN PARTIAL FULFILMENT OF REQUIREMENTS FOR

THE DEGREE OF MASTER OF ARTS IN TESOL

SUPERVISOR: Dr L A M Q UANG DONG

I would like to give my sincere gratitude to Dr Nguyen Van Dai and M.Ed Nguyen Thai Ha - the leaders and the whole staff members o f the Department o f Post-Graduate Studies at Hanoi U niversity fo r their useful lectures and assistance, valuable comments, and suggestions.

I acknowledge the excellent assistance o f Dr Pham Xuan Khang- Head o f Fundamental Scientific Faculty, m y colleagues，and students in Electric Power University fo r their support, assistance，and cooperation in performing this work

I would be remiss i f I d id n 't express m y special thanks to my parents, m y younger sister whose encouragement and care were accompanying me during the long journey to the university O f course, nothing o f this would have been possible i f it had not been for the love and support from my husband who has helped me in so many ways to ease the burden o f the work Needless to say, their patience and support have kept me going this far and their love w ill keep me going further

ABSTR AC T

This study adopts both quantitative and qualitative approaches to evaluate the textbook

American Headway 1 for the first year junior college students in Electric Power University

The study attempts to find out how much the textbook fits with the course aims and the students' needs on the content and methodology

Data were gathered by means o f existing information and questionnaires The existing information data were collected by only one subject- the researcher The questionnaires data were obtained by a total o f 200 students from 5 classes and 7 teachers o f English

The results indicate that the textbook was basically suited to the course objectives and the students7 needs Nevertheless, the textbook failed to provide the students with pronunciation Moreover, it did not completely satisfy the students^ needs on its content (some d ifficult grammatical structures, difficu lt and abundant vocabulary, the difficulties o f writing, listening, and speaking skill), its methodology (some difficult kinds o f exercises, the organization o f pair and group activities, some aids, necessary supplementary missing materials), and its time requirement

Based on the findings, a number o f recommendations were made to overcome its weaknesses by using the adding, replacing, and modifying technique as well as to enhance English teaching and learning for the first year junior college students in Electric Power University

LIST OF TABLES AND FIGURES

Table 1 The Factors that need examining in the materials evaluation process 13Table 2 The detailed information o f the student questionnaires 29

Table 4 The detailed information o f the topics in the textbook 44Figure 1 Evaluation Model by Hutchinson & Waters (1987) 11Figure 2 Evaluation Model by McDonough & Shaw (1993) 13

Figure 4 Materials adaptation framework by McDonough & Shaw (1993) 23Figure 5 The students，judgments about the aims o f the textbook 38Figure 6 The teachers’ judgments about the aims o f the textbook 39Figure 7 The students^ judgments about the Presentation & Practice o f the 45

language points & language skills

Figure 8 The teachers9 judgments about the Presentation & Practice o f the 45

language points & language skills

Figure 9 The students’ judgments about the difficu lt level o f the language 46

points & language skills given in the textbook

Figure 10 The teachers’ judgments about the difficu lt level o f the language 46

points & language skills given in the textbook

Figure 11 The students9 judgments about the variety, attractiveness o f the text- 47

types & the fam iliarity, interest o f the topics in the textbook

Figure 12 The teachers5 judgments about the variety, attractiveness o f the text- 47

types & the fam iliarity, interest o f the topics in the textbook

Figure 13 The students’ judgments about the teaching-learning techniques in the 51

TABLE OF CONTENTS

ACKNOWLEDGEMENTS i

ABSTRACT ii

LIST OF TABLES AND FIGURES iii

LIST OF ABBREVIATIO NS iv

TABLE OF CONTENTS V CHAPTER I: INTRODUCTION 1

1.1 Background to the Study and Statement o f the Problem 1

1.1.1 Overview o f Electric Power U niversity 1

1.1.2 Statement o f the problem 2

1.2 Aims o f the study 3

1.3 Research questions 3

1.4 Limitations o f the study 4

1.5 Scope o f the study 4

1.6 Organization o f the study 5

CHAPTER II: LITERATURE REVIEW 6

2.1 Some basic concepts in the study 6

2.1.1 Evaluation 6

2.1.2 Needs 6

2.1.3 Learning styles 7

2.1.4 Textbooks 7

2.2 Materials Evaluation 7

2.2.1 The role o f materials 7

2.2.2 Types o f materials evaluation 8

2.2.3 Guidelines for evaluation 9

2.2.3.1 Guidelines for evaluation by Cunningsworth (1995) 9

2.23.2 Guidelines for evaluation by Grant (1987) 10

2.2.4 Types o f materials evaluators 10

2.2.5 Evaluation models for materials evaluation 11

2.2.5.1 Evaluation model by Hutchinson and Waters 11

2.2.5.2 Evaluation model by McDonough & Shaw 12

2.2.5.3 Evaluation model by E llis 14

2.2.6 Tools for gathering information 15

2.3 Previous studies 18

2.4 Adapting materials 19

2.4.1 Reasons for adapting materials 19

2.4.2 Techniques for adapting materials 21

CHAPTER III: M ETHODOLOGY 25

3.1 Data collection tools 25

3.1.1 The statement o f the existing inform ation 25

3.1.2 Questionnaires 26

3.1.2.1 The statements o f the questionnaires 26

3.1.2.2 Question formats 27

3.1.2.3 Student questionnaires 28

3.1.2.4 Teacher questionnaires 29

3.2 Subjects 30

3.2.1 The researcher as a subject 30

3.2.2 The teachers 30

3.2.3 The students 31

3.3 Data collection procedures 31

3.3.1 Existing information collection procedures 31

3.3.1.1 Literature review 31

3.3.1.2 The aims o f the course 32

3.3.1.3 Textbook 32

3.3.2 Teacher questionnaire collection procedures 32

3.3.3 Student questionnaire collection procedures 33

CHAPTER IV: RESULTS AND DISCUSSION 34

4.1 The results 34

4.1.1 The fitness o f the textbook to the aims o f the course 34

4.1.1.1 Statements o f aim s 34

4.1.1.3 Questionnaire results 38

4.1.1.4 Matching and discussion 39

4.1.2 The fitness o f the textbook content to the students’ needs 40

4.1.2.1 Textbook analysis results 40

4.1.2.2 Questionnaire results 44

4.1.2.3 Matching and discussion 48

4.1.3 The fitness o f the textbook methodology to the students’ learning styles and expectations 49

4.1.3.1 Textbook analysis results 49

4.1.3.2 Questionnaire results 51

4.1.3.3 Matching and discussion 52

4.1.4 Suggestions for the textbook improvement 53

4.1.4.1 Teacher questionnaire results 54

4.1.4.2 Student questionnaire results 55

4.2 Major findings 56

4.2.1 Strong points 56

4.2.2 Weak points 56

CHAPTER 5: RECOMMENDATIONS AND CONCLUSION 59

5.1 Recommendations 59

5.1.1 Areas o f unfitness 59

5.1.1.1 A im s 59

5.1.1.2 Content 59

5.1.1.3 Methodology 60

5.1.1.4 Time allocated to lessons 61

5.1.2 Techniques for adaptation 61

5.1.2.1 Adding 61

5.1.2.2 M odifying 66

5.1.2.3 Replacing 69

5.1.3 A sample unit for adaptation 69

5.1.3.1 Rationale for the adaptation 69

5.1.3.2 Suggested adaptation 69

5.2 Suggestions for further study 71

5.3 Conclusion 71

REFERENCES 74

APPENDIX 1 80

APPENDIX 2 84

APPENDIX 3 92

APPENDIX 4 100

APPENDIX 5 104

APPENDIX 6 105

APPENDIX 7 113

C H A P TE R I: IN T R O D U C T IO N

1.1 Background to the Study and Statement o f the Problem

1.1.1 Overview o f Electric Power University

The School o f Electricity was established in 1966 Thanks to the development o f the country, the increasing requirement o f human resources in the field o f Electricity, the School o f Electricity has also been developing It was upgraded into Electric Power College

in 2001 and then Electric Power University (EPU) in 2006 Nowadays, EPU still maintains

to train at three levels: junior college, college, and university in eight specialties: Electric Power System, Information Technology, Thermal Power Generation, Hydro Power, Automation Technology, Energy Management, Electric Mechanics, Electronic Mechanics, and Electronic Telecommunication

Up to now, EPU has not had an English Faculty yet Therefore, English teaching which is a subject in Fundamental Science Faculty is undertaken by twelve female teachers More than half o f them finished Master o f Arts (M A) courses with English teaching experiences from

3 to 25 years

The student body has been increasing to the current number o f more than 2,000 because o f the high demand for human resources in the field o f Electricity Their backgrounds o f English are varied since some had learnt English or other foreign languages at school, but those from mountainous areas either have never been exposed to a foreign language or have very low level o f English

English has been taught as a compulsory subject in EPU since 1996 Students study General English (GE) in two terms (150 periods) at three levels (university, college and junior college) and continue to learn English for Specific Purposes (ESP) in one term (60 periods) at two levels (university and college) English textbooks currently applied in EPU are diverse, including Lifelines, American Headway, Headway，and New Headway Many

meetings have been taking place in order to discuss the selection o f textbooks, to strengthen the quality o f English teaching and learning in EPU The textbooks, yet, remain

problematic, especially for junior college students In 2004, the system o f modules was designed and used After two years, this system o f modules was replaced with the adoption

have not considerably been improved The students still have a lot o f difficulties in exploring the textbook

1.1.2 Statement o f the problem

Fostering the quality o f teaching and learning is always a burning problem o f each school There are many factors which directly affect the training results, for instance, goals, materials, teaching, testing, equipments, and the students՜ attitude, etc However, in the limited scope o f a thesis and with the expectation o f contributing partly to enhancing the quality o f English teaching and learning in EPU, the researcher made a decision to investigate, study a textbook- a kind o f material since it is one o f the important factors determining the quality o f teaching and learning The textbook examined here is American Headway 1 (A H I) which is currently applied for the first-year junior college students in

EPU In addition, there are some other reasons which urged me to conduct this study as follows:

In fact, English teaching and learning for the first-year junior college students in EPU still suffers from a lot o f restrictions This is shown by the results o f the term-end examinations which have been low in recent years More than 50% o f the students usually get the English marks below average and certainly these students have to retake the English course A lot

o f students may reluctantly learn English to face with the examinations Furthermore, many teachers o f English have controversial views on the application o f A H I currently used for

the junior-college students in EPU As a teacher who has directly been teaching English in EPU, I may understand the most clearly about the existing problems

It is likely that A H I is a quite nice textbook both in content and form The information is

update as it was published in 2001 Nonetheless, it is not a perfect one, especially for the

ju n io r college students Perhaps, it has some weaknesses, for example, some difficult grammatical structures, insufficient pronunciation practice, and so forth Consequently,

most o f the students often make mistakes o f those grammatical structures or have wrong pronunciation o f some very basic words.

So far, there have been just a small number o f studies about the materials evaluation, for instance, Nguyen Thi Hai Ha (2004)，Nguyen Thi Thanh Van (2004)，Tran Thi Thuy Nga (2005)，Hoang Hoa Lien (2005)，and Pham Thi Thanh Huong (2006) examined ESP materials; Le Thi Ngoe Diep (2006) and Vu Thi Yen Nga (2004) evaluated General English materials (Powerbase，Lifelines); Akin & Guceri (2001) and Ngo Thi Thu Huong (2007)

assessed in-house materials, and so forth Nevertheless, there has not been a thesis about the evaluation o f A H I by Soars, L & J yet.

From the above reasons, the researcher decided to investigate, study, and adapt A H I

currently used for the first-year junior college students in EPU Hopefully, the study w ill be

o f some significance to the researcher, the teachers, the first year junior college students, and the authorities in EPU as the findings o f the study w ill supply them with scientific evidence to either enhance the quality o f the textbook or to abandon it altogether Moreover, this is also an opportunity for the researcher to learn to analyze a study, evaluate, and report about an issue before completing her Master o f Arts (M A ) course at Hanoi University

1.2 Aim s o f the study

The study aims to find out how much A H I fits with the course aims and the students’ needs

in terms o f the content and methodology The unsatisfactory areas o f the textbook are, then, adapted to improve its effectiveness for future courses

1.3 Research questions

According to Cunningsworth (1995)，there are many criteria for textbook evaluation, for example, aims and approaches, design and organization, language content, skills, topic, methodology, teacher’ s book, and practical considerations Due to various constraints and the limitations o f this study, only a number among these criteria can be evaluated in order

to find the answers to the following research questions:

L How much do the aims o f A H I f it with the aims o f EPU's English program ?

2 How much does the content o f A H I satisfy the students ’ needs?

3 How responsive to the students ’ learning styles and expectations is the methodology provided in A H I?

In order to answer the specific questions above, the researcher first reviews the literature on materials evaluation Based on the theories in the literature, the researcher analyses the course aims and the textbook, and then carries out a survey to explore the teachers9 and

judgments about the textbook Finally, the researcher recommends the textbook adaptation

1.4 Lim itations o f the study

In this thesis, three factors might affect the outcomes o f the evaluation research as follows: First, the evaluators were all the insiders Thus, their subjective points o f views may influence the textbook evaluation

Second, the suggested adaptations are just what the researcher thinks they should be, based

on her specific teaching context in EPU These suggestions, therefore, may be subjective and unsuitable for different students in a different teaching context

Finally, the researcher only applied three criteria and two data collection tools in order to evaluate the fitness o f the textbook to the aims o f the course and the students’ needs

1.5 Scope o f the study

A strong consensus has built up worldwide in recent decades in support o f a large number

o f evaluative criteria suggested by some writers such as Cunningsworth (1984), Hutchinson

& Waters (1987), McDonough & Shaw (1993)，Brown (1995)，Littlejohn (1998)，and Ur (1991) However, due to the time constraints, the researcher o f the study is only interested

in Hutchinson & Waters (1987)’criteria in the textbook evaluation as follows:

♦ Content o f the textbook (language points, language skills, text-types, and topics).

♦ Methodology o f the textbook (kinds o f activities/exercises, teaching-learning techniques, aids, flexibility, and language guidance)

It is also noted that some sub-criteria chosen for the study are modified so that they are better suited to the researcher’ s particular teaching-learning context

1.6 O rganization o f the study

The study is organized into five main chapters as follows:

Chapter 1 discusses the background to the study and statement o f the problem, the aims, the research questions, the limitations, the scope, and the organization o f the study

Chapter 2 presents a review o f literature, concentrates on the issues relating to materials evaluation

Chapter 3 describes the data collection instruments, the subjects as well as the data collection procedures

Chapter 4 discusses the findings o f the study, points out the strengths and weaknesses o f the textbook

Chapter 5 presents solutions to improve the textbook, suggests for further study, and concludes the study

C H A P TE R I I : L IT E R A T U R E R E V IE W

2.1 Some basic concepts in the study

The title o f the thesis and the research questions result in the demand for exploring some key concepts which are o f great importance before going more deeply into the study

2.1.1 Evaluation

There are a lot o f definitions o f materials evaluation, however, the one o f Hutchinson and Waters (1987) is likely to be the clearest because it provides a comprehensive understanding o f the evaluation process They regard evaluation as a matter o f judging the fitness o f something for a particular purpose

2.1.2 Needs

In deed, the term needs is not as straightforward as it might appear When we read or hear

it, many words and expressions are triggered in our minds: uWants-Desires-Dernands- Expectation-Motivations-Lacks-Constraints֊Requirements^^ (Brindley 1984: 28，cited in

Richards 2001; 54) More specifically and practically, Robinson (1991; 7) considers needs

as what the students themselves would like to gain from the language course Needs can be classified into different types, for example, target needs and learning needs (Hutchinson & Waters 1987)，situation needs and language needs (Brown 1995)，etc Yet, in brief, the term

“ needs” is very broad one referring to the language demand o f the particular learners and local learning situation

Furthermore, five needs experienced by beginner students should be considered when analyzing the students’ needs Those five needs are: the need to communicate effectively, the need to be familiar with the language systems, the need for challenge, the need to take

on more responsibility for their own learning, and the need for cross-cultural awareness (Abbs & Freebaim 1990，cited in Cunningsworth 1995: 9ᄀ).

2.1.3 Learning styles

According to Richards, Platt & Weber Richards et a l (1992: 61), a learning style can be

considered the particular way in which a learner tries to learn something In second or foreign language learning, different learners may prefer different solutions to learning problems For example, some learners may want explanations for grammatical rules; others may not need explanations Some learners may feel w riting down words or sentences helps them to remember the words or sentences Others may find they remember things better i f they are associated with pictures Obviously, a learner may have more than one learning styles, a teacher may also have different teaching styles, and a textbook may have various teaching-learning techniques The problem is to have the unification among the learners, the teachers, and the textbook (Grant 1987: 11)

2.1.4 Textbooks

Materials may be printed (books, worksheets, etc), non-printed (cassette or audio materials, videos, etc), or both printed and non-printed ones (self-access or the Internet ones) (Richards 2001; Brown 1995) A textbook, therefore, can be viewed as a kind o f materials According to Richards (2001: 254-255), a textbook not only provides a structure and a syllabus for a program, a variety o f learning resources, effective language models and input but also helps standardize instruction or maintain quality, etc However, a textbook may not reflect students’ needs As a result, the impact o f a textbook on a program，teachers, and learners has to be carefully assessed

2.2 M aterials Evaluation

2.2.1 The role o f materials

Teaching materials are also o f great importance in language classrooms and have undergone many fluctuations and dramatic shifts over the years There are a lot o f different views on their roles Yet, materials may have a great influence mainly on learners, teachers, and contexts

For the learners, materials are regarded as sources for their language input, language practice, communicative interaction, useful learning aid, motivation, stimulation，and increase o f knowledge (Richards 2001; Cunningsworth 1995; Dudley-Evans & St John 1998; Tomlinson 1998; and Grant 1987).

For the teachers, materials are considered forms o f teacher training- they provide ideas on how to plan and teach lessons as well as formats that teachers can use, supports for less experienced ones who have yet to gain in confidence, and supports for saving them an extraordinary amount o f time (Richards 2001: 251; Cunningsworth 1995: 7)

For the contexts, materials can function as syllabuses These syllabuses reflect learning objectives that have already been determined, identify what and the order in which it should

be taught/ learned，and indicate what methods should be used (Cunningsworth 1995: 7; Ur 1991: 184; and Grant 1987: 8) In addition, materials are valid, useful and labor-saving tools (Grant 1987，W illiam 1983，Ansary & Babaii 2002) As a result，A H I can be

considered a syllabus in the case that there is not an English syllabus for the first-year junior college students in EPU

Nobody can deny the importance o f materials in an English course However, teachers should keep away from much reliance on them which should be viewed as better servants than masters (Cunningsworth 1995: 15) because “ the perfect textbook does not exist.” (Grant 1987)

2.2.2 Types o f m aterials evaluation

Robinson (1991: 59) proposes three types o f materials evaluation: preliminary, formative, and summative Preliminary evaluation takes place before a course begins and involves selecting the most appropriate from the publications that are available Formative evaluation is conducted while the course is ongoing so that modifications can be made to the materials whereas summative evaluation takes place at the end o f a course and addresses the question o f whether the materials have been effective

Cunningsworth (1995: 14) introduces three types o f materials evaluation: pre-use, in-use,

after use According to Cunningsworth, pre-use is the most d ifficult kind o f evaluation whilst post-use aims to identify strengths and weaknesses for future decisions.

Richards (2001: 288-291) refers to three types: illuminative, formative, and summative Illuminative evaluation seeks to find out how different aspects o f the program work or are being implemented Formative evaluation focuses on ongoing development and improvement o f the program Summative evaluation seeks to make decisions about the worth or value o f different aspects o f the curriculum and takes place after a program has been implemented

Although the different terms are used to demonstrate the types o f materials evaluation, they are basically similar First, preliminary, pre-use, and illuminative evaluations mean a process to seek for good materials Next, formative and in-use evaluations interpret how materials are being exploited Certainly, summative and post-use evaluations are used to determine how effectively materials have been used

2.2.3 Guidelines fo r evaluation

2.2.3.1 Guidelines fo r evaluation by Cunningsworth (1995)

Cunningsworth (1995: 15-17) offers four guidelines in approaching any materials evaluation exercise as follows:

♦ Course books should correspond to learners’ needs They should match the aims and objectives o f the language-learning program

♦ Course books should reflect the uses (present or future) which learners w ill make o f the language Select course books which w ill help to equip students to use language effectively for their own purposes

♦ Course books should take account o f students’ needs as learners and should facilitate their learning processes, without dogmatically imposing a rigid 'method*

♦ Course books should have a clear role as a support for learning Like teachers, they mediate between the target language and the learner

2.2.3.2 Guidelines fo r evaluation by G rant (1987)

Grant (1987: 118-121) suggests the three stages o f evaluation: initial, detailed, and in-use

♦ Initial evaluation involves the ‘ C ATALYST, test applied in the classroom The eight letters in the word including с (Communicative) - A (Aims) - T (Teachability) - A (Available Add-ons) - L (Level) - Y (Your impression) - s (Student interest) ֊ T (Tried and Tested) represent the eight criteria by which the decision is made whether a textbook is suitable for the classroom

♦ Detailed evaluation is carried out to decide how far the course book suits the student, the teacher, and the syllabus Grant also provides a three ֊֊ part questionnaire which should be applied to one o f the textbook, i f possible, to two different textbooks and then should be discussed with colleagues

♦ In-use evaluation deals with the constant evaluation by using the questionnaire supplied

or modified and one can ensure that the teacher is the master, and not the slave, o f the textbook!

evaluation during the evaluation process

2.2.4 Types o f materials evaluators

There are two types o f materials evaluators: insiders and outsiders (Tomlinson 1998; Dudley-Evans & St John 1998; Robinson 1991; and Richards 2001)

The outsiders are the ones who are not involved in the program, for example, consultants, inspectors, and administrators They may provide an objective approach to their materials evaluation and can supplement the teachers’ perceptions o f what happens in a course with independent observation and opinions (Robinson 1991: 68; Richards 2001: 291) However,

as no observer is truly objective, they w ill have their own views on language teaching methods, administrative procedures, etc It w ill take them much time to be aware o f the local needs and constraints, too

The insiders are the course designers, teachers, and students Obviously, these insiders can provide the most valid information in the evaluation process because they can understand the learning and teaching situation and learners’ needs than any others (Tomlinson 1998; Richards 2001; and Robinson 1991).

2.2.5 Evaluation Models for materials evaluation

2.2.5.1 Evaluation Model by Hutchinson & Waters

Hutchinson & Waters (1987) provide a framework for materials evaluation (see Figure 1)

Figure 1: Evaluation Model by Hutchinson and Waters (1987: 96)

As shown in Figure 1，evaluators should follow four major steps: setting out the criteria; carrying out the subjective analysis; implementing the objective analysis; and matching the findings Three options may happen after matching as follows: 0 = does not match the desired feature; 1 = partly matches the desired feature; and 2 = closely matches the desired feature When the materials partially mismatch the desired features in some criteria, the shortcomings should be adapted.

Furthermore, the authors also supply four basic criteria (see Appendix 1)

The first criterion, audience^ involves the analysis o f the students’ personal information (the students’ age, sex, knowledge o f English, interest and their educational backgrounds)

The second criterion deals with the aims o f the course and the materials

The next criterion, content, helps evaluators to have an overview o f the content treatment o f

the materials When analyzing the content subjectively and objectively, researchers should focus on some sub-criteria (language description, language points, language skills, text- types, topics, content organization, and sequence)

Methodology, the last criterion, concerns the theories o f learning, the learners’ attitudes to

learning English, types o f tasks/exercises, teaching-learning techniques, aids, language guidance and flexibility o f the materials

In addition to the main criteria above, Hutchinson & Waters (1987) also refer to some other criteria, for instance, the price and the availability o f the materials

2.2.5.2 Evaluation M odel by McDonough & Shaw

McDonough & Shaw (1993) offer a three-stage evaluation model (see Figure 2) As indicated in Figure 2，the first stage, the external evaluation, concentrates on the claims

made for the materials by the author/ publisher Next, the internal evaluation is concerned with analyzing the extent to which the factors in the External evaluation actually match the internal consistency and organization o f the materials as stated by the author/ publisher The final stage, the overall evaluation, deals with the factors that help any evaluator to

aims to enable the evaluator to decide the extent to which the materials have realized theirstated objectives.

Figure 2: Evaluation Model by McDonough & Shaw (1993: 75)

Also, according to this model, the following factors need to be examined in each stage:The External evaluation The In te rn a l evaluation The overall evaluation

丁he intended audience

The pronciency level

The context and presentation

The table o f contents

The use o f visuals and

The inclusion o f tests

The treatment and presentation o f the skills The sequencing and grading

o f the materials The type o f reading，

listening, speaking and writing materials The appropriateness o f tests and exercises,

The self-study provision The teacher-leamer

6 balance’ in use o f the materials

The usability fa cto r, the

extent to which the materials can be integrated into a particular syllabus as 'core，

o r 4 supplementary9

The generalizability fa cto r:

the usefulness o f some parts

o f the materials

The adaptability fa cto r: the

possibility o f materials modification such as adding/extracting or using several parts o f the materials

in another context

The fle x ib ility fa cto r: the

flexibility o f sequencing and grading; the flexibility o f the materials themselves to be integrated easily into various types o f syllabus

Table 1: The Factors that need examining in the materials evaluation process

In the materials evaluation process，McDonough & Shaw note that it is unnecessary to examine both external and internal evaluation simultaneously After completing the first

stage, the evaluator can decide the materials’ appropriateness to adopt or select them I f the evaluation shows the materials to be potentially appropriate and need to be more inspected, the internal evaluation w ill be conducted I f not, it is advisable to exit at the first stage and

evaluate other materials i f necessary Even after the second stage, evaluators still have the option not to select the materials i f they so wish However, this is often avoided i f a detailed inspection o f the materials is undertaken Also, the success or failure o f the materials can be determined only after those materials are used in the classroom with real learners

2.2.5.3 Evaluation Model by Ellis

Figure 3: Evaluation Model by Ellis (1998: 228)

Ellis (1998) argues that any evaluations are likely to be the results o f a whole series o f

suggests a model that can be best carried out as a series o f micro-evaluations (see Figure 3) This modal aims to identify the match between task planned and task in use Some dimensions used for macro evaluation such as approach, purpose, focus, scope, evaluator, and types o f information can be applied in a micro evaluation process.

According to the framework, evaluating a task consists o f five main steps: describing the task, planning the evaluation, collecting information, analyzing information, reaching conclusions, and making recommendations for future teaching

In summary, E llis’ model may be more manageable and specific than the ones o f Hutchinson & Waters, McDonough & Shaw when evaluating a task is implemented However, E llis’ model may be time-consuming and require a lot o f efforts i f there is a comprehensive demand for materials evaluation McDonough & Shaw’ s model may save time i f the materials can be seen inappropriately at the first stage However, i f the first evaluation shows that the materials may be potentially suitable, and need to be more inspected, then a second evaluation needs to be carried out Perhaps, what, first, distinguishes Hutchinson & Waters9 model from other models is the clear-cut and logical description o f the stages Second, the proposed criteria are presented more simply and

systematically which can help evaluators visualize what must be done to carry out materials

evaluation Third, the guidelines offer a straightforward analytical matching process to evaluators As a result, the researcher decided to choose Hutchinson & Waters9 model with some modifications to follow in the thesis

2.2.6 Tools fo r gathering inform ation

Robinson (1991) suggests seven basic procedures for data collection in materials evaluation: questionnaires, checklists, interviews, observation, discussion, existing information (records), and assessment

♦ Questionnaires

Questionnaires are any written instruments that present respondents with a series o f questions or statements to which they are to react either by writing out their answers or selecting from among existing answers (Brown 2001: 6) They are the most widely used

tool for many reasons First, they are relatively cheap, quick, and efficient, and probably better suited to large-scale surveys Second，the data are more likely to be standardized, uniform, and consistent across subjects Third, they can be anonymous, control subconscious bias, and other sources o f bias (Brown 2001: 77) However, the obvious issues with questionnaires are respondents’ reluctance to fill in and return them, or occasionally their possible misinterpretation o f questions (Robinson 1991; Brown 2001).

♦ Checklists

To some certain extent, a checklist is similar to a questionnaire Usually, a checklist is much shorter than a questionnaire and can be used when the evaluation focuses on small- scale aspects o f a program, for example, speaking skill or reading skill o f a syllabus (Robinson 1991:70)

♦ Interviews

Interviews are procedures used for gathering oral data in particular categories ( if the interview is well planned and structured in advance), but also for gathering data that was not anticipated at the outset (Brown 2001: 5) Interviews can be conducted with individuals,

in groups, or by telephone for some reasons First, interviews are worthy o f later follow-up study through structured questionnaires and provide the opportunity for more extended exploration o f the points than do questionnaires or checklists Second, they can have a high return rate with fewer incomplete answers, involve realia, control over the environment and over the order or make other observations Finally, they are relatively flexible, personal, and complex with quite rich data (written and spoken) (Brown 2001: 75) Nonetheless, they are relatively time-consuming, expensive, typically done on a small-scale, and can never be completely anonymous

♦ Observation

Observations involve direct on-the-spot examination o f language use, learning, or training (Brown 2001: 4) Also, classroom observation is essential to fUmish the evaluator with assumed important process variables that characterize the work in the classroom (Mackay

quite easy to adm inister N evertheless, it m ay not be easy to produce w ell-focusedconclusions because the data is so unstructured and multi-faceted or it can be d ifficu lt to predict what sort o f material you w ill get, and so on (Wray, Trott, Bloomer, with Reay & Butler 1998: 187 Wray et a l 1998: 187) In addition, it may be possible to observe past

students in operation at work to see whether their course has prepared them effectively or not (Robinson 1991:71)

♦ Discussion

This technique refers to class discussion on a regular part o f a program, for instance, speaking activities, followed by a report from each class summarizing what has been said (Robinson 1991: 71)

♦ Existing inform ation (Records)

Existing information aims to utilize any preexisting information that may be available It can include internal sources (such as files or records that may be on hand when the needs analysis begins), or external sources (such as library resources or letters exchanged with other existing programs w ith similar students (Brown 1995: 46) Existing information, therefore, should be taken into consideration before wasting energy developing brand-new information-gathering tools (Brown 2001: 3) Robinson (1991: 71) uses the term records

instead o f existing inform ation Records may include the original planning documents,

statements o f needs, aims and objectives, teacher’ s lesson records, teachers and students’ diaries o f their experience, the w riting letter or postcard, etc

♦ Assessment

Assessment traditionally includes tests, examinations, and the evaluation o f students’ projects, written work, etc (Robinson 1991: 71) Assessment can provide fairly efficient ways o f gathering large-scale information for a variety o f purposes on students’ abilities, knowledge, or skill (Brown 2001: 4)

In short, all o f these tools have their strengths and weaknesses Nevertheless, basing on the condition, time, and the real context, the researcher decided to select existing information and questionnaires to implement the study.

in the subsequent courses

Tran Thi Thuy Nga (2005) used document analysis, questionnaires, and interviews to judge

the English material for the first and second year Marine students at Nha Tran g Navy Academy The purpose o f her study is to examine how far English fo r M arine Cadets

realizes the prescribed course objectives and meets the course requirements on the content and methodology Three teachers o f English and a syllabus designer were interviewed, and

85 second-year marine students filled in the questionnaires She exploited the evaluation model by Hutchinson & Waters (1987) The findings show that the material basically matches the aims and methodology o f the course requirements, but partly accords with the course content requirements, for example, insufficient reading skills and pronunciation As

a result, supplementing and adapting materials were made to improve the material

Vu Thi Yen Nga’ s study (2004) may be viewed as a signpost for the researcher since there was not a syllabus in her context She evaluated Lifelines (Pre-Intermediate) in the teaching

context o f Quang Ninh Teachers’ Training College The main purpose o f the study is to

course's aims Document analysis and questionnaires were exploited respectively toanalyze the textbook and to find the students^ preferences and difficulties Six teachers o f English and 220 students participated in the study The findings state that the aims o f the textbook and the course coincide but its content and methodology are not completely appropriate to the students' knowledge level in terms o f some language points, micro-skills, topics，etc Therefore, adaptation is the only best way to overcome the mismatch above.

Le Thi Ngoe Diep (2006) assessed Powerbase- Elementary for first year students at

National Economics University The instruments she employed are the same as Vu Thi Yen Nga (2004) with the participation o f 150 first-year students and 10 teachers o f English Her thesis aims to find out whether the textbook satisfies the requirements o f the English Department in terms o f the aims, content and methodology or not Consequently, she found that some requirements were met in terms o f the amount o f grammar, structures, language functions, macro-skills, micro-skills, the text types, topics and the methodology Moreover, she also explored some weaknesses, for example, easy grammatical exercises, vocabulary and some difficult tasks o f listening, speaking and Curriculum Vitae (C.V) w riting skills, English cultures, pronunciation From the findings o f the thesis, she made some suggestions，for instance, adding grammatical, vocabulary and pronunciation exercises, c.v

w riting tasks, and replacing some difficult writing/listening tasks

In conclusion, A kin & Guceri’ s study may be the most systematical, time-consuming and expensive For Vietnamese studies, most o f them have the same features as follows: using questionnaires and document analysis, post-use evaluations, using Hutchinson & Waters’ model flexibly, adaptation recommendations However, it would have been better i f these researchers had applied more instruments for all participants and graphs to present the results

2.4 Adapting m aterials

2.4.1 Reasons fo r adapting materials

It is likely that the reasons for adapting materials by McDonough & Shaw are the most specific and useful for evaluators to compare and make references

McDonough & Shaw (1993: 86) provide a list o f reasons as follows:

♦ Not enough grammar coverage in general or practice o f grammar points o f particular difficulty these learners;

♦ The communicative focus means that grammar is presented unsystematically;

♦ Reading passages contain too much unknown vocabulary;

♦ Comprehension questions are too easy, because the answers can be lifted directly from the text with no real understanding;

♦ Listening passages are inauthentic, because they sound too much like written material being read out;

♦ Not enough guidance on pronunciation;

♦ Subject-matter inappropriate for learners o f this age and intellectual level;

♦ Photographs and other illustrative material not culturally acceptable;

♦ Amount o f material too great/ too little to cover in the time allocated to lessons;

♦ No guidance for teachers on handling group work and role play activities with a large class;

♦ Dialogues too formal, and not really representative o f everyday speech;

♦ Audio material difficult to use because o f problems to do with room size and technical equipment;

♦ Too much or too little variety in the activities; vocabulary list and a key to the exercises would be helpful;

♦ Accompanying tests needed

In addition, Cunningsworth (1995: 136- 137) also refers to some reasons for adapting

♦ Methods (e.g., an exercise may be too mechanical, lacking in meaning, too complicated).

♦ Language content (e.g., there may be too much emphasis on grammar items that your students learn easily and not enough emphasis on those that they find difficult)

♦ Subject matter (e.g., topics may not be interesting to the students or they may be outdated

or not authentic enough)

♦ Balance o f skills (e.g., there may be too much emphasis on skills in the written language

or on skills in the spoken language, or there may not be enough on integrating skills)

♦ Progression and grading (the order o f language items may need changing to fit an outside syllabus or the staging may need to be made steeper or shallower)

♦ Cultural content (there may be some cultural references that need omitting or changing)

♦ Image (a course book may project an unfriendly image through poor layout, low quality visuals, etc)

2.4.2 Techniques fo r adapting materials

Richards (2001: 260) offers six procedures for materials adaptation: m odifying content^ adding o r deleting content, reorganizing content, addressing omissions, m odifying tasks,

Maley (1998: 281) suggests some strategies to make the published course bearable, or more effective First, give it a rest means that additional materials not in the course book (songs,

rhymes, games, cartoons, off-air recordings, video clips, etc) are introduced and sometimes used as “ warmers” or “ coolers” Second, change it by a number o f available options:

omissions, addition, reduction, extension, rewritirìg/m odifìcation, replacement, re-ordering, and branching.

Cunningsworth (1995: 136- 138) notes that adapting materials involves one or more o f the following: leaving out some parts o f the material; adding material (published or your own); replacing material w ith something more suitable; changing the published material to make

it more suitable for your use Also, a number o f different areas may need supplementing,

but among the commonest are: reading, listening, pronunciation, vocabulary, and grammar

McDonough & Shaw's techniques may be the most specific and clearest because they propose a framework for adaptation (see Figure 4) According to the framework, there are five main techniques for adapting materials: adding including expanding and extending; deleting including subtracting and abridging; modifying including re-writing and re­

structuring; S im plifying; and re-o rd erin g

♦ A dding

The notion o f addition refers to the process o f supplementing materials by putting more into them quantitatively or qualitatively The quantitative means that more o f the same component in the materials is supplied without changing the methodology or the methodological framework o f the original materials For instance, i f students find the explanation o f a new grammar point rather difficult, further exercises must be added before they begin the practice material On the contrary, the qualitative way refers to the development o f the methodology in new directions, i.e by putting a new component into the materials For instance, i f there is insufficient coverage o f the listening skill, the reading passage provided may also be paralleled by the provision o f listening comprehension material including the same vocabulary and ideas

♦ Deleting or omitting

Deletion is clearly the opposite process to that o f addition, i.e things deemed inappropriate, offensive, unproductive for a particular group must be left out For instance, i f some o f the language functions presented in a communicative material are considered to be less useful for learners to use in the target language environment, they can be cut out \jiv in g сИгес^оп^ or 'greetings’ may be useful whereas other functions such as 4expressing sympathy’ may not

♦ Sim plifying

Simplification, basically, is a re-writing activity o f modification Many elements o f a material can be simplified However, this technique has mainly applied to texts (reading passages) The texts with their sentence structures, the grammatical structures and lexical content are often simplified in order to be closer to the proficiency level o f a particular group o f learners.

Figure 4: Materials adaptation framework by McDonough & Shaw (1993: 96)

♦ Modifying

M odifying means an internal change in the approach or focus o f an exercise or other piece

o f material This technique can be applied to any aspect o f "content，and sub-divided into two specific ways: re-writing and re-structuring Re-writing refers to the modification o f the linguistic content For example, i f easy end-of-text comprehension questions just require the students to use information directly from the text to answer, they can be modified so that students have to interpret what they have read or relate different sections o f the text to each other Re-Structuring, unlike re-writing, applies to classroom management, specifically, to the structuring o f the class For example, there are some role-play activities for group o f certain size in the materials I f students are in a large class, it is necessary to assign one role to a number o f students at the same time In a nutshell, the modification o f the materials can be a change o f a somewhat small component in that material, such as an exercise, a text or at a classroom activity

♦ Re-ordering

This technique suggests the possibility o f putting parts o f a material in a different order, or adjusting the sequence o f presentation within a unit, or taking units in a different sequence from that originally intended For example, “ w ill，，，“ going to” indicating the future are often presented in a great deal o f materials However, it is advisable for teachers to show the relationship between time reference and grammatical tense in a more accurate way Thus, the simple present and present continuous which also indicate the future should be included

in the material

C H A P T E R I I I : M E T H O D O L O G Y

Chapter 3 consists o f three parts: data collection instruments, participants, and data collection procedures

3.1 Data collection tools

Evaluating A H I turns two instruments: existing information and questionnaires into

advantage 丁his section presents the statement o f each tool, and the designing o f questionnaires

3.1.1 The statement o f the existing inform ation

As pointed out in the literature review, existing information should always be taken into account before wasting energy developing brand-new information-gathering tools In other words, it should definitely be considered first, in order to avoid the problem o f reinventing

a wheel that already exists (Brown 2001: 4) Creswell (2003) uses the term documents

instead o f existing inform ation Documents include public documents (e.g., minutes o f

meetings, and newspapers), private document (e.g., journals, diaries, and letters), and e- mail discussions Documents enable researchers to obtain the language and words o f

participants, to be accessed at a time convenient, to gain thoughtful data, and to save the time and expense o f transcribing, etc Nonetheless, documents have to face with the

following weaknesses: the unavailability o f protected information, the requirement o f searching out the information in hard-to-find places, the incompletion, and the unauthenticity or inaccuracy o f the documents (Creswell 2003: 187)

In fact, for materials evaluation, existing inform ation is seen as a very essential data

collection instrument In this thesis, existing information comes from the literature, the existing textbook (student book, teachers’ book, workbook), and the course aims By this tool, the data o f the course and textbook aims, the textbook content, and the textbook methodology were gathered

3.1.2 Questionnaires

3.1.2.1 The statement o f the questionnaires

A questionnaire is a set o f questions on a topic or group o f topics designed to be answered

by a respondent (Richards et a l 1992: 303) Questionnaires are easy not only to get

information from a lot o f people very quickly with low cost in time and money but also to analyze answers o f closed questions, convenient for respondents to fill at any time with less pressure for an immediate response Also, they can provide suggestive data for testing a hypothesis and make respondents more or less avoid interviewer bias (Gillham 2000: 6; Creswell 2003: 186) Moreover, questionnaires are best used in association with other types

o f data elicitation because a fuller picture o f the data can be assessed i f it is approached from more than one angel (Wray et a l 1998: 167) Nonetheless, questionnaires have some

disadvantages, for instance, low response rate unless sample 'captive,，problems o f motivating respondents, the need for brevity and relatively simple questions, lack o f control over order and context o f answering questions, respondent literacy problems, and so forth (Gillham 2000: 8) Hence, it is important to design valid, reliable, and unambiguous questionnaires (Richards et a l 1992: 303).

In order to reduce the weak points o f the questionnaires, the researcher made an attempt to:

♦ keep the questionnaire as short and simple as possible;

♦ translate the questionnaires into Vietnamese to avoid misunderstanding, to ensure the accuracy o f the data, and to make the students & teachers feel free to express their specific thoughts and opinions;

♦ design more closed-response questions than open-response questions;

♦ explain clearly each item in the questionnaires before the students as well as the teachers fill in them to avoid getting the wrong responses;

♦ use quantitative questionnaires to make the data easier to analyze;

♦ carry out a pilot study in which some respondents were asked to complete the questionnaires and stated how/ whether there is any misunderstanding Certainly, the researcher adapted some vague questions before they were administered on a large-scale;

♦ gather questionnaires immediately after an hour in class in order to maximize return rate

In summary, the researcher selected questionnaires to collect data for the textbook evaluation since it originated from the aims o f the thesis (finding out the students and the teachers’ opinions and attitudes towards the textbook on a large-scale), and the desire to exploit the advantages o f questionnaires stated

3.1.2.2 Question formats

There are two basic kinds o f response-question formats: open-response questions and closed-response questions (Brown 2001: 35-44)

♦ Open-response questions

The open-response questions are ones for which the respondents create the answers orally

or in writing The open-response questions are exploratory in nature, relatively easy to write, and diversified in answers Nevertheless, they have more disadvantages than closed- response questions For example, they are relatively d ifficu lt to answer, to code, to analyze,

to interpret, to show reliability and validity or the respondents are more likely to skip, resulting in some answers being irrelevant and relatively subjective They have two forms

o f questions: fill- in questions and short-answer questions which both require the

respondents to fill in information in their own words However, they differ from the length

o f answers, for instance, usually one word for fill- in questions but phrases or sentences for short-answer questions.

♦ Closed-response questions

The closed-response questions are ones for which respondents must select from among optional answers presented as part o f the question They have more advantages than open- response ones and are commonly exploited For instance, they are relatively easy to answer,

to code, to analyze, to interpret, to show reliability and validity or they provide more

uniform ity across questions with relatively objective answers H ow ever, they are fairlynarrow in providing a range o f possible answers, less exploratory in nature, and relatively

d ifficu lt to write They have four main forms o f questions: alternative answer questions, Likert-scaie questions, checklist questions, and ranking questions Alternative answer questions can be any questions that provide alternative answers from which respondents

must select They are varied and usually imposed bv the imagination o f the questionnaire

designer Likerî-scale questions are effective for gathering respondents’ views, opinions,

and attitudes about language-related issues ranging on dimensions: e.g., very serious to slight, important to unimportant, like to dislike, and agree to disagree Checklist questions

are any ones that present a list all o f the items that apply to them Ranking questions are

ones that require the respondents to rank order concepts, objects, people, and so forth The ranking is usually done in terms o f some overall scale, such as least to most interesting, least to most important, etc

3.1.2.3 Student questionnaires

The student questionnaires (see Appendix 2) start with open-response questions (fill-in and short-answer ones) in order to explore the students9 personal information: class, sex,

English books already used, years o f English study, and purpose o f learning English Then,

there are 15 questions aimed to gather the students’ judgments about the textbook A H I

mainly on the aims, content, and methodology These questions are subdivided into four main parts:

Part I involves students’ evaluation on the aims o f the textbook

Part II investigates students’ judgments on the textbook content

Part III examines the students’ opinions on the textbook methodology

Part IV explores the information on the suggestions for the textbook improvement

Table 2 below presents the detailed information o f the questionnaires Most o f the questions above are closed-response ones Therefore, the data can be completed quickly and analyzed

Parts Types o f inform ation and Question form ats QuestionsI

The aims o f the textbook relating to the course aims Likert-scale

(l=excellent, 2=good, 3=adequate, 4=weak, 5=totally lacking)

Q.l

The opinions on the amount o f language points, and language skills

Likert-scale (too much-adequate-not much-not at all).

Q.2

I I

The d ifficu lt level o f the language points and the language skills

Likert-scale (very difficult-difficult-normal-easy-very easy).

Q.3

The variety and attraction o f the text-types

The fam iliarity and interest o f the topics

Likert -scale (strongly agree, agree, neutral, disagree, strongly disagree).

The necessary level o f the missing parts

Ranking (from the most to least necessary)

Q.9

Some overall opinions about the textbook and the workbook

L ike rt scale (1= strongly agree, 2= agree, 3= neutral, 4= disagree, 5=

strongly disagree)

Q l l

Time and difficulties for self-studying Short answer Q.12

IV Suggestions for the textbook improvement

Short-answer

Q.13 + 14+ 15

Table 2: The detailed information o f the student questionnaires

3.1.2.4 Teacher questionnaires

The teacher questionnaire (see Appendix 3) also contains four parts under the headings:

Aims o f American Headway 1，Content o f American Headway 1，Methodology o f American

some insight into the four parts, some open-response questions were exploited to seek out the teachers’ personal information such as age, qualification, working experience, and time working with A H I.

Part I aims at exploring the teachers’ opinions on the aims o f the textbook

Part II seeks out their judgments on the textbook content

Part III tries to examine their assessments on the textbook methodology

Part IV concentrates on suggestions for the textbook improvement.

In general, the teacher questionnaires are the same as the student questionnaires Only Question no 12 and no 15 are a bit different Question no 12 only tries to discover their opinions on students' difficulties in self-studying English at home Question no 15 aims to find out their ideas on the changes o f the content sequenced within a unit and throughout the textbook

3.2 Subjects

As indicated in chapter II, both outsiders and insiders are regarded as precious sources for materials evaluation (Robinson 1991; Tomlinson 1998; Dudley-Evans and St John 1998; and Richards 2001) Nonetheless, in this study, only insiders including teachers and students got involved in the textbook evaluation

3.2.1 The researcher as a subject

Existing information analysis was implemented by only one subject- the researcher who took the responsibility for gathering the data from the literature, the course aims, the textbook and its accompanying workbook, teachers9 book The researcher here, one o f the teachers using A H I in EPU, was viewed as an insider She may have a quite deep

understanding o f the context, the level o f students, and the textbook She was closely concerned with the evaluation process including collecting, analyzing the data, and giving recommendations As a consequence, her subjective points o f views may be imposed in the thesis Certainly, the students and teachers’ judgments can help the researcher to reduce this limitation