000084266 EVALUATING THE ESP TEXTBOOK APPLIED FOR TERM 3 INFORMATION TECHNOLOGY STUDENTS AT THE FOUNDATION STUDIES DEPARTMENT, HANOI UNIVERSITY ĐÁNH GIÁ SÁCH GIÁO KHOA ESP DÙNG CHO SINH VIÊN CÔNG NGHỆ THÔNG TIN HỌC KỲ 3 TẠI KHOA CƠ SỞ, TRƯỜNG ĐẠI HỌC HÀ N

000084266 EVALUATING THE ESP TEXTBOOK APPLIED FOR TERM 3 INFORMATION TECHNOLOGY STUDENTS AT THE FOUNDATION STUDIES DEPARTMENT, HANOI UNIVERSITY ĐÁNH GIÁ SÁCH GIÁO KHOA ESP DÙNG CHO SINH VIÊN CÔNG NGHỆ THÔNG TIN HỌC KỲ 3 TẠI KHOA CƠ SỞ, TRƯỜNG ĐẠI HỌC HÀ NỘI

0 0 0 0 8 4 2 6 6

M IN ISTRY OF ED U CA TIO N A N D TRAINING

H A N O I U N IV E R S I T Y

1

P H A M X U A N H U Y

EVALUATING THE ESP TEXTBOOK

APPLIED FOR TERM 3 INFORMATION

submitted in partial fulfilment

o f requirements o f the degree

o f Master in T E S O L

supervisor: N G U Y EN TIEN DUNG M.A

Hanoi

A u g u s t 2014

TA BLE O F CO NTENTS PAGE

C H A PTER ONE: INTRODUCTION

C H A PTER TW O: LITERATURE REVIEW

2.2.2 M odels for the Evaluation o f Language Teaching

C H A PTER TH REE: M ETH O DO LO G Y

C H A PTER FIVE: R E C O M M E N D A T IO N S AND

5.1 Recom m endations 41

5.1.3 Recommendations on the m ethodology

5.1.4 Som e suggestions on teaching and learning with the 43material

5.3 Lim itations o f the study and suggestions for further research 47

A ppendix 2: Q uestionnaire on evaluating m aterials for students 53

A ppendix 7: The checklist o f Hutchinson & W aters ( 1987) 66

N a m e o f th e sis: EV A LU A TIN G THE ESP TEXTBOOK APPLIED FOR TERM 3

IN FO R M A TIO N TEC H N O LO G Y STU DEN TS AT TH E FOUNDATION STUDIES

D EPA R TM EN T HANOI UNIVERSITY

A C K N O W L E D G E M E N T S

First o f all, I w ould like to express m y gratitude to m y thesis supervisor M r N guyen Tien Dung, M A w ho has given me useful com m ents, rem arks and engagem ent in correcting the drafts o f my thesis

Furtherm ore, I w ould like to thank all the m em bers o f the D epartm ent o f Postgraduates Studies at Hanoi U niversity for their valuable assistance, lectures and recom m endations

My special thanks go to Dr N guyen Van Trao D ean o f the D epartm ent o f Postgraduates Studies Hanoi U niversity and Dr H oang G ia Thu, Dean o f the Faculty o f M anagem ent and Tourism , Hanoi U niversity for their relentless help and encouragem ent

A lso, I am thankful to m y colleagues and students at Hanoi U niversity w ho have w illingly shared th eir precious tim e during the survey questionnaires process

H ow ever, this acknow ledgem ent w ould not be com plete if 1 did not express m y gratitude

to m y loved ones, w ho have supported and encouraged m e throughout the entire process,

both by keeping me harm onious and helping m e putting pieces together I will be grateful

forever for their love

A BSTR AC T

'O xfo rd English fo r C om puting by Keith B oeckner and P Charles B rown has been used

as the textbook for Information Technology students in term 3 at the Foundation Studies

D epartm ent, Hanoi U niversity since 2006 This study aim ed to evaluate the above textbook by applying the checklist o f Hutchinson and W aters (1987) in term s o f aims, content and m ethodology to determ ine w hether the material is suitable for the students according to their needs The results o f the research proves that the m aterial m eets most o f the aims, content and m ethodology o f the ESP course Based on the findings, some recom m endations about how to continue using the m aterial in the future o r what changes

to be m ade to im prove the effectiveness o f the m aterial for future use are put forward follow ed by suggestion for future research

LIST O F A B B R EV IA T IO N S

LIST OF FIG U R ES AN D TABLES

PAGE

Figure 2: The M aterial Evaluation Model proposed by M cD onough and Shaw 10(1993, p 75)

Figure 4: The m aterial evaluation m odel o f Hutchinson and W aters (1987,

T able la Means, standard deviations, frequencies, and percentages o f the

T able lb M eans, standard deviations, frequencies, and percentages o f the

T able 2a Means, standard deviations, frequencies, and percentages o f the

T able 2b M eans, standard deviations, frequencies, and percentages o f the

Table 3a M eans, standard deviations, frequencies, and percentages o f the

Table 3b Means, standard deviations, frequencies, and percentages o f the

Table 4a Means, standard deviations, frequencies, and percentages o f the

Table 4b M eans, standard deviations, frequencies, and percentages o f the

Table 5a M eans, standard deviations, frequencies, and percentages o f the

Table 5b M eans, standard deviations, frequencies, and percentages o f the

Table 6a M eans, standard deviations, frequencies, and percentages o f the

T able 6b M eans, standard deviations, frequencies, and percentages o f the

v

CH A PTER 1: INTRO DUCTIO N 1.1 BACK G RO UND TO T H E STUDY

During the language teaching and learning process, m aterials play a very important role, especially in ESP which is indispensable Learning is enhanced through materials,

w hich assist teaching and enable both the instructors and learners to achieve their aims

A ccording to Hutchinson and W aters, textbook evaluation is basically a straightforw ard, analytical ‘m atching process: m atching needs to available solutions’ (H utchinson and W aters, 1987 p.97) M y ow n view is that this issue is rather more

em otive and controversial for teachers; m any w ould agree with Sw ales (1980) that textbooks, especially course-books, represent a ‘problem ’, and in extrem e cases are exam ples o f educational failure I w ould like to explore the reasons for such strong reactions, and to put forward possible evaluative solutions 1 wish to concentrate on coursebooks because, w hether we like it or not, these represent for both students and teachers the visible heart o f any ELT program This high profde m eans that the definition and application o f system atic criteria for assessing course-books are vital Supplem entary textbooks and m aterials on the other hand may not carry the same burden The evaluative criteria for these can to some extent remain im plicit, or beallowed to define them selves more inform ally in the local situation

Evaluation is a process in w hich the aim s o f the evaluation, the subject o fevaluation, evaluators, types o f evaluation, data collection tools are all considered

A ccording to Brown (1995), evaluation is one o f the elem ents o f a dynam ic system which

is view ed as language teaching and language program developm ent To clarify this view point, Robinson (1991) initiates a chain o f “ wh-” questions to lead one through the

m ost im portant points in relation to evaluation com prising “ Why carry out an evaluation?” , “ W hat is the subject o f the evaluation?” , “ who carries out the evaluation?”

“ How is the evaluation carried out?” , “ W hat next: what will happen to the results?” Robinson, 1991, p.66)

'T extb o ok ’ is generally defined by M icrosoft Encarta (2005) as a book that treats a subject com prehensively and is used by students as a basis for study This term , also called

text, is the m ost prevalent term to refer to a book exploited in teaching-learning situations

In many circum stances, the expectation is that teaching will be based on a single textbook, although other materials may be used at the teacher's discretion Textbooks as well as ESP give teachers guidance in course and activity design Furthermore, they satisfy students'

demand or desire o f acquiring som ething specifically to work from and for further study at home N evertheless, the selection o f a suitable ESP textbook is not an easy jo b for both teachers and academic managers Hence, this paper discusses ESP textbook evaluation to

be used in ESL classroom s as Hutchinson and W aters (1987, p.96) notes, "evaluation is a

m atter o f judging the fitness o f som ething for a particular purpose” M oreover, “ in any kind o f evaluation, the decision finally m ade is likely to be the better for being based on a system atic check o f all the important variables" (H utchinson and W aters 1987, p.96) The evaluation could also lead to “a large investm ent o f money in a published course or a large investm ent o f time in hom e-produced or adapted m aterials.” (ibid, p.97) Evaluation can

be seen to be necessary in language teaching and studying

Hanoi University is a public one founded on the basis o f Hanoi U niversity o f Foreign Language Studies (established in 1959) It trains language students and recently has offered some new m ajor subjects such as Business A dm inistration, Tourism and International studies, and Inform ation Technology (IT) Besides, English is considered as the language o f instruction at Hanoi U niversity and it is used to teach all professional subjects thus the students should have a very good English proficiency so as to attend the English-m edium instructed courses D uring the first year at Hanoi U niversity, students are required to learn English for three term s, each o f which lasts ten weeks In the first two term s, students learn the basics o f English with the textbooks aim ing at reviewing, consolidating and developing their language proficiency, acquiring basic English before taking the ESP (English for Specific Purposes) course in the third term Fort those who

m ajor in Information Technology, the textbook "Oxford English for C om puting" is used to fam iliarize the students with fundam ental concepts and vocabulary related to their IT major

‘'O xford English fo r C om puting' by Keith Boeckner and P C harles Brown,

published in 1993 by O xford U niversity Press is an ESP textbook for students m ajoring in Inform ation Technology and it has been w idely used in many universities and colleges in Vietnam , such as Can Tho U niversity, Posts and Telecom m unications Institute o f Technology, the FPT University, the Industrial U niversity o f Ho Chi M inh City - Thai Binh branch and in other countries (for instance, KBU International College in M alaysia,

A G H University o f Science and Technology in Poland, etc) The texbook has been used for Information Technology students in term 3 at Hanoi University since 2007, but it has not been evaluated on a thorough basis ever since

1.2 AIM S OF THE STUDY

1.2.1 Aims

The study aim s at evaluating the ESP coursebook English fo r Com puting, by

applying the checklist o f Hutchinson and W aters (1987) in terms o f aim s, content and

m ethodology to determ ine w hether the m aterial is suitable for the students according to their needs Based on the findings, som e suggestions about whether to continue using the material in the future or what changes to be m ade improve the effectiveness o f the material for future use

1.2.2 T he research questions

The study will be carried out to answ er the following research questions:

1 To what extent is the ESP coursebook, O xford English f o r C om puting, suitable to

IT students at the Foundation Studies D epartment Hanoi U niversity in terms o faims, content and m ethodology?

2 What suggestions can be offered to adapt the ESP coursebook to m ake it moreappropriate?

1.3 Scope o f the study

Pending previous researchers, the evaluator o f the study will consider the ESP

coursebook, O xford English fo r Computing, in four stages including criteria definition,

subjective analysis, objective analysis and m atching with the m odel suggested by Hutchinson and W aters (1987) The coursebook will be evaluated to find out w hether it is prospectively appropriate to the students at the Foundation Studies D epartm ent, Hanoi

U niversity by considering three criteria, namely the aims, content a n d methodology.

1.4 The significance o f the study

The ESP coursebook undoubtedly plays an important part in the English learning process o f the students at the Foundation Studies D epartment, Hanoi U niversity as it is assigned to be the prescribed textbook for ESP teaching This evaluation may give the academ ic managers and teachers o f the Foundation Studies Department, Hanoi University necessary information about the strengths and w eaknesses o f the ESP coursebook and what should be done to improve its quality T he study will offer the students who are using the coursebook a chance to express their ideas about the book and the ESP course, which is quite meaningful to the researcher and the teachers o f English at the Foundation Studies Department, Hanoi University as well

CH A PTER 2 LITER A TU R E REVIEW

This chapter aim s to provide a fram ework to assess O xford E nglish f o r C om puting

It consists o f five main parts: the first shows the m aterials in English language program The next part defines materials evaluation in w hich the author dem onstrates the roles o f material evaluation, theoretical studies on textbook/m aterial evaluation and the textbook evaluation checklist The third w orks on research instruments The fourth part presents a num ber o f previous studies done on m aterial evaluation in different contexts and the last one is the sum mary o f this chapter

2.1 T he roles o f textbooks in language teaching and learning

2.1.1 T he roles o f textbooks

Textbooks play an im portant role in teaching and learning a subject ‘T extbook' is a term used together with ‘coursebook’ and 'm aterial’ to indicate a supporting means o f teaching and learning These term s have been used interchangeably by different linguists

o r educators in their research G ower, Philips, and W alters (2005) asserted that

“coursebook usually com prises a set o f materials: student’s or pupil’s book (to be used in class), student’s o r pupil’s workbook (often used for supplem entary class w ork, individual

w ork o r hom ew ork), cassettes (for use in class o r at hom e), teacher’s book and sometimes even a video.” (G ow er et al., 2005, p.77) These m aterials are often used as the aids by teachers at schools and colleges to develop different aspects o f language Harm er (2009)

m entioned that a good coursebook includes appropriate language, content and sequencing, and the fascinating topics as well as treatm ent o f various language skills

G ow er et al., (2005), G raves (2000), and Ur (1996) also show th e similarity in discussing the roles and the importance o f textbooks in learning and teaching G ow er et al., (2005) refer to many advantages o f using a good coursebook such as providing security for teachers and students, a syllabus graded roughly to the level o f the students, giving variety and a balanced diet o f language work, continuity and progression, a ready­

m ade source o f tried and tested activities, providing guidelines for lessons and m entioning possible difficulties G raves (2000) indicates that a textbook provides a syllabus for the course, the security for the students, the basis for assessing students, helps teachers save tim e in finding and developing such m aterials, gives teacher supporting m aterials (e.g., teachers guide, cassettes, w orksheets, video) Besides, the textbooks can provide consistency w ithin a program across a given level, if all teachers use the sam e textbook If textbooks follow a sequence, it provides consistency betw een levels Ur (1996) also has

the same view o f the roles o f the textbooks when stating that textbooks serve as a syllabus,

a supply o f ready-made texts and tasks, and instructions to inexperienced teachers

M oreover, she m entions the econom y o f textbooks by considering a book as the cheapest way o f providing material for each learner as well as the convenience by referring that a textbook is a convenient package Besides, she concludes that a textbook can helps a learner becom e m ore autonom ous because without textbooks a learner m ay depend more

on their teacher, which should be avoided

From these findings, one can conclude that textbooks are very useful not only for the teachers but also for the learners Therefore, both the teachers and learners should make use o f the advantages o f the textbooks for better teaching and learning M oreover, the knowledge and the criteria o f the textbooks can be used for assessm ent o f students’ learning

To use a coursebook in the m ost effective way, how ever, we need to consider not only its advantages but also disadvantages G ow er et al., (2005) identifies several pitfalls

o f using a coursebook Firstly, "it is not easy to find a coursebook that will suit the needs and interests o f all students in terms o f the language syllabus and the topics o f the texts.’’ (ibid, p 77), hence, the teacher will have to use the sam e coursebook for students o f different age and background As a result, learners may not like the book and not be

w illing to use it Secondly, using a coursebook only can m ake students feel bored and result in teachers’ less being creative in their search for extra m aterials to interest and

m otivate students Ur (1996) also notes the draw backs o f using a textbook including inadequacy, irrelevance, lack o f interest, limitation, hom ogeneity, and overeasiness

2.1.2 Roles o f ESP coursebooks

A ccording to D udley-Evans and St John (1998), ESP m eets specific learners' needs, uses "underlying m ethodology and activities o f the discipline it serves” (Dudley-Evans and St John 1998) and is centered on language appropriate to these activities in terms o f

“ gram m ar, lexis, register, skills, discourse and genre” (El-Sakran 2012 p 1) They also

state that ESP may be designed for specific disciplines, using different m ethodology from that o f general English Especially, it is “ likely to be designed for adult learners, either at a tertiary level or in a professional work situation” (ibid, p 1) M oreover, ESP assumes som e basic knowledge o f the language system

It can be seen that ESP course in general and ESP textbooks in particular are necessary for students when learning their professional subjects because they provide know ledge relating to the m ajor and the professional w ork situations

To recapitulate, teaching m aterials such as English textbooks and ESP tcxtbooks/coursebooks are very im portant in teaching and learning languages Therefore,

it is necessary to design or select a good teaching material for a course Besides the advantages, the limitations o f using textbooks cannot be ignored It m eans that the textbook/coursebook which has been used o r is going to be used as m ajor teaching

m aterial for a course should be evaluated in order to find out w hether that textbook/coursebook is suitable to its subject or not, or what should changes or

im provem ents should be m ade make it a trustw orthy material

2.2 M aterial evaluation

There are so many researchers who have made a num bers o f definitions o f materials evaluation They all have tried to provide a m ore com prehensive and inclusive definitions

D udley-Evans and John (1998, p 128) suppose that evaluation is a com plete process

“ which begins with determ ining what inform ation to gather and ends w ith bringing about change in current activities or influencing future one" Brown (1995) defines materials evaluation “ is the com plete process o f determ ining the suitability o f the m aterials for a particular program ” (p.159)

A m ong various definitions o f m aterials evaluation, Tom linson (1998, p.3) defines

m aterials evaluation as "the system atic appraisal o f the value o f m aterials in relation to their objectives and to the objectives o f the learners using them " and this definition seems

to be the m ost w idely accepted Via this definition, Tom linson (1998) implicitly indicates that the value o f a material should be determ ined by considering w hether the learning points are potential useful to the learners, w hether the learning syllabus can be suitable for the learners' needs Obviously, this definition is well aligned with the learner-centered approach

C oncurrently Hutchinson and W aters (1987) stated that “evaluation is concerned

w ith relative m erit” They also m ake it clear that w hether the book is praiseworthy depends largely on its suitability to the local setting as “there is no absolute good or bad- only degree o f fitness for the required purposes” (p.96) They share the idea with

T om linson that "the evaluation process should be system atic and is the best seen as

m atching exercise: m atching your analyzed needs to available solution” (H utchinson and

W aters, 1987, p 105) Thus, the essence o f this definition is its attention to the students' needs in evaluating materials

It is clear that no textbook is perfect and the ability to evaluate teaching materials

effectively is a very im portant professional activity for all EFL teachers (M cD onough & Shaw 1993) Therefore, the evaluation o f English language course book for any level o f education is a decisive process and the decision for evaluating the course b o o k 's suitability according to the le arn er's needs and teachers' requirem ents has to be considered as an important activity.

As Brown (1995) defines, textbook evaluation o r material evaluation is the process

o f determ ining "the suitability o f the m aterials for a particular program ” (B row n, 1995, p 159) Hutchinson and W ater (1987) identifies evaluation as “a m atter o f ju d g in g the fitness

o f something for a particular purpose-’ (ibid, p 96) Rea-Diekens and G erm aine (1992), quoted in Soori, K afipour, and Soury (2011) also consider evaluation as “a dynamic process which investigates the suitability and appropriateness o f an existing practice" (p 481)

In conclusion, m aterials evaluation is an im portant area in teaching ESP because this process offers a practical result o f effective course developm ent and provides students with m aterials that will equip them with the know ledge they will need in their professional subject Through the definitions o f m aterials evaluation, it can be inferred that m aterials evaluation involves the ju d g m en ts o f the value o f the m aterials in relation to the objectives and requirem ents determ ined

2.2.1 Roles o f m aterial evaluation

In ESP, according to D udley-Evans and John (1998 p 121), m aterial evaluation is

an indispensable part o f the key stages in ESP process

E valuation -► Need analysis

Teaching - learning

Figure 1 T h e key stages in ESP process

D udley-Evans and John (1998, p 121) claim that needs analysis is the process o f establishing the w hat and the how o f the course, and the evaluation is the process o f establishing the effectiveness Therefore, after being used in the classroom , teaching

m aterials need to be evaluated to see if they have worked well for the intended contexts

and students Because m aterials evaluation helps to identify particular the strengths and the w eaknesses o f the m aterials in used (Sheldon 1998) The strong points o f the m aterials and the shortcom ings o f certain exercises, tasks, and entire texts will help the researcher judge the effectiveness and efficiencies o f the materials The results will lead to some suggestions o f changes o f the m aterials or o f any other elem ents o f the program.

A great num ber o f textbooks/m aterials that can be selected as the teaching materials for a subject Nevertheless, it is necessary to decide which is the most suitable for a certain group o f students To make that feasible, researchers should carry out the material evaluation process Besides, the appraisal will help to identify certain strengths and

w eaknesses o f the m aterials in use From that, the teachers or evaluators w ill find out the solution to overcome its limitations

Hutchinson and W aters (1987) m aintain that the evaluation results may lead to "a large investment o f money in a published course or a large investment o f time in home- produced or adapted m aterials." (p 97)

Rea-Dickens and G erm aine (1992), quoted in Soori, Kafipour, and Soury (2011) also add, evaluation is "a useful device for both teachers and m aterial writers as an underlying elem ent in the developm ent o f innovations and m odifications within the teaching/ learning context.” (p 481)

As already mentioned the role o f m aterials, a textbook or courscbook should be adequate to m eet all the needs o f the students during the course to reach achievem ent in target language learning That is why textbook selection and evaluation for a course appears to be a very important issue for all parties involved in the m aterials selection and evaluation process

2.2.2 M odels for the Evaluation o f Language T eaching Material

There have been m any m odels suggested for evaluation o f language teaching

m aterials To different researchers, different types o f evaluation are proposed These specific types are based on a certain point o f tim e, purpose, context, etc Ur (1996) introduces a model o f evaluation w ith three stages including “deciding on criteria”,

“ applying criteria” , and “ sum m ary” In the “deciding on criteria", Ur (1996) m entions that

th e evaluators should study the list o f criteria w hich is available in (Ur, 1996, p 186) and decide the level o f importance o f each criterion In order to categorize the criteria more effectively, the evaluators can w ork in group and com pare their ideas w ith those o f the colleagues In the second stage named “applying criteria”, the evaluators should take a locally-used coursebook and exam ine it using the criteria in the list m entioned above In

the last stage called "sum m ary", the evaluators should make some overall evaluation o f the coursebook It is likely most useful to work on this summary with other evaluators who work on the sam e book with yours Stevick (1971), cited by Brown (1995), suggests that the m aterials should be evaluated in term s o f qualities, dim ensions, and com ponents There are three qualities including strength, lightness, and transparency (as opposed to

w eakness, heaviness, opacity) Three dim ensions consist o f linguistic, social and topical ones Four com ponents include occasions for use, sample o f language use, lexical exploration, and exploration o f structural relationships

Ellis (1997) suggests the practice o f a detail em pirical evaluation and focuses on evaluation at the task level with reference to its actual teaching and learning context This model w ants to identify the m atch betw een task plan and task in use

A ccording to Ellis, some dim ensions used for macro evaluation such as approach, purpose, focus, scopes, evaluator and types o f information can be applied in micro evaluation process He introduces the follow ing steps when this m odel is applied

1 C hoosing a task to evaluate

2 D escribing the task which requires the specification o f the content o f a task in term s o f input, procedures, language activities and outcom es

3 Planning the evaluation w hich involves answ ering a num ber o f questions such as purpose o f evaluation, audience, evaluator, content, m ethod and timing

4 C ollecting the information w hich is about how the task was perform ed, what learning took place as a result o f perform ing the task, and teacher’s and learner’s opinion about the tasks

5 A nalyzing the inform ation w hich involves tw o types o f analyzing data, namely, quantitative and qualitative analysis o f the information collected

6 Reaching conclusions and m aking recom m endations in w hich reaching conclusions relating to what has been discovered as a result o f the evaluation o f the task and m aking recom m endations for future teaching

7 W riting the report

(p.237)

M cD onough and Shaw (1993) suggest another model o f m aterial evaluation, a com bination o f macro-evaluation and m icro-evaluation w ith three stages: external evaluation, internal evaluation and an overall evaluation External evaluation offers, a

b rief ‘'overview ” o f the m aterials from the outside It consists o f an exam ination o f the claim s m ade on the cover o f the students’ and teacher's books, the introduction and the

tabic o f contents The evaluation, at this stage, aim s to find out w hether the material is potentially suitable for m ore detailed analysis It is claim ed that this should be followed by

an internal evaluation w hich requires an in-depth investigation into the m aterials What is important for the evaluator at this stage is finding out to what extent the factors mentioned

at the external evaluation stage match up with the internal consistency and organization o f the materials It is strongly claim ed that in order to exam ine what w ill actually be presented in the textbook such as skills, the grading exercises and tasks, the methodology, the teaching and learning techniques, etc at least two units or more units o f the book are required to be examined The last stage in this model is called overall evaluation The overall evaluation concerns the suitability o f the m aterials by considering the evaluative param eters such as the usability factor, the generalizability, adaptability, and flexibility factor This m odel can be seen in detail in Figure 1 below:

M acro-evaluation —— > inappropriate/potentially appropriate —

Littlejohn (1998) suggests a prelim inary frame work for m aterials analysis and evaluation consists o f four steps as follows:

A n a ly sis o f the ta r g e t situ a tio n o f M a te ria l a n a ly sis

use

F rom a n aly sis:

- T h e c u ltu re c o n te x t - W h at is th e ir e x p lic it n a tu re ?

- T h e c o u rse (p ro p o se d a im s, c o n ten t - W h at is im p lied by th e ir use?

S u p p le m e n t th e m a te ria l m a k e th e m a terial a c ritic a l o b je ct

Figure 3: T h e M aterial E valuation M odel proposed by Littlejohn (1998)

2.2.3 C riteria for M aterial Evaluation

M aterials evaluation is a com plex m atter because there are many variables that may

affect the success or failure o f textbooks when they are in use Therefore, developing criteria to assess materials is one o f the m ajor steps to be taken in all the proposed models for materials evaluation It is also necessary to assess the suitability o f m aterials to the needs o f learners in particular teaching o r learning situations because it contributes to the effectiveness o f that particular teaching or learning situation in several classroom settings.

Ur (1996, p 186) introduce nineteen criteria for coursebook assessm ent such as explicit objectives; approach educationally and socially acceptable to target community; clear, attractive layout; appropriate visual m aterials; interesting topics and tasks; varied topics and tasks; clear instruction; system atic coverage o f syllabus; clearly organized and graded content; periodic review and test sections; plenty o f authentic language, good pronunciation explanation and practice; good vocabulary explanation and practice; good gram m ar presentation and practice; fluency practice in all four skills; encourages learners

to develop their ow n learning strategies; adequate guidance for the teacher; audio cassettes; and readily available locally

Brown (1995) develops a checklist for adopting textbooks consisting o f five perspectives such as m aterials background (including author’s education and experience, and publisher’s reputation); fit to curriculum (in terms o f approach, syllabus, needs, goals and objectives, and content); physical characteristics (such as layout, organization, editorial qualities and m aterial quality); logistical characteristics (such as price, auxiliary parts and availability); and teachability (including teacher edition, review and acceptability

am ong teachers)

It can be seen that criteria offered by different authors for the evaluation o f materials range from a set o f guidelines to m ore com prehensive checklist H ow ever, since there is a not strict criterion that may be suitable and applicable in all teaching o r learning situations

w hen the concern is the evaluation o f m aterials, the selection o f the criteria may be quite subjective Since the teaching and learning environm ents are different, the background, needs and interest o f learners in these environm ents are not the same, criteria applied in each study are also different A s a result, it is necessary to study carefully the different criteria and checklist for textbooks to find out the m ost suitable one for each teaching and learning environment

2.2.4 ESP coursebook evaluation

Hutchinson and W aters (1987) introduce a m odel in which m aterials evaluation is concerned as a m atching process in which the suitability o f m aterials for a specific purpose is questioned in term s o f the needs identified This m atching process has four

- Define the criteria on w hich the evaluation will be based

- A nalyze the nature and underlying principles o f the particular teaching/learning situation

- A nalyze the nature and underlying principles o f the available m aterials

- C om pare the findings o f the two analyses

These stages are described apparently in the figure 3 below:

Figure 4: T he m aterial evaluation m odel o f H utchinson and W aters (1987, p.96)

In sum m ary, Hutchinson and W aters (1987) suggest a logical m odel for materials evaluation It helps evaluators know exactly what m ust be done to analyze the m aterials in com parison with the course requirements

H utchinson and W aters (1987) also suggest a num ber o f criteria to evaluate the

m aterials Som e m ajor criteria are audience, aim s, content, and methodology In the first place, to evaluate the audience, consideration should be taken into ages, sex, nationalities, know ledge o f English and specialties, etc Next, the aim s o f the course and the aim s o f the

m aterials should be identified Then, the content o f the m aterials should be evaluated in term s o f language, skills, texts, and content organization Finally, the m ethodology o f the textbook w'ould be exam ined in term s o f learning theories, learners' expectations about learning English, kinds o f exercises, teaching/learning techniques, aids, guidance for

teaching and flexibility o f the materials.

El-Sakran (2012) based on the ideas o f Ellis (1997) and M cD onough and Shaw (1993) mentioned the two types o f evaluation o f an ESP textbook including external evaluation and internal evaluation External evaluation exam ines w hat the textbook

w riter(s) say about the book, and careful exam ination details on the intended audience and proficiency level; the context o f use; presentation o f materials, grading, layout and methodology; supplem entary m aterials and learning resources; visuals contained, and culture Internal evaluation covers the skills to be developed and their presentation in the materials; supporting audio or video recordings; authenticity o f m aterials; suitability o f

m aterials for different learning styles, and individual; pair and group learning activities.Robinson (1991) introduces three types o f m aterials evaluation: preliminary, sum m ative and formative Preliminary evaluation is norm ally carried out before an ESP course begins to select the m ost appropriate from the m aterials that are available Sum m ative evaluation takes place at the end o f a course to inform that w hether the

m aterials have been effective o r not Form ative evaluation is conducted during the course

to modify the materials

Robinson (1991) also refers to a series o f “w h-"questions to guide evaluators through m ain points related to evaluation such as rationale, subject, evaluator, procedures for data collection, and discussion o f the results The question o f “ W hy carry out an evaluation?” m entions the rationale for evaluation The subject o f evaluation is referred in the question o f “ W hat is the subject o f the evaluation exercise?” The subjects in different types o f evaluation are not the same Form ative evaluations typically look at such factors

as teachers' and students’ attitudes or at the usability o f new instructional m aterials, while sum m ative evaluations may look at all aspects o f a program The evaluators including outsiders and insiders are m entioned in the question o f “ W ho carries out the evaluation?”

O utsider is a person w ho has not been involved in the program Insiders are course designers, teachers, and representatives from the m anagem ent and may be students The question o f “ How is the evaluation carried out?” indicates the basic procedures for data collection including questionnaires, checklist, rating scales, interview s, observation, discussion, records, and assessm ent T he last question o f “ What happens next?” shows

w hat will happen to the results o f evaluation

In conclusion, there have been a lot o f studies o f evaluating teaching m aterials or textbook; however, each researcher aim ed at a different teaching m aterial Each one has also chosen his/her ow n research m ethodology in order to get his/her ow n aim s depending

on the requirements o f his/her context.

2.3 Previous studies on m aterial evaluation

In each nation, the m aterials used in learning and teaching are different There have been a lot o f studies evaluating the textbooks o r m aterials which are being used at schools,

or colleges Each study aim s at its ow n objectives depending on the m aterials and the learners' needs, thus, the researchers select their ow n m ethodology to carry out their research

ESP m aterials are the useful m eans in language teaching "T eaching m aterials are tools that can be figuratively cut up into com ponent pieces and then rearranged to suit the needs, abilities, and interests o f the students in the course." (G raves 1999 p.27) They are anything that the teacher may use in order to help learners get better understanding Such

m aterials can exist in different forms: they can be textbooks, workbooks, texts, activities,

In language teaching, anything which can be used by teachers or learners to

facilitate the learning o f a language Materials may be linguistic, visual,

auditory, or kinesthetic, and they may be presented in print, audio or video

form, on CD-ROM S, on the Internet or through live performance or display.

(Longman Dictionary o f Language Teaching and Applied Linguistics)

ESP materials must be varied so that to attract and m otivate students; they appear

in most times interesting, fun and clear Som etim es, m aterials are appropriate for the content but som etim es they need to be m odified and creative; the teacher has to adapt or supplem ent in order to fit the local context

ESP m aterials can be done for different reasons and can be presented in different forms Kennedy and Bolitho (op.cit) points out that ESP materials are helpful in making students read then get the meaning, read then write such as sum m arizing o r paraphrasing, listen when teachers read from them , speak when the teacher motivates his students to com m unicate, or do different tasks when the teacher integrates different skills Activities are very useful m eans in language teaching They are o f different forms: reading, listening, speaking and writing Each specialty is based on some skills according to students' needs and the form o f the present lesson

In Littlejohn's view point, a close analysis o f m aterials them selves should be seen

as a prelim inary step to materials evaluation Thus, "w hen com paring a description o f a textbook with a description o f a context in order to establish w hether that textbook might

be suitable for that context we are evaluating’’ (cited in M cGrath 2002:22) It can be

consequently inferred that evaluation differentiates itself from analysis in the sense o f context-relatedness In the same vein, Hutchinson& W aters regard what distinguishes evaluation from analysis is that “evaluation is [then], concerned with relative m erit" (H utchinson T., & W aters, A 1987:96) They also m ake it clear that w hether the book is praisew orthy depends largely on its suitability to the local setting, as “there is no absolute good or bad - only degrees o f fitness for the required purposes.“

M aterials evaluation or more specifically textbook evaluation has been studied from different research perspectives A close study o f the literature show s that there is no consensus am ong researchers’ uses o f the term ‘approach' in m aterials evaluation

M cGrath, I (2002) synchronizes “a system atic approach" with “a cyclical approach" with three stages A popular classification is that o f Ellis, R (1997) w ho states that the literature on educational evaluation identifies two broad approaches: the ‘objectives

m odel' and the ‘responsive evaluation’ based on quantitative and qualitative standpoints respectively

M aterials should also be contextualised to the experiences, realities and first languages o f the learners An im portant part o f this involves an aw areness on the part o f the teacher-designer o f the “sociao-cultural appropriacy” (Jolly & Bolitho, 1998 p l 11) o f things such as the designer’s ow n style o f presenting material, o f arranging groups, and so

on It is essential the m aterials designer is inform ed about the culture-specific learning processes o f the intended learners, and for many groups this may m ean adjusting the intended balance o f what teachers m ay regard as m ore enjoyable activities and those o f a

m ore serious nature M aterials should link explicitly to what the learners already know, to their first language and cultures, and very importantly, should alert learners to any areas o f significant cultural differences

Litz (2005) in his evaluation o f a textbook (English Firsthand 2 ) used in South

K orea aim ed to determ ine the overall pedagogical value and suitability o f the book tow ards the desired and attainable goals o f the new ly structured interm ediate EFL curriculum Survey questionnaires w ere used as the data collection instrum ents The questionnaire consisted o f the follow ing categories: practical considerations (price, accessories, m ethodology etc.), layout and design, range and balance o f activities, skills appropriateness and integration, social and cultural considerations, subject content, and language types presented in the textbook Besides, they were used in analyzing students' needs in o rder to “ make a clear picture o f the com patibility betw een actual students' needs and the perceived goals and objectives o f the EFL program " (p 11) At the end o f this

study, he provided some recom m endations to improve the book He suggested the inclusion o f more consciousness-raising activities, genuine negotiation o f meaning tasks, and effective cooperation learning strategies.

So far there have been num erous studies done by V ietnam ese researchers about textbook evaluation Those who have evaluated ESP m aterials in their studies include Nguyen Phuong D ung's (2002), Vo Thi A nh D ao's (2003), Nguyen Thi Bich Lien's (2004), Vu Phuong H ien's (2004), Hoang Hoa L ien 's (2005), Nguyen Thanh Liem 's (2006), Tran Trung D ung's (2009) The review o f the studies below could help select the suitable research m ethodology to conduct this study

Using the criteria proposed by H utchison and W aters (1987), N guyen Thi Bich Lien (2004) found out whether the course-book Life-lines Pre-interm ediate was appropriate to the program and the students' needs in term s o f aim s, content and methodology The subjects o f the study were 100 third-year students and 7 teachers o f Foreign Languages D epartment in Ha Noi Junior T eachers' Training College The author realized that after sum m arizing them , such actions o f adopting, rejecting, adapting, supplem enting were important for a com plete evaluation process Thus, a combination model o f Littlejohn and Hutchison was the m ost suitable for her study The methods used

in the study were docum ent analysis, questionnaire for the students and interviews with the teachers

In another study, Vu Phuong Hien (2004) shared the sam e purpose as to measure

w hether the material was suitable to the course objectives and she also used the same data collection instruments, i.e docum ent analysis, interview s and questionnaires However, her study only ceased here, m aterial adaptation was not given to im prove the book; therefore, its conclusion for m ajor findings only served for references

A nother evaluation was conducted by Tran Trung Dung (2009) o f the current textbook used for the first year non-English m ajor students at V ietnam University o f Commerce In this study, its author tried to investigate w hether the textbook met the students' need in terms o f aim s and objectives, content and m ethodology The results in the study showed that the course book should be adapted to use again as its aim s and objectives o f the course book were appropriate to the students' needs However, some changes must be made so as to make the learning and teaching o f English at VUC more effective Based on the findings, som e suggestions regarding adding, modifying, and restructuring some certain item s in the book were made Besides, the m ethodology used

for teaching the course book was also adjusted so as to facilitate the students most in their learning.

In conclusion, many aspects o f evaluation in general and evaluation o f materials in particular in ESL/EFL classroom s have been discussed in num erous studies However, the evaluation o f ESP coursebooks used in ESL classroom s o f Hanoi U niversity has been hardly found Therefore, it is essential to carry out a study on this issue T he evaluation o f

the textbook O xford English fo r C om puting is to be conducted with an attem pt to realize

its features discussed above:

systematicity context-relatedness taking into account students' opinions

From what has been review ed above, the m ethods selected for this study would be docum ent analysis, questionnaires for the students and interview s with the teachers

The follow ing parts enclosing approaches, types and the roles o f m aterials evaluation will

be pursued in light o f these characteristics

C H A PT E R III M ETH O DO LO G Y 3.1 Setting

The study was be carried out to answ er two research questions: “T o what extent is

the ESP coursebook O xford English fo r Computing, suitable to IT students at the

Foundation Studies Department Hanoi U niversity in term s o f aim s, content and

m ethodology o f the textbook?” and "W hat suggestions can be offered to adapt the ESP coursebook to make it more appropriate?” The evaluation process followed the model o f four stages including defining criteria, subjective analysis, objective analysis, and

m atching with three criteria suggested by Hutchinson and W aters (1987) nam ely the aims, content, and methodology The study w as conducted at the Foundation Studies

D epartm ent, Hanoi University from April to M ay, 2014 The questionnaires were be delivered to the students and a num ber o f students were also interviewed in their self-study time in the afternoon Som e teachers w ere be interviewed in their w orking places at their convenience

3.2 Data collection instrum ents

It is necessary to have scientific m ethods to evaluate effectively m aterials There are several instrum ents to choose Brown (1995), Hutchinson and W aters (1987), Robinson (1991) claim that most m aterials are evaluated using one or more o f the following instrum ents: docum ent analysis, questionnaires, interviews, checklist, records, and assessm ent

In the context o f this study, three o f the above m entioned instruments: questionnaires, interview and docum ent analysis were employed The docum ent analysis

w as necessary for the evaluator to understand the nature o f the m aterials and the course requirem ents The questionnaires enabled the perceptions o f teachers and students to be obtained It is believed that these instrum ents were suitable with the ev alu ato r's time constraint and the materials evaluation purpose

3.2.1 Docum ent analysis

Document analysis is the use o f the content and other techniques to analyze and sum m arize printed m aterials and existing information (Taylor, Steel Douglah, 1996)

D ocum ent for evaluation can be in the form o f original planning docum ent, statement o f needs, aims, and objectives com ing from the syllabus and teaching m aterials (Robinson 1991) A ccording to Brown (1995) and Nunan (1992), this instrument is used to collect qualitative data that concern with qualities and non-num erical characteristics In evaluation, the analysis o f existing docum ent can provide useful inform ation One o f the

biggest advantages o f docum ent analysis is in some situations, a docum ent represents a reflection o f reality, so it is useful for determ ining value, interest or public attitudes.

In order for subjective analysis to be carried out the curriculum o f the ESP course including aim s, syllabus and content and m ethodology were be analyzed The ESP

coursebook English f o r Computing, used at the Foundation Studies Departm ent Hanoi

U niversity w as assessed to carry out the analysis The evaluation process followed some

o f the criteria suggested by Hutchinson and W aters (1987) such as aim s (w hat are the aims

o f the m aterials?), content (types o f linguistic description, language point, skills, kinds o f texts, subject-m atter areas, topics, organization o f content), and m ethodology (theory/ies

o f learning the materials based on, kinds o f exercises/tasks, teaching-learning techniques, aids, guidance the materials provide, and flexibility o f materials) Due to the time limit the researcher carried out the study o f the coursebook by exam ining its content and

m ethodology in which some specific criteria w ould be considered such as topics, skills, gram m ar and vocabulary, exercises/tasks, learning techniques The m atching stage led to the conclusion that the coursebook is potentially appropriate to the students at the Foundation Studies D epartment, Hanoi U niversity A fter that, questionnaires were designed and delivered to the students

ESP Syllabus Analysis

The requirem ents o f ESP syllabus com prise a num ber o f aspects such as aims, content, m ethodology, tim e allocation and the format o f the test In this analysis, a prudent look w ould be paid to the aim s and objective o f the course as well as its implied requirem ents for the material in term s o f content and m ethodology For the detailed description o f the ESP syllabus, see A ppendix 1

ESP M aterial Analysis

The second docum ent investigated in this study is English f o r Com puting This

analysis is carried out by em ploying three criteria for m aterials evaluation established by

H utchinson and W aters (1987) w hich proved to have led to num erous ESP material evaluation projects in Vietnam in order to have a clear sight o f the m aterial The analysis aimed at exam ining w hether the m aterial un use met the actual aim s, content and

m ethodology o f the course requirem ents o r not

3.2.2 Q uestionnaires

In this study, the questionnaire aim s to look for supplem entary inform ation from the teacher and student subjects on the aim s, content and m ethodology o f the ESP material The choice o f questionnaires as a m ethod to collect data for evaluation com es from the

following reasons: Firstly, the students can complete the questionnaires at any time convenient for them so there is less pressure for an immediate response Secondly, it is easy to get information from a large num ber o f participants at the same tim e with high response rate (W eible & W allace, 1998) Q uestionnaire is one o f the most popular instrum ents to collect data (N unan, 1992) There are several different types o f questionnaires in a survey research but the m ajor types used in education are mailed questionnaires and electronic questionnaires (Crcswell, 2005) The best advantage o f this instrum ent is that it provides an easy quick form o f data collection Therefore, data can be collected on a large scale and reach a geographically dispersed sam ple o f a population

M oreover, it is economical because it involves only duplication or m ailed expenses (C resw ell 2005)

Q uestionnaires items can be relatively closed o r open ended A closed item is one in

w hich the range o f possible responses is determ ined by the researcher (yes-no response or selection o f an item) An open item is one in w hich the subject can decide w hat to say and how to say it Close-ended questions in the questionnaire helped the researcher analyze the data conveniently Q uestionnaires can consist entirely o f closed questions, entirely o f open questions or m ixture o f closed and open questions (N unan 1992, p 143) In questionnaires, a Likert scale is com m only used, and it is the m ost w idely used scale in survey research The Likert technique presents a set o f attitude statem ents (available at

w w w ciiltsock.indirect.co.uk/M uH om e/cshtm /Likert.htm l) Subjects are asked to express agreem ent or disagreem ent o f a five-point scale: “ strongly disagree", “ disagree“, “neither agree nor disagree", “agree” , and “ strongly agree” Som etim es a four-point scale is used since the m iddle option o f “ neither disagree nor agree” is not available

A part from the advantages, questionnaires also express negative features Creswell (2005, p 136) claim s that “ individuals may lack any personal investment in the study and decide not to return the instrum ent” Besides, the researcher does not have a m eans for explaining questions so respondents m ay m isinterpret item s in the questionnaires (C resw ell, 2005)

In this study, the questionnaire for students was designed based on Hutchinson and

W aters (1987) checklist o f criteria for evaluation There were two sets o f questionnaires to confirm w hether or not and how appropriate the ESP coursebook is to HANU students, one for students and one for teachers Both sets o f questionnaires w ere designed to investigate the opinion o f teachers and students tow ards appropriateness o f the textbook, concentrating in some criteria o f the textbook as m entioned above such as aims, content

and m ethodology The two sets o f questionnaires were in English as th e students and teachers are proficient enough to understand the language o f the questionnaires and it is easier for the researcher to collect the authentic ideas from them.

3.2.3 Interviews A ccording to Cresw ell (2005, p 136) interview is "a form on

which the researcher asks the questions from an interview guide, listens for answ-ers or observes behaviors, and records responses on the survey”

D epending on the evaluation goals and the specific evaluation context, the evaluators can make use o f the following three types o f qualitative interview format: the structured, sem i-structured, and unstructured (Patton cited in Lynch, 1996) The sem i­structured interview' has been the m ost favorite instrument in the field o f interpretive researcher B row n (1995) believes that this kind o f interview can be collected with richer data than a structured interview o r questionnaires

G enerally, interview m ay be face-to-face, voice-to-voice, or screen-to-screen (Brown, 1995; Robinson, 1991) Creswell (2005) highlights three m ajor types o f interview s used in education: one-on-one interview s, focus group interview s, and telephone interview s This can help interview ers get true opinions o f the participants involved and it makes easy to com pare answers “ R esearcher m ay also prejudice participant’s answ ers, know ingly o r unknow ingly, though either com m ents or body language” (C resw ell, 2005, p.361) H ow ever, som e authors such as Brow n (1995), Taylor and Steel (1996) claim that interview s are tim e-consum ing and expensive Cresw ell (2005,

p 3 6 1) has pointed out “ not all interview ees are com fortably disclosing inform ation about them selves during the interview ”

Interview s will be carried out after the data from questionnaires w hich are collected

A num ber o f questions will be set up to ask teachers and students for m ore in-depth inform ation These questions will be related to the students’ experiences in learning the ESP coursebook, their opinions on som e aspects in content and m ethodology o f the textbook Besides, some teachers will be asked about their opinion, experiences, and assessm ent on some issues in the coursebook

3.3 Participants o f study

3.3.1 T he teachers

There w ill be five out o f over fifty teachers o f English at the Foundation Studies

D epartm ent, Hanoi U niversity, aged from 26 to 40 participating in the study The chosen teachers have to be experienced in w orking with the ESP coursebook for at least three years, and they will be still using this coursebook for teaching when th e study is being

conducted After getting the data from questionnaires, five o f the teachers w ho have been leaching this textbook will be selected to for the interview.

3.3.2 The students

About 50 first-year students (aged from 19 to 21 ) in four classes will take part in this study All o f them will be learning from the ESP coursebook when the study is being conducted By the time the questionnaires are delivered to them (April & M ay 2014) they will have finished one-third o f the units in the coursebook After getting the data from the questionnaires, 8 o f 100 students were selected to take part in the interview which was carried out by the evaluator A nother 50 students were second-year Information

Technology students These students had studied the whole textbook o f English fo r

C om puting the previous year; therefore they know which know ledge studied from the

book has been applied for their second year o f professional subject

3.4 Data collection procedures

3.4.1 Document analysis

The criteria for evaluation identified were based on the purpose o f the study The criteria chosen focused on the aims, the content and m ethodology o f the m aterial Then the researcher analyzed the m aterial basing on the defined criteria to identify the actual aims, the content, the methodology A fter analyzing the material, the researcher made a com parison between the results o f the m aterial analysis with the requirem ents o f the ESP course

3.4.2 Q uestionnaire

The questionnaire for m aterial evaluation w as adm inistered to 100 students from the four classes o f Inform ation Technology students o f HANU 50 o f w hom are first year

students who are studying the coursebook English f o r C om puting, and the rest o f 50

students are second-year student o f Inform ation Technology o f HANU Before completing the questionnaire, the students were asked to read through all the questions so that any difficulties in understanding and answ ering the questions w ould be explained carefully by the researcher A fter the questionnaires from students were returned, they were checked again to find if there were any m istakes and any changes needed Then the researcher sum m arized for analysis from 100 copies o f the questionnaire w hich w ere distributed to the students There were also 5 questionnaires for the teachers They were also distributed

to ESP teachers o f English w ho had been teaching this subject for the past several years in Foundation Studies D epartment Hanoi University From the results achieved, a com parison between subjective analysis and objective analysis was m ade to find out

w hether they m atch each other or not The research findings help to find out both the match and mismatch between the m aterial and the requirem ents o f the syllabus Based on those findings, som e suggestions were m ade to improve the effectiveness o f the material for future use.

3.4.3 Interview

A fter that, the ESP teachers o f English would be invited to express their viewpoints

on the ESP material by taking part in an interview with the researcher T he interview plan

w ith 10 guiding questions was brought to the interview as an aid to guide the researcher during the interview The guide also w orked as a backup and checklist to m ake sure all the issues have been discusses However, the guide was not followed in great details, as the interview ees were preferred to be in charge o f the discussion them selves W hile the interview was being carried out, the researcher kept a copy o f the interview questions to guide him through the interview and he could take notes directly on the guide

The interview with the two teachers w as then conducted to probe into the research findings The time and place for the interview was set up due to the arrangem ent between the researcher and the participants The entire interview took the participants over h alf an hour long to com plete On the whole, the interview w as conducted with the interviewees

in a relaxed and frankly spoken manner

Interview data were then qualitatively analyzed to provide the researcher useful indication to find out the m ism atch betw een the ESP m aterial and the content requirements

o f the course as well as ways o f im proving the effectiveness o f the m aterial for future use

3.5 Summary

This chapter has given a detailed description o f the subjects o f the study, instrum ents and procedures o f data collection A total o f 5 ESP teachers o f English and 100 IT students o f HANU w ere recruited Tw o kinds o f existing docum ents were used to form a basis for evaluation Thus, based on the three above defined criteria: the aims, the content and the

m ethodology, the questionnaire and interview s are designed and conducted to generate

m ore evidence on the com patibility betw een the ESP m aterials and the course requirement

CH A PTER 4: RESULTS AND DISCUSSION

In this chapter, the results o f the students' and teachers' opinions about w hether the textbook O xford English for Com puting [Keith Boeekner, P C harles Brown] could meet the requirem ents o f the ESP course and the discussion o f these results are demonstrated The judgm ent is categorized following the checklist o f Hutchinson and W aters (1987) in terms o f aim s, content and m ethodology Based on the result o f the analysis, discussions about the appropriateness o f the textbook stated above will be provided

4.1 Aim s o f the course and the material

The appropriateness o f the textbook above is determ ined upon the analysis o f the aim s o f the ESP course The defined aim s o f the course for students o f com puting w ere stated with the following aims:

• To w hat extent the ESP coursebook, O xford English fo r C om puting, is suitable to

students at Foundation Studies D epartment, Hanoi University in terms o f aims, content and m ethodology;

• W hat suggestions can be offered to adapt the ESP coursebook to make it more appropriate

By the end o f the course, students will be able to:

• B uild their vocabulary base in com puting subject

• D evelop multiple English language skills to com prehend and discuss authentic topics in Computing efficiently

• Increase their comfort and confidence for their m ajor course

From the docum ent analysis above, it could be concluded whether the m aterials basically met partly the aim s o f the course

Then, to find out if the aim s o f the m aterials m eet the students' needs, the researcher com pared the m aterials evaluation w ith the teachers and students' opinions from the questionnaires

Results o f the Data Analysis

M aterial analysis is implemented based on the student and teacher questionnaires After collecting th e questionnaires, the researcher conducts data analysis to offer tables for analysis

For the first table about Aims, the first research question was: “To what extent is the ESP course book suitable to IT students at the Foundation Studies D epartm ent, Hanoi

U niversity in terms o f aim s? The section o f •Aims’ in the checklist had four items for

which the frequencies (n), percentages (% ), m eans (M ) and standard deviations (SD ) were estimated (see Table 1).

T able la M eans, standard deviations, frequencies, and percentages o f the question

on Aim s for students.

Note: Strongly disagree = 1, Disagree = 2, Neutral = 3, Agree = 4 Strongly agree = 5

Table lb M eans, standard deviations, frequencies, and percentages o f the question

on Aim s for teachcrs.

Note: Strongly disagree = 1, Disagree = 2, Neutral = 3, Agree = 4 Strongly agree = 5

A s Table 1 illustrates, over h a lf o f the students (62% ) declared that they had learnt a lot o f w ords related to their specialized field from the textbook, while not many learners (13% ) disagreed with the item (M = 3.55, SD = 968)

A lso corresponding to this opinion, all the teachers from Agree (20% ) to Strongly Agree (80% ) revealed that the book had helped their students to learn and increase technical vocabulary (M = 4.80, SD = 447)

N early h a lf o f the students (40% ) believed that their gram m atical knowledge had

im proved after learning this textbook, likewise 40% offered neutral ideas and only 20% stated their disagreem ent (M = 3.23, SD = 908) A bout this claim , m ore than h alf o f the teachers (60% ) said the book helped their students improve their gram m ar knowledge, only 20% disagreed

For Question 3 whether the book could develop the reading skill o f the students to help them read m aterials related to their majors, nearly 1/3 o f the students (37%) expressed neutral ideas, only 15% disagreed while nearly h alf the students ( 48% ) stated their agreem ent and all the teachers (100% ) strongly agreed (M = 5.00 SD = 000).

Next, with Question 4, not many students (33% ) stated that they could w rite on the subjects related to their specialized field after learning this textbook while most o f the teachers (80% ) agreed with this statem ent (M = 4.00, SD = 707)

For the last item, over h a lf (54% ) stated their agreem ent that they could attend discussions on the subjects related to their specialized field after learning this textbook and most o f the teachers (80% ) also agreed to that (M = 3.80 SD = 477)

Discussion

As shown in the Table la for students, the highest mean (3.55) in the Aim s section is related to the vocabulary Among 100 students, 13% revealed disagreem ent, 25% neutral and 62% expressed their agreem ent This m eans most students showed their agreement about the aim o f vocabulary and they offered positive evaluation that the book helped them study and increase their technical vocabulary This could be easily understood because this is an ESP textbook, with its focus on specialized vocabulary and the grammar

is understood to have been covered sufficiently in the earlier stages from the BEL and

EA P stages T he lowest mean (3.13) related to the aim on writing about the topics o f students' specialized field after learning this textbook The results indicate that the writing

section o f the textbook, O xford English fo r C om puting, requires some m odifications based

on the students’ needs and interest For this, sufficient variety should also be considered in selecting the aim s for w riting o f the textbook

Sim ilarly, the results obtained in table lb o f teachers showed the m ean range o f the aim section w as from 3.60 (lowest) and 5.00 (highest) This revealed th e aim s that the book posed w as close to the agreem ent scale Hence, it seemed that the aim s posed o f the book could achieve the results as expected

C unningsw orth (1995) points out that there should be a correspondence among textbooks, aim s and objectives o f the language learning program , and the learners' needs

He indicates that those m aterials that cannot cover the related interesting topics are in the risk o f losing their users' attention Therefore, we should consider the needs o f the new generation in this technology-oriented era and prepare the writing section that can help students im prove their writing after the course

4.2 C ontents o f the material

Table 2a M eans, standard deviations, frequencies, and percentages o f the question

on Design and O rganization for students.

Note: Strongly disagree = 1, Disagree = 2 , N eutral =3 A gree = 4 Strongly agree = 5

T able 2b M eans, standard deviations, frequencies, and percentages o f the question

on Design and O rganization for teachers.

A s shown in Table 2a 52% o f student said that the illustrations in the book helped them understand the lessons easily (M = 3.43, SD = 807) m eanwhile 80% o f the teachers offered neutral ideas and 20% o f the teacher agreed, only 13 students (13% ) disagreed

A lthough many com m ents on the illustrations o f the book help them to easily understand the lesson content, not m any students (43% ) said that those illustrations stim ulated their creativity, while 22% disagreed with Question No.7 (M = 3.20, SD = 910)

W ith Question 8 that the review helped reinforce what they learned in the previous lessons, nearly h alf o f the students (48% ) agreed or strongly agreed, only 19% o f the students disagreed with this statem ent (M = 3.36, SD = 905) This was also agreed by

m ost teachers (60%)

For the last question o f this section involving whether the textbook had enough exercises for the students to practise, more than h alf o f the students and teachers (60% ) agreed that the book provided sufficient exercises for them to practise, only 18% o f students disagreed (M = 3.58 SD = 1.056).

As the results show, the average range o f the item "Design and O rganization" is from 3.20 (low est) to 3.58 (highest) C onsidering the range between “totally disagree" and

"totally agree", the average range obtained for the questions relating to "Design and

O rganization " o f the textbook is associated with the agreed scale Therefore, it seems that the Design and Organization o f the textbook can be effective as expected and some changes may be needed to help stim ulate the students' creativity

The results from the questionnaires reveal that for the Design and Organization,

m ost students were not satisfied with the illustrations in each unit as the pictures were prim itive, not colorful thus it did not m otivate the learners to understand the contents o f the lessons through the illustration However, most students agreed that the illustrations stim ulated them to be creative Not many students agreed that the review sections helped them consolidate what had been learnt in previous sections and there w ere also not many students w ho agreed that there were enough exercises for them to practise

T able 3a M eans, standard deviations, frequencies, and percentages o f the question

on C ontents for students.

Note: Strongly disagree = 1, Disagree = 2, N eutral = 3, A gree = 4, Strongly agree = 5

T able 3b M eans, standard deviations, frequencies, and percentages o f the question

on C ontents for teachers.

Note: Strongly disagree = 1, Disagree = 2, Neutral = 3, A gree = 4, Strongly agree = 5

As shown in table 3a, 40 students (40% ) agreed that there were too m any new words and technical term s presented in each unit, 33% expressed neutral ideas and 27% disagreed with the above statem ent (M = 3.19, SD = 1.032)

For Q uestion 11, nearly h a lf (48% ) o f the students and 40% o f the teachers agreed that it w as easy to understand new w ords and technical term s in the context while nearly a fifth o f the students (19% ) w ere opposed to this idea Approxim ately a third o f them (33% ) expressed neutral idea about this (M = 3.33, SD = 965)

W hen questioned about w hether it was easy to understand new w ords and technical term s through draw ings or pictures (Q uestion 12), m ost students (51% ) agreed with this

w hereas only 13% o f those questioned disagreed with it (M = 3.43, SD = .782) Nearly

h a lf o f the teachers (40% ) also revealed that the presentation o f new words and expressions in the context and draw ings are suitable w ith the level o f the students

Question 13 would like the respondents to agree with the fact that students can practise effectively new w ords and technical term s through tasks/exercises and over half

o f the students (61% ) were consent with this idea O nly a few o f them disagreed with this (13% ) and only 25% expressed neutral idea about this (M = 3.58, SD = 912)

Just equally over h a lf o f the students (56% ) fixed that the gram m atical structures were presented clearly and easily enough for them to understand and 60% o f the teachers also agreed with this statement O nly 11% o f the students disagreed with this fact A third

o f them (33% ) revealed neutral idea to this (M = 3.57, SD = 879)

In further explanation to this, gram m ar was presented separately in Language focus section in each unit, and students could practice through sam ple and the exercises, structures were also introduced here The students were given some exam ples o f language focus and then they read, find and underline the w ord, fill in the gap with correct w ords or using correct form o f verb in brackets, fill in the gaps using the w ords to com plete the