Evaluation of the effectiveness of online learning program as part of the english course to second year non english major students at hanoi university of industry

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIESFACULTY OF POST-GRADUATE STUDIES ---  ---BÙI THỊ THU GIANG EVALUATION OF THE EFFECTIVENESS OF ONLIN

VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES

FACULTY OF POST-GRADUATE STUDIES

- 

-BÙI THỊ THU GIANG

EVALUATION OF THE EFFECTIVENESS OF ONLINE LEARNING PROGRAM

AS PART OF THE ENGLISH COURSE TO SECOND-YEAR NON-ENGLISH

MAJOR STUDENTS AT HANOI UNIVERSITY OF INDUSTRY

(Đánh giá hiệu quả của chương trình tiếng Anh trực tuyến như một phần của chương trình tiếng Anh đối với sinh viên không chuyên năm thứ hai

trường Đại học Công nghiệp Hà Nội)

M.A Minor Programme Thesis THESIS SUMMARY

Field: English Language Teaching Methodology

Code: 60 14 10

Supervisor: Khoa Anh Việt, M.A.

HA NOI – 2012

TABLE OF CONTENTS

Declaration i

Acknowledgements ii

Abstract iii

Table of contents iv

List of abbreviations vi

List of figures, tables and charts vii

PART A: INTRODUCTION 1

1 Rationale 1

2 Aims and objectives of the study 2

3 Scope of the study 3

4 Methods of the study 3

5 Design of the study 3

PART B: DEVELOPMENT 5

CHAPTER 1: LITERATURE REVIEW 5

1.1 Online learning 5

1.1.1 Definitions of online learning 5

1.1.2 Reasons for online learning 6

1.2.3 Challenges in online learning 7

1.2 Evaluation of online learning program 8

1.2.1 The importance of evaluation 8

1.2.2 Types of evaluation 10

1.2.3 Evaluation of online learning program and the conceptual framework 10 1.3 Online learning at Hanoi University of Industry 14

CHAPTER 2: METHODOLOGY 15

2.1 The setting of the research 15

2.2 Overview of the syllabus with the adoption of Online Learning Program 15

2.3 Subjects 17

2.3.1 Questionnaire respondents 17

2.3.2 Interview informants 17

2.4 Data collection instruments 18

2.4.1 Questionnaire 18

2.4.2 Interview 18

2.4.3 Pre-test, post-test and students‟ online learning reports 18

2.5 Data analysis procedures 19

CHAPTER 3: DATA ANALYSIS AND DISCUSSION 20

3.1 Data analysis 20

3.1.1 Questionnaire 20

3.1.2 Interview 29

3.1.3 Pre-test, post-test and online learning reports 31

3.2 Major findings 36

3.2.1 Students‟ and teachers‟ opinions and attitudes toward the OLP 36

3.2.2 The effectiveness of the OLP on students‟ English learning 37

3.2.3 Some suggestions to maximize the effectiveness of the OLP 37

PART C: CONCLUSION 39

1 Summary of the study 39

2 Limitations and suggestions for further study 40

REFERENCES 41 APPENDIX 1 (Student questionnaire – English version) I

APPENDIX 2 (Student questionnaire – Vietnamese version) IV

APPENDIX 3 (Teacher interview) VII

APPENDIX 4 (Pre-test) VIII

APPENDIX 5 (Post-test) XIII

LIST OF ABBREVIATIONS

HaUI: Hanoi University of Industry

FTTT: Faculty of Teacher Training and Tourism

FME: Faculty of Mechanical Engineering

OLP: Online Learning Program

ID: Instructional Development

WBT: Web Based Training

ICT: Information Communication Technology

QTDLKS1: Vietnamese abbreviation for Hotel and Tourism Administration Class 1

CĐT2: Vietnamese abbreviation for Electromechanical Engineering Class 2CK1: Vietnamese abbreviation for Mechanical Engineering Class 1

LIST OF FIGURES, TABLES AND CHARTS

Figure 1: Evaluation as central to practice, from Gustafson & Branch (2002)Figure 2: Conceptual framework applied to the analysis of PBJ tutorial evaluationTable 1: Students‟ opinions on the three aspects of the OLP

Chart 1: The importance of learning English

Chart 2: The importance of Online Learning Program

Chart 3: The benefits of the OLP to students

Chart 4: The results of pre-test of CĐT2

Chart 5: The results of pre-test of CK1

Chart 6: The results of pre-test of QTDLKS1

Chart 7: The results of post-test of CĐT2

Chart 8: The results of post-test of CK1

Chart 9: The results of post-test of QTDLKS1

Chart 10: The results of pre-test and post-test of CĐT2

Chart 11: The results of pre-test and post-test of CK1

Chart 12: The results of pre-test and post-test of QTDLKS1

Chart 13: The results of pre-test and post-test

PART A: INTRODUCTION

1 Rationale

As the era of information technology has come with significant contribution

to education, especially to teaching and learning foreign languages, computers havebecome one of the most dominant devices in the development and delivery ofaudio-visual products, multimedia presentations, visual materials and end-usersoftware Opportunities such as Internet access, distance learning capabilities, andapplications software are tools of the new millennium and they are often used tomake the educational environment more relevant, rich and rewarding (Ennis-Cole &Lawhon, 2004) Belz (2003) also affirms that Internet communication tools are used

to promote the development of not only foreign language linguistic competence butalso intercultural competence Riley (2000, as cited in Yilmaz & Orhan, 2010)emphasizes that teaching and learning that use technology effectively can lead to agreater academic achievement and make a real difference in the lives of students.Richards (2006) stresses on the increasing support for learning in the future with theuse of information and communication technologies – especially the onlineeducational possibilities of the Internet resources and communications Onlinelearning, as an incredible application of information technology, has thereforereceived rave reviews from a large number of educators all over the world As theresult, the number of online educators with many online learning websites isincreasing day by day and online learning plays an increasingly important role inteaching and learning the most popular language in the world – English At HanoiUniversity of Industry (HaUI), English is taught for a wide range of students ofmore than 10 faculties and departments This fact results in the demand ofinnovation in teaching and learning style of which online learning is a choice.However, because this is a very new way to be applied, the challenges as well asshortcomings are unavoidable On carrying out the study, the researcher hopes that

it can be a big help in deciding whether or not or in what way to teach and learnEnglish more effectively with the adaptation of online learning

Another important reason for the study is the decreasing excitement instudents‟ learning When online learning program was introduced into the course,most students seemed interested and hard working with the subject They generallyfinished the lessons assigned on time in the very first weeks However, about someweeks later, they seemed to neglect their study both online and in class Many ofstudents submitted the lessons late and blamed mainly for the technical mistakes.Their study in class became a bit less exciting than that of the first weeks Each timestudents were asked about online lessons, many of them did not remember anythingabout the content and how they finished the lessons Basing on these problems, theresearcher decided to carry out a study on the effectiveness of Online LearningProgram (OLP) to students who took part in it.

2 Aims and objectives of the study

The study is designed to bring a new look to the OLP as integrated to themain English courses at Hanoi University of Industry In other words, this program

is part of the blended learning course in training English for non-major students.The study, therefore, aims at evaluating the effectiveness of the OLP to second-yearnon-major students In particular, it firstly aims at investigating students‟ andteachers‟ attitudes towards the OLP Secondly, it is intended to explore howeffective the OLP as part of the course is on students‟ learning through figuring outthe benefits and the learning results they had during and after its implementation.Providing some suggestions by students and teachers to maximize the educationaloutcomes of this program is the final purpose of this study

In order to achieve these aims, the study seeks to answer the followingresearch questions:

(1) What are students and teachers’ opinions and attitudes towards Online Learning Program?

Online Learning Program?

3 Scope of the study

The study is about the evaluation of the effectiveness of the OLP as part ofthe English course to second-year non-English major students In other words, thisprogram is part of the blended learning English course for students of Intake 5 atHaUI, except for students of Faculty of Foreign Languages The evaluation of theeffectiveness of this program focuses on evaluation of the users‟ reaction andlearning as the result of the program This concerns the suitability of theinstructional design, the friendliness to the users (students) and how supportiveadministrators and the teachers are to students‟ learning, basing on students andteachers‟ evaluation in survey questionnaires and structured interviews In order todecide the effects of the program on students‟ learning achievement, data from theironline learning reports, pre-test and post-test are collected for analysis Thesuggestions for improving the effectiveness of the program by the teachers andstudents are also addressed in the study

4 Methods of the study

The study employed both qualitative and quantitative research method Thedata were collected using survey questionnaires for students, structured interviewsfor teachers, students‟ online learning reports accompanied with pre-test and post-test In this study, survey questionnaires were used to collect the data that show thestudents' opinions and attitudes towards the OLP they have just finished Structuredinterviews were employed to teachers to get information on their opinions andattitudes to the OLP in general as well as its effectiveness to their students Lastly,students‟ online learning reports and results from pre-test and post-test werecollected to evaluate their learning progress after the introduction of the OLP

5 Design of the study

The study consists of 3 parts:

This part consists of rationale, aims, scope, method, and design of the study

- Part B: Development:

This part is the main content including 3 chapters:

+ Chapter 1: is the literature review which deals with the concepts of onlinelearning, evaluation of online learning program, and online learning at Hanoi University

of Industry

+ Chapter 2: is about the methodology of the study

+ Chapter 3: is about the data analysis, discussion and major findings

Part C: Conclusion:

This part presents the summary of the study, limitations and the suggestions for further study

PART B: DEVELOPMENTCHAPTER 1: LITERATURE REVIEW 1.1 Online learning

1.1.1 Definitions of online learning

Online learning, also known as e-learning, is a learning experience deliveredvia a computer and the Internet Course work is conducted through electronicforums, discussion groups, external resources, quizzes, social rooms, and onlinesubmitted assignments Learning activities also include pair work, small group workand project work and throughout the course learners have the guidance of onlinetutors According to Shank and Sitze (2004) online learning involves the use ofnetwork technologies (such as the Internet and business networks) for delivering,supporting, and assessing formal and informal instruction The Internet and othernew media technologies such as MP3s, PDAs and iPods are common tools todeliver the learning content Putzhuber (2003) mentions the terms „online learning‟which is closely connected to e-learning He defines it as “the delivery of learningand training using electronic media, computers, internet and intranet.” Another termwhich also shares the identical meaning with online learning is WBT (Web BasedTraining) As to him, e-learning can be used as the main method of delivery oftraining or as a combined approach with classroom-based training

This facility has been profoundly important and therefore used by a largenumber of institutions and organizations all over the world in a wide range of fields.According to Wang and Ip (2010), recent pervasive social, technological andeconomic changes have significantly influenced educational contexts, and thusrequire radical rethinking the delivery of higher education (Rigmor, et al 2000), ande-learning has currently been particularly attractive for educational purposes inrecent years because of its worldwide accessibility, capabilities, and interactivefunction (Li & Hart, 1996; Lin, 2006) The Sloan Consortium is a leading researcher

of online learning In a survey of more than 2,500 US universities and colleges(2007), they find that over 3.9 million students were taking at least one

online course during the fall 2007 term - a 12 percent increase over the numberreported the previous year They concluded: “online enrollments have continued togrow at rates far in excess of the total higher education student population, with themost recent data demonstrating no signs of slowing.” There has been a significantrise in the number of universities and colleges making online courses available inthe last few years 96 percent of the very largest institutions (more than 15,000 totalenrollments) have some online offerings, as outlined in the report „Making theGrade: Online Education in the United States‟ by the Sloan Consortium in 2006.

1.1.2 Reasons for online learning

Because of the increasing popularity over the world, online learning provesmany undeniable benefits These are the main reasons to use online learningprovided by Shank and Sitze (2004):

First of all, it can improve access and flexibility People can log ineverywhere, every time to complete a lesson or refer to learning materials

Secondly, the delivery speed is faster and it is cost-saving For organizationsthat need to convey targeted information that quickly becomes outdated, onlinemodules are almost always faster and cheaper than flying trainers across the countryand requiring learners to sit in a classroom for a set number of hours

Thirdly, control is improved and standardized Online learning presents acommon, consistent message to large groups of learners regardless of location

Last but not least, communication and collaboration can be enhanced thoughonline learning Certain software tools allow learners to communicate with eachother, collaborate on projects, and share documents without the need for travel

Educationally, online learning can make class discussion richer with moreideas from those who are not really confident in class The interaction with learningcontent, therefore, will be deeper Students can rewind over lectures, or researchpoints that the lecturer has made before moving on to their next point Differentlearning styles, visually and auditorily, can also be accommodated to satisfylearners‟ needs Because of more interaction among individuals, more friendships

can be built and the diversity of views can be boosted from them Other advantagesinclude reinforced learning, privacy and wealth of information (Kung, 2005).Paudyal (2006) affirms the strength of e-learning: “the physical presence isbecoming less significant and the earth is becoming a global village due to webtechnology and development in ICT (information communication technology).”

About the application of online learning, Moloudi (2011), in his report onOnline and Face-to-face Peer Review: Measures of Implementation in ESL WritingClasses, finds that both online and face-to-face peer reviews have positive effects onstudents‟ writing progress; however, when transferred to a computer lab,participating students turn to be more motivated and productive in online peerreview Trần Thanh Điện and Esichaikul‟s study (2007) aims at comparingstudents‟ learning achievements between traditional and online learningenvironment with the sample of thirty undergraduate students at Can Tho University

of Vietnam The results show a significant difference in learning outcomes betweentwo learning environments, preferable to online learning Kung (2005) in his studyalso figures out that students found the assignments they did on the websiteworthwhile, interesting and beneficial for learning English

1.2.3 Challenges in online learning

It is clearly seen that online learning plays a crucial part in education; it is,however, not to say that there are no challenges faced by teachers and students whoengaged in the course Current research has turned the attention from focusing onthe linguistic and affective characteristics of computer-assisted discussion in singleclassrooms to long-distance collaboration which is challenging to arrange, because

it involves diverse learners with different cultural backgrounds, communicativeexpectations, and rhetorical frameworks (Kern, Ware, & Warschauer, 2004) As forBelz (2002), institutional differences in computer access, academic calendars andaccreditation systems are also important factors for learners‟ participation levelsand interaction O‟Down and Eberbach (2004) point out that the roles teachers play

in organizing telecollaboration to promote greater learner autonomy or intercultural

competence has been generally neglected (as cited in Wang & Ip, 2010) Actually,the learning process of teachers, according to Richards (2005), to integrate ICTtools and programs effectively and confidently can be frustrating and often is notachieved without adequate support Wang and Ip (2010) also mentioned factorsaffecting online learning, i.e., regarding computer literacy, successful skills andstrategies for face-to-face learning will not be suffice for online learning becauselearners need a basic-medium level of ICT competence (Hara & Kling, 1999).What‟s more, in order to have good outcomes, students should have socialinteraction, computer literacy, computer self-efficacy, and positive online learnerqualities Otherwise, their learning will be poor with predetermined pace oflearning, poorly functioning learner experiences, dissatisfaction, and negative onlinelearner qualities They also emphasized that some of the aforementioned factors arerelated to student skills and attitudes while others are related to educational designwhich teachers should take good care of (Pillay, Irving & Tones, 2007) FacultyFocus (2009) stressed two of the most common frustrations of online learning whichshould be minimized, i.e., confusing course organization (how course elements arestructured within the course) and unclear navigation (what links or buttons are used

to access these elements) Wang and Ip (2010) in their study about tasks andchallenges faced by teachers in handling on online project in which the participantsare freshmen students from Macau and Taiwan (15 each) also find some challengessuch as the different academic calendars of the two colleges, technical difficulties,motivating students, and the most outstanding is the constraints on the misalignment

of academic calendars and the differences in student contact hours

1.2 Evaluation of online learning program

1.2.1 The importance of evaluation

According to Brill and Park (2011), evaluation “plays an essential role inalmost all instructional development (ID) models and training cycles.” In thefollowing figure (Gustafson & Branch, 2002), „evaluate‟ is positioned at the core of

the model, emphasizing its centrality to the ID process Evaluation must therefore

be embedded as a fundamental component serving all other components andapproach it as an interactive part for any ID practice that is engaged in continuousimprovement

Figure 1: Evaluation as central to practice, from Gustafson & Branch (2002)

As an interactive process, evaluation helps educators conceive of instructionnot as an end point to ID but as part of a circle for ongoing improvement It thus can

be said that evaluation plays “the key role of quality control over the cycle.”(Bramley, 1996, as cited in Brill & Park, 2011)

Kirkpatrick (1998) identifies the reasons for evaluation:

 To justify the existence of the training department by showing how it

contributes to the organizations objectives and goals

 To decide whether to continue or discontinue training programs

 To gain information on how to improve future training programs

Fisher and Ruffino (1996) mention some consequences in case the evaluation is not carried out:

 A participant may continue to take courses in which they fail to learn

 Course modifications may not be based on participant outcome data andchanges may cause effective training activities to be replaced by ineffective activities.

 Training may be viewed as a wasteful use of resources

To sum up, evaluation is a necessary process and an important feedback tool toensure the development and improvement of a program

1.2.3 Evaluation of online learning program and the conceptual framework

Although the benefits as well as challenges in online learning are obvious, it

is not easy to evaluate an online course According to Son (2005) the online coursewhich is carried out on a website is evaluated in terms of its purpose, accuracy,currency, authority, loading speed, usefulness, organization, navigation, reliability,authenticity, interactivity, feedback, multimedia, communication and integration.Trần Thanh Điện and Esichaikul (2007) in their case study about evaluating onlinelearning environment provide three models of evaluation: objectives-orientedevaluation, CIPP model, and Kirkpatrick‟s evaluation model Objectives-orientedevaluation is to compare learning results to online learning objectives From thisapproach, the objectives can be determined whether they are met and appropriate ornot CIPP model stands for evaluations of context, inputs, processes, and products

It is considered a comprehensive framework using both formative and summative

evaluations Kirkpatrick‟s evaluation model was chosen for their study This is afour-level model developed by Kirkpatrick (1998) which is the most well-knownand widely used for classifying areas of evaluation These four levels are themeasurement to assess reaction, learning, behavior, and results In his study, Thair(2008) also mentions Kirkpatrick‟s model as the basis for his adaptive evaluationmodel On the reaction level, it is essential to find out whether the participants werepleased with the program and to assess the correlates of this pleasure In otherwords, it is to investigate what the participants thought about the program, includingmaterials, instructors, facilities, methodology and content This can be done throughreaction questionnaire The purpose of learning level is to examine the content ofthe material learned It is concerned with measuring the knowledge principles, facts,techniques and skills presented in a program, i.e., to see how they understood andabsorbed the material On behavior level, it is the job performance that is to bemeasured to examine how affective the program is on participants‟ performance intheir jobs The evaluation at result level, lastly, aims at organizational improvement.

At this level, some of the results can be examined are cost savings, work output,improvement and quality changes

Although the Kirkpatrick‟s framework provides a useful basis for evaluationplan design, Aldito et al (2006) state that an evaluation methodology specificallyfor e-learning applications is necessary but not available They emphasize usability,accessibility and didactic effectiveness as important components of e-learningevaluation Horton (2001) recommends that the criteria for evaluating e-learninginclude: business issues, technical issues, content, instructional design, practice andfeedback, usability, media, navigation and control, and motivation Lam andMcNaught (2007) suggest that both technical and pedagogical elements beconsidered in evaluation Other scholars also suggest their own evaluation modelsharing similar characteristics with Kirkpatrick‟s to some extent Thair (2008)mentions Treadway Parker‟s model with four types of evaluation: job performance,group performance, participant satisfaction and participant knowledge gained

According to Parker, most evaluation studies focus on the last two categories.Jackson and Kulp (1979, as cited in Thair, 2008) develop a slightly different modelwith four levels of program results: reaction outcomes, capability outcomes,application outcomes and worth outcomes among which „reaction outcomes‟presents the participants‟ opinions of the program as a whole or as specificcomponents such as program content, materials, methods or activities, „capabilityoutcomes‟ covers what participants are expected to know, think, do or produce bythe end of the program, „application outcomes‟ refers to what the participantsknow, think, do or produce in the real-world settings, and „worth outcomes‟represents the extent to which an organization benefits from training in terms ofmoney, time, effort, or resources invested.

In his study, Thair (2008) proposes an adaptation of Kirkpatrick‟s modelwhich can be applied to e-learning environments The proposed model containsthree areas:

The learning phase measures the actual learning as the direct result of the course It is to assess whether the learner has learned the information or acquired theskills necessary to apply to the pertinent area

e-The results phase examines the benefit of the knowledge acquired, the abilityfor an employee to function effectively and efficiently after the training as well asthe overall intrinsic and extrinsic benefits for both the employee and employer

In the study on evaluating online tutorials for university faculty, staff, andstudents, Brill and Park (2011) propose a conceptual framework applied to theevaluation of those online tutorials as follows:

Figure 2: Conceptual framework applied to the analysis of PBJ tutorial evaluation

This conceptual framework shares the same categories as those proposed byThair (2008) including three levels of reaction (interaction), learning and transfer(results) Learning conditions, learning and learning application are the threedimensions of the evaluation In order to evaluate the evaluand‟s reaction, twocomponents of learning conditions are examined including instructional design anduser friendliness Seven categories in learning conditions play an essential role indetermining the extent the users are satisfied with the tutorials, how these affecttheir learning and to what extent they can apply what they learned to their work

For online-learning program in blended learning course at HaUI, Thair‟sadaptation of Kirkpatrick‟s model and the above conceptual framework applied byBrill and Park provide a strong basis for the evaluation of its effectiveness Because

of the nature and the scope of the study, only two levels of reaction (interaction) andlearning are chosen to evaluate

1.3 Online learning at Hanoi University of Industry

Faculty of Foreign Languages is a new faculty at Hanoi University ofIndustry Seeing the fact that the growth of English is accelerating at very high rateand becoming more and more important to students‟ future careers, the Universityhas paid great attention to teaching and learning English New technological andeducational achievements are always priorities to be applied to have better teachingand learning outcomes Therefore, online learning received considerateconsideration from leaders of the Faculty to be adopted as part of two Englishcourses for non-English major students with the hope that their English knowledge

as well as skills will be highly improved This is the first time online learning hasbeen applied into teaching and learning English at Hanoi University of Industry incooperation with GK Group-an online learning provider

CHAPTER 2: METHODOLOGY

2.1 The setting of the research

The research is conducted at the Faculty of Teacher Training and Tourism(FTTT) and Faculty of Mechanical Engineering (FME) at HaUI Students in theFTTT are said to be generally better at English in comparison with those from otherdepartments in the university Many of them have a wider range of vocabulary andacquire better knowledge of grammar while most students in the FME are of lowerlevels Before they take online learning as part of their English course, they areassumed to be at elementary level as they have finished the two first terms learningEnglish with the course book New Headway Elementary- third edition In the thirdterm, they once again work with New Headway – third edition but at pre-intermediate level This is an integrated course book; therefore, the purposes of thiscourse are enhancing students‟ knowledge of grammar, vocabulary, pronunciationand improving four skills of speaking, listening, reading and writing

2.2 Overview of the syllabus with the adoption of Online Learning Program

After students finished their first English course in the first term, two nextcourses were chosen to be learned in accordance with online program: they wereGeneral English 2 and General English 3 of which New Headway pre-intermediatewas the course book The book was divided into two parts The first part consisted

of 7 first units and the second included 7 last units Each part of the book was learntduring one term This means that the content of the book was learnt during twoterms when students finished their first year (General English 2) and began theirsecond year (General English 3) There were, therefore, two parts of online learningprogram integrated to two courses For the General English 2, the online programhad 7 units in consistence with 7 first units to be learnt during the term 7 last unitsfor General English 3 were learnt in accordance with 7 units of online programwhich is under investigation The names of units in the course book and those inonline program are exactly the same and the content of each unit in the program isrelated to the names of units Each online unit has three to five lessons of which two

are compulsory and the others are optional Each lesson is designed with activities

of different types (multiple choices, True/False, error correction…) to improvestudents‟ grammar, vocabulary and three skills: reading, listening and writing Theprogram does not support speaking skill in forms of direct conversations ordiscussions Some activities of indirect forms and pronunciation activities arereplaced instead

Each teacher in charge of teaching these courses and each student were givenaccounts to get access to the website: http://www.haui.vietnamlearning.vn Whenentering the website, after confirming registration information and someinstallations, students could start learning each unit Each time students learned onthis website was recorded on another website to be only used by the teachers whotaught them Because this program focused on individual‟s learning achievements,there was little interaction among learners Students finished the activities in eachlesson and their answers were recorded to be automatically marked by the program.Students then could know whether they had right or wrong answers and correctthem until they are right Each time students finished the activities, they knew theresults

Each teacher participated in these courses had 5 periods (equivalent to 225minutes) to help students in their learning They could answer questions related tothe lessons or knowledge related to the course Teachers and students could arrangetime themselves provided that it is at their convenience Other technical troubleswould be solved by GK Group via emails or on telephone Besides having an onlineaccount, teachers also had a common account to another website:http://lms.haui.vietnamlearning.vn tocheck their students‟ learning progress Everyweek, students were assigned a lesson of each unit which was parallel with the unitlearned in class One week later, teacher exported the report telling their students‟learning process From this report, teacher could know which students had notfinished their lessons Because this is an integrated part to the main course, the mark

students got from it was just one of two progress marks among four marks neededfor the course.

As for the syllabus, the blended learning course was carried out in 90 periods(65 of which were carried out face-to-face in class and 25 others were done online

by students) within 15 weeks of the term Every week, there was one face-to-facemeeting of four periods (each one lasted for 45 minutes) in class which involvedmainly with traditional teaching method Every two weeks, one unit was taught.Therefore, 7 last units in New Headway - pre-intermediate and 7 units of OnlineLearning Program were evenly distributed to 15 weeks plus three tests

2.3 Subjects

2.3.1 Questionnaire respondents

The respondent group for survey questionnaire includes 165 second-yearstudents among which are 55 students from class QTDLKS1, FTTT, 55 studentsfrom class CĐT2, and 55 others from class CK1, all of them come from FME, HaUIwith the age ranging from 18 to 22 They come from different backgrounds andenvironments, varied from cities to mountainous areas of Vietnam The majority ofstudents have learnt English for about 7 years, but a few of them started learning itwhen they attended the university Before learning General English 3 with onlinelearning program, they had finished the first term with General English 1 atelementary level and second term with blended learning course at pre-intermediatelevel

2.3.2 Interview informants

10 teachers of English at Faculty of Foreign Languages are chosen tointerview Their age ranges from 25 to 35 with at least 2 years of teachingexperience at HaUI 5 of them have MA degrees, 3 are taking part in post-graduateprogram and 2 have BA degrees Most of them have very good command of Englishand are experienced in teaching English to students

2.4 Data collection instruments

2.4.1 Questionnaire

The purpose of the questionnaire is to explore students‟ attitudes andopinions towards online learning as part of the course, the benefits students get fromonline learning, the challenges they come up with during the course, and theirsuggestions for better use of online learning in their learning of English The surveyquestionnaire was translated into Vietnamese to help them understand the questionsclearly and give correct answers

2.4.2 Interview

Structured interviews were carried out to 10 teachers so as to investigate theiropinions, attitudes and the suggestions for maximum use of online learning Theinterviews were carried out when it was convenient, either at Faculty of ForeignLanguages or during the break time between two-period classes in the staff room.The interview lasted from 10-15 minutes for each teacher

2.4.3 Pre-test, post-test and students’ online learning reports

implementation of the OLP in the term, two tests were designed The pre-test was taken

at the beginning of the course (in the second week of the term), and the post-test wascarried out by the end of the course (in the 14th week of the term) The content of the twotests were mainly taken from the online materials with familiar types of exercises tostudents These tests include grammar and vocabulary, reading, listening, writing andspeaking and were carried out offline (in class)

online learning reports were exported by their teachers from the website for learningmanagement The data in each weekly report show students‟ completion of the lesson(%), the score they got (on the scale of 100) and the time they spent learning that lesson

At the end of the term, the final report was exported to show students‟ completion of alllessons (%), the score they

got for the whole term ( also on the scale of 100), the whole amount of timethey spent on learning all the lessons and the total entries (times they gotaccess to the website to learn) These results accompanied with those of pre-test and post-test are the basis to evaluate students‟ online learning.

2.5 Data analysis procedures

To analyze data, both quantitative and qualitative methods are employed.Information collected from questionnaires will be quantified and displayed in chartsand tables In order to figure out the common themes, data from interviews will becoded Data from online reports, pre-test and post-test will be assessed to evaluatestudents‟ learning

CHAPTER 3: DATA ANALYSIS AND DISCUSSION 3.1 Data analysis

3.1.1 Questionnaire

The questionnaire aims at investigating the students‟ opinions and attitudes

to OLP 19 questions were designed to uncover this purpose in which the first

question and the second question aiming at examining the importance of learning

English and the OLP The 14 following items were to discover students‟ opinions

and attitudes toward the program in terms of three aspects of instructional design,

user-friendliness and support In order to find out the difficulties they have during

their learning related to these items, question 17 was addressed Question 18

searched for the benefits of the program to the students and their suggestions to

improve its effectiveness were supposed to reveal in the last question Among 165

copies of the questionnaire delivered, the number of copies returned was 160 and

they were all valid

3.1.1.1 The importance of learning English

The first question was designed to examine how important learning English

is to the respondents As shown in the results, all of the students taking part in the

survey highly appreciate the importance of English and the necessity to learn this

popular language 76% agreed that learning English is strongly important and 24%

thought that it is very important No one said that learning English is so so, not very

important or strongly unimportant

Chart 1: The importance of learning English

0 0

very important

so so not very important

76 strongly unimportant

3.1.1.2 Students‟ opinions and attitudes toward the Online Learning

Program

To seek for students‟ attitude to the OLP implemented at HaUI, the second

question addressed the importance of this program No one chose strongly

unimportant; therefore, the answers varied from strongly important to not very

important 9% found that the OLP was strongly important, 49% said that it was very

important while only 3% agreed that it was not very important 38% thought that it

was so so, which means the importance of the OLP was neutral to them Among the

respondents, those from CĐT2 were in the first position in their evaluation of the

importance of the OLP with 66% agreed that it was strongly important and very

important while those from QTDLKS1 and CK1 came with lower percentage of

56% and 54% respectively It can be said that to the respondents of CĐT2, they

showed more appreciation to the necessity for OLP to be introduced into the course

The following chart shows the details:

Chart 2: The importance of Online Learning Program

strongly important 38

very important

so so not very important

In order to go deeper into the evaluation of OLP, the 14 following items were

designed concerning three aspects: instructional design, user-friendliness and

support In instructional design, students were expected to show their opinions

about the clarity and suitability of the objectives of the program, the relevance of

topics and content, the opportunity to practice and the appropriateness of the

exercises to their level To examine how friendly OLP was to users, students wereasked about the interface, the design of the website, the accessibility and navigation,the instructions, the graphics and multimedia, the texts and the knowledge presented

in the website The aspect of support was to find out information about the time forlearning and the support from the provider as well as teachers to students‟ learning

Table 1: Students’ opinions on the three aspects of the OLP

As can be seen in the table, the opinions toward three aspects were quitepositive The majority of the respondents showed that they strongly agreed oragreed with items provided For instructional design, the percentage ranks from 71-77% strongly agree and agree, 4-24% undecided, 7-22% disagree and stronglydisagree For user-friendliness, the percentage of strongly agree and agree accountsfor 73-81%, undecided 9-18%, disagree and strongly disagree 7-21% Thepercentage for support are 76-82%, 6-12%, and 6-17% for strongly agree and agree,undecided, disagree and strongly disagree respectively So it can be said that thosewho showed their agreement outnumbered those with their disagreement

In instructional design, item 3 was designed to seek for the information aboutthe objectives of the OLP 28% strongly agreed and 43% agreed that the objectiveswere clear and met their goals in learning while only 5% disagreed with it and 24%remained undecided Regarding the topics and the content of units in the website,77% strongly agreed and agreed that they were relevant to what they wanted tolearn That means the topics and the content met their needs However, there was20% who disagreed and strongly disagreed indicating that the topics and the contentmight not be suitable to their needs or unfamiliar with them Students were alsoasked about the opportunity to practice in each lesson they learned in the website.For this item, 29% strongly agreed and 46% agreed that practice part was available

to them in each lesson while 14% disagreed and 4% strongly agreed with it.Concerning the exercises they had to do in each lesson, 75% strongly agreed andagreed that they were appropriate for their level This, however, is not true to 12%

of them

To decide the level of user-friendliness of the OLP, item 7 was designed toask students about the interface of the website 81% strongly agreed and agreed thatthe interface was user-friendly while 7% disagreed with user-friendly characteristic

of the website With regard to the artistic feature of the website, 15% stronglyagreed and 49% agreed that it was designed in an eye-catching way The percentage

of those who disagreed and strongly disagreed with this item was 18% Below aresome pictures that show the interface and part of the website‟s design

The following item was designed to examine the accessibility of the website.69% strongly agreed and agreed that the website was easy to access while 21%disagreed and strongly agreed This shows that these students had difficulty inaccessing the website to learn When learning on the website, the navigation andcontrol are very important, so it is another item to be examined 73% stronglyagreed and agreed that it was easy to move around without getting lost 17%showed their disagreement with it, which means they had troubles with navigationand control during their learning time The next item was to figure out the clarity ofthe instructions for tasks in each lesson 23% strongly agreed and 56% agreed thatthe instructions were clear and easy to understand This is very important to students

as it affects the way they understand and do the exercises Item 12 was to ask aboutthe help of media in their learning 73% strongly agreed and agreed that thegraphics and multimedia were helpful to them The number of respondents whodisagreed or strongly disagreed with the helpfulness of media used in the OLPaccounts for 12% Similarly, the percentage of students who disagreed that the textswere written in an understandable level was 10% while that of respondents withtheir agreement was 75% This indicates that the texts used in the OLP were

suitable for them to read and understand In order to find out information about theway the knowledge was presented in the website, item 14 was designed 77% saidthat they could understand the knowledge presented in each lesson In contrast, 11%disagreed that the knowledge was presented in an understandable way implying thatthese students had problems understanding the knowledge because of itspresentation.

This is the first time an online learning program has been introduced into thesyllabus, so the support for students plays an important role Item 15 was to askstudents‟ opinions about learning time and item 16 asked students about the supportthey received from the service provider and their teachers during their learningprocess A large number of students highly evaluated the flexibility of learning time(82%); only 6% disagreed that the time was inflexible During the term, the supportthey received from the provider and their teachers was agreed to be always available(76% strongly agreed and agreed) while 17% of them disagreed with that

The purpose of question 17 was to uncover the difficulties students had withitems 3-16 during their learning process Most of students‟ responses focused onproblems with items 4, 5, 8, 9, 10, and 16 Item 4 addressed the relevance of topicsand content to what students want to learn For this item, many respondents said thatsome topics and content were not familiar with students Sometimes, there weremany topics in one unit leading to confusion in their learning Item 5 mentioned theopportunity to practice in each lesson For this item, students showed theirexpectation to have more tasks to practice 4 skills instead of many grammarexercises They said that although the number of exercises was large, they did nothave many chances to practice their English skills because most of them weregrammar exercises and they were repeated causing boredom to students Item 8 wasabout the design of the website 18% thought that it was not designed in an eye-catching way They said that the design is too simple, not beautiful as other onlinelearning websites Concerning the accessibility, the reasons for 21% and 17% ofdisagreement for items 8 and 9 respectively were the difficulties to get access to the

website and the problems occurred frequently while they were learning ornavigating in the website Most of these were stated to be technical problems Initem 16, the support from the provider and teachers was mentioned 17% ofrespondents did not agree that the support was always available to them They didnot mention the teachers‟ but the provider‟s They said that they hardly receivedany support from the provider when they had technical problems It can be impliedthat students really needed support from the provider to solve some technicalproblems while they were learning They did not mention their teachers maybebecause they usually received the support from them for content-related matters.

Although the problems during learning process are unavoidable, students cannot deny the benefits the OLP brought to them as shown in their answers forquestion 18

Chart 3: The benefits of the OLP to students

stated they improved their self-study ability This is very important as enhancingstudents‟ autonomy is one of the main purposes of the program One more benefit

of the OLP is that it made 61% respondents more interested in English and 36%more motivated in class though this number was not quite high They alsocommented that the OLP was generally good, useful, convenient and flexible intime

In order to improve the effectiveness of the program, question 19 wasdesigned Number one in the list of suggestions from students was the necessity touse more audio-visual aids such as pictures, audios, video clips to make the lessonsmore interesting Another suggestion related to entertaining matter or students‟motivation They need more songs, funny stories or game shows to be introduced sothat they can relax after learning The topics were suggested to be more familiarwith their daily lives so that they could understand them better and do exercisesmore effectively They also expected that there should be more exercises onpronunciation and the answers for speaking part should be recorded and marked.For vocabulary and the answers for other parts such as reading, grammar, they needmore detailed explanation so that they can have deeper understanding what they arelearning Relating the answers for each exercise, they suggested the answers for thefirst time be saved or there should be no checking with available answer keys rightfrom the beginning on the website They thought that it would be better if they gotthe keys in another way, from their teachers for example They also said that theprovider should pay more attention to the keys as some of them are not correctresulting in demotivation to learners Because this program focused on students‟autonomy, it was suggested that there should be stricter control on their learning toavoid their compulsory and negative attitude to the OLP The last suggestionconcerned with the support and technical problems They thought that it would benecessary to have the help from the provider when they have troubles whilelearning They need high quality website with as few technical problems aspossible Because many of them did not use Internet Explorer in their computers,

they hoped that the program could be run on Mozilla Firefox or Google Chrome fortheir convenience.

3.1.2 Interview

The structured interview aims at investigating teachers‟ attitude and opinionsabout the OLP In particular, they were asked about the online materials, the level ofdifficulty of these exercises to their students, the frequency of giving reports andsupport, the benefits of the program, the difficulties they faced during the time theprogram was implemented and their suggestions to improve its effectiveness

Their answers to the first question showed that they appreciated the OLPwith 80% of agreement on the importance of this program though no one thoughtthat it is strongly important Similarly, for the second question, 8 out of ten teacherssaid that it is good, accounting for 80%, the other 2 disagreed with the thought that

it is not good This reveals that they had a positive attitude to the OLP in general Interms of online materials, almost all of them (90%) found that they are good with avariety of well-designed exercises and suitable to the syllabus However, whenbeing asked about the level of difficulty of these exercises to their students, one ofthem said that these exercises are difficult to their students Another said that thegrammar exercises and reading tasks are easy while some listening tasks are easyand some are difficult For speaking and writing tasks, one teacher stated that theyare difficult to weak students only; however, there are not many weak students inher class, so in general, the level of difficulty is medium The rest agreed that theseexercises are suitable to their students‟ level Therefore, 80% of teachersinterviewed found the OLP suitable to their students with regard to the level ofdifficulty of exercises Concerning the management of online reports and support,100% said they frequently gave reports (one or every two weeks) to promote andremind students about their learning, and provided help whenever their studentsneed via emails or in class However, they reported that most of the students‟questions related to technical problems rather than the content of each lesson.Sometimes, they also asked for explanation about the answers which they found

unsatisfactory, which exposed teachers to an awkward situation These teachersmentioned that they themselves found some answers incorrect, but when theycontacted with the provider, they rarely received a satisfactory answer Regardingthe benefits of the OLP, all teachers agreed that the program helped studentsconsolidate their knowledge, especially on grammar, and improve 4 skills Theyalso claimed the benefits of IT skills improvement and the flexibility of time Inspite of these benefits, they did not deny the challenges they faced with during theimplementation of the program All of them complained that it was really difficult

to manage students‟ work They did not know whether their students did theexercises themselves or had their friends do it for them They also stated thatbecause the reports were available for them to export, they did not know whetherthe scores were exact or not Sometimes, they found incorrect answers but havedifficulty explaining to their students as these answers were not given by them butthe provider Another challenge related to the support from the provider Whenstudents asked them some questions which were of the provider‟s responsibility,they hardly received the provider‟s feedbacks for those questions or justunsatisfactory answers These teachers also agreed on low web access They saidthat sometimes it took them a lot of time to export the reports or move around thewebsite

In order to make the program more effective, they firstly suggested a betterway to control students‟ work It could be changes in deadline for each task or theanswers should be saved for each time students gave another choice and theseshould be included in the reports for teachers to assess their students‟ learningprogress Another suggestion was improving web access and program quality Theyneed a website with least technical problems and higher quality program withreliable answer keys 8 out of 10 teachers had the idea that the program should bedesigned in a way that requires more interaction between teachers and students, oramong students They argued that it would be more interesting if teachers andstudents could work online so that they could explain to students when they had

problems and students could share experiences while they were learning They alsogave an alternative solution for this, i.e., wi-fi network should be available at schoolfor the convenience of checking students‟ work and explaining problematic issues.

3.1.3 Pre-test, post-test and online learning reports

3.1.3.1 Online learning reports

Examining weekly online reports of each class, it was found that studentsalmost finished their exercises on time, except for one or first two weeks The finalonline reports of three classes which showed the percentage of completion, scores,time, and entries were quite positive The percentage of completion was at least40% and 98% at most The scores were quite high with the average point of 80 andthe entries ranged from 12 to 70 No one had high percentage of completion andscores with a few entries

3.1.3.2 Pre-test and post-test

The two tests were designed basing on the content in the OLP and werecarried out offline (in class) with strict supervision to make sure students do the testthemselves and to ensure objective and exact scores

The following charts show the results of pre-test of each class under fourcategories: under average (0-4), average (5-6), fairly good (7-8), and good (9-10)

Chart 4: The results of pre-test of CĐT2

56 60

50

42 40

30 20 10

2

0

average