(LUẬN VĂN THẠC SĨ) Evaluation of the effectiveness of Online Learning Program as part of the English course to second year non English major students at Hanoi University of Industry

1 EVALUATION OF THE EFFECTIVENESS OF ONLINE LEARNING PROGRAM AS PART OF THE ENGLISH COURSE TO SECOND-YEAR NON-ENGLISH MAJOR STUDENTS AT HANOI UNIVERSITY OF INDUSTRY Đánh giá hiệu quả c

1

EVALUATION OF THE EFFECTIVENESS OF ONLINE LEARNING PROGRAM AS PART OF THE ENGLISH COURSE TO SECOND-YEAR NON-ENGLISH MAJOR

STUDENTS AT HANOI UNIVERSITY OF INDUSTRY

(Đánh giá hiệu quả của chương trình tiếng Anh trực tuyến như một phần của chương trình tiếng Anh đối với sinh viên không chuyên năm thứ hai

trường Đại học Công nghiệp Hà Nội)

M.A Minor Programme Thesis

THESIS SUMMARY

Field: English Language Teaching Methodology

Code: 60 14 10 Supervisor: Khoa Anh Việt, M.A

HA NOI – 2012

TIEU LUAN MOI download : skknchat@gmail.com

Table of contents iv

List of abbreviations vi

List of figures, tables and charts vii

PART A: INTRODUCTION 1

1 Rationale 1

2 Aims and objectives of the study 2

3 Scope of the study 3

4 Methods of the study 3

5 Design of the study 3

PART B: DEVELOPMENT 5

CHAPTER 1: LITERATURE REVIEW 5

1.1 Online learning 5

1.1.1 Definitions of online learning 5

1.1.2 Reasons for online learning 6

1.2.3 Challenges in online learning 7

1.2 Evaluation of online learning program 8

1.2.1 The importance of evaluation 8

1.2.2 Types of evaluation 10

1.2.3 Evaluation of online learning program and the conceptual framework 10

1.3 Online learning at Hanoi University of Industry 14

CHAPTER 2: METHODOLOGY 15

2.1 The setting of the research 15

2.2 Overview of the syllabus with the adoption of Online Learning Program 15

2.4.2 Interview 18

2.4.3 Pre-test, post-test and students‟ online learning reports 18

2.5 Data analysis procedures 19

CHAPTER 3: DATA ANALYSIS AND DISCUSSION 20

3.1 Data analysis 20

3.1.1 Questionnaire 20

3.1.2 Interview 29

3.1.3 Pre-test, post-test and online learning reports 31

3.2 Major findings 36

3.2.1 Students‟ and teachers‟ opinions and attitudes toward the OLP 36

3.2.2 The effectiveness of the OLP on students‟ English learning 37

3.2.3 Some suggestions to maximize the effectiveness of the OLP 37

PART C: CONCLUSION 39

1 Summary of the study 39

2 Limitations and suggestions for further study 40

REFERENCES 41 APPENDIX 1 (Student questionnaire – English version) I APPENDIX 2 (Student questionnaire – Vietnamese version) IV APPENDIX 3 (Teacher interview) VII APPENDIX 4 (Pre-test) VIII APPENDIX 5 (Post-test) XIII

OLP: Online Learning Program

ID: Instructional Development

WBT: Web Based Training

ICT: Information Communication Technology

QTDLKS1: Vietnamese abbreviation for Hotel and Tourism Administration Class 1 CĐT2: Vietnamese abbreviation for Electromechanical Engineering Class 2

CK1: Vietnamese abbreviation for Mechanical Engineering Class 1

LIST OF FIGURES, TABLES AND CHARTS

Figure 1: Evaluation as central to practice, from Gustafson & Branch (2002) Figure 2: Conceptual framework applied to the analysis of PBJ tutorial evaluation Table 1: Students‟ opinions on the three aspects of the OLP

Chart 1: The importance of learning English Chart 2: The importance of Online Learning Program Chart 3: The benefits of the OLP to students

Chart 4: The results of pre-test of CĐT2 Chart 5: The results of pre-test of CK1 Chart 6: The results of pre-test of QTDLKS1 Chart 7: The results of post-test of CĐT2 Chart 8: The results of post-test of CK1 Chart 9: The results of post-test of QTDLKS1 Chart 10: The results of pre-test and post-test of CĐT2 Chart 11: The results of pre-test and post-test of CK1 Chart 12: The results of pre-test and post-test of QTDLKS1 Chart 13: The results of pre-test and post-test

PART A: INTRODUCTION

1 Rationale

As the era of information technology has come with significant contribution

to education, especially to teaching and learning foreign languages, computers have become one of the most dominant devices in the development and delivery of audio-visual products, multimedia presentations, visual materials and end-user software Opportunities such as Internet access, distance learning capabilities, and applications software are tools of the new millennium and they are often used to make the educational environment more relevant, rich and rewarding (Ennis-Cole & Lawhon, 2004) Belz (2003) also affirms that Internet communication tools are used

to promote the development of not only foreign language linguistic competence but also intercultural competence Riley (2000, as cited in Yilmaz & Orhan, 2010) emphasizes that teaching and learning that use technology effectively can lead to a greater academic achievement and make a real difference in the lives of students Richards (2006) stresses on the increasing support for learning in the future with the use of information and communication technologies – especially the online educational possibilities of the Internet resources and communications Online learning, as an incredible application of information technology, has therefore received rave reviews from a large number of educators all over the world As the result, the number of online educators with many online learning websites is increasing day by day and online learning plays an increasingly important role in teaching and learning the most popular language in the world – English At Hanoi University of Industry (HaUI), English is taught for a wide range of students of more than 10 faculties and departments This fact results in the demand of innovation in teaching and learning style of which online learning is a choice However, because this is a very new way to be applied, the challenges as well as shortcomings are unavoidable On carrying out the study, the researcher hopes that

it can be a big help in deciding whether or not or in what way to teach and learn English more effectively with the adaptation of online learning

Another important reason for the study is the decreasing excitement in students‟ learning When online learning program was introduced into the course, most students seemed interested and hard working with the subject They generally finished the lessons assigned on time in the very first weeks However, about some weeks later, they seemed to neglect their study both online and in class Many of students submitted the lessons late and blamed mainly for the technical mistakes Their study in class became a bit less exciting than that of the first weeks Each time students were asked about online lessons, many of them did not remember anything about the content and how they finished the lessons Basing on these problems, the researcher decided to carry out a study on the effectiveness of Online Learning Program (OLP) to students who took part in it

2 Aims and objectives of the study

The study is designed to bring a new look to the OLP as integrated to the main English courses at Hanoi University of Industry In other words, this program

is part of the blended learning course in training English for non-major students The study, therefore, aims at evaluating the effectiveness of the OLP to second-year non-major students In particular, it firstly aims at investigating students‟ and teachers‟ attitudes towards the OLP Secondly, it is intended to explore how effective the OLP as part of the course is on students‟ learning through figuring out the benefits and the learning results they had during and after its implementation Providing some suggestions by students and teachers to maximize the educational outcomes of this program is the final purpose of this study

In order to achieve these aims, the study seeks to answer the following research questions:

(1) What are students and teachers’ opinions and attitudes towards Online Learning Program?

(2) How effective is Online Learning Program to students’ English learning?

(3) What are students’ and teachers’ suggestions to maximize the effectiveness of Online Learning Program?

3 Scope of the study

The study is about the evaluation of the effectiveness of the OLP as part of the English course to second-year non-English major students In other words, this program is part of the blended learning English course for students of Intake 5 at HaUI, except for students of Faculty of Foreign Languages The evaluation of the effectiveness of this program focuses on evaluation of the users‟ reaction and learning as the result of the program This concerns the suitability of the instructional design, the friendliness to the users (students) and how supportive administrators and the teachers are to students‟ learning, basing on students and teachers‟ evaluation in survey questionnaires and structured interviews In order to decide the effects of the program on students‟ learning achievement, data from their online learning reports, pre-test and post-test are collected for analysis The suggestions for improving the effectiveness of the program by the teachers and

students are also addressed in the study

4 Methods of the study

The study employed both qualitative and quantitative research method The data were collected using survey questionnaires for students, structured interviews for teachers, students‟ online learning reports accompanied with pre-test and post- test In this study, survey questionnaires were used to collect the data that show the students' opinions and attitudes towards the OLP they have just finished Structured interviews were employed to teachers to get information on their opinions and attitudes to the OLP in general as well as its effectiveness to their students Lastly, students‟ online learning reports and results from pre-test and post-test were collected to evaluate their learning progress after the introduction of the OLP

5 Design of the study

The study consists of 3 parts:

- Part A: Introduction:

This part consists of rationale, aims, scope, method, and design of the study

- Part B: Development:

This part is the main content including 3 chapters:

+ Chapter 1: is the literature review which deals with the concepts of online learning, evaluation of online learning program, and online learning at Hanoi University of Industry

+ Chapter 2: is about the methodology of the study + Chapter 3: is about the data analysis, discussion and major findings

Part C: Conclusion:

This part presents the summary of the study, limitations and the suggestions for further study

PART B: DEVELOPMENT CHAPTER 1: LITERATURE REVIEW 1.1 Online learning

1.1.1 Definitions of online learning

Online learning, also known as e-learning, is a learning experience delivered via a computer and the Internet Course work is conducted through electronic forums, discussion groups, external resources, quizzes, social rooms, and online submitted assignments Learning activities also include pair work, small group work and project work and throughout the course learners have the guidance of online tutors According to Shank and Sitze (2004) online learning involves the use of network technologies (such as the Internet and business networks) for delivering, supporting, and assessing formal and informal instruction The Internet and other new media technologies such as MP3s, PDAs and iPods are common tools to deliver the learning content Putzhuber (2003) mentions the terms „online learning‟ which is closely connected to e-learning He defines it as “the delivery of learning and training using electronic media, computers, internet and intranet.” Another term which also shares the identical meaning with online learning is WBT (Web Based Training) As to him, e-learning can be used as the main method of delivery of training or as a combined approach with classroom-based training

This facility has been profoundly important and therefore used by a large number of institutions and organizations all over the world in a wide range of fields According to Wang and Ip (2010), recent pervasive social, technological and economic changes have significantly influenced educational contexts, and thus require radical rethinking the delivery of higher education (Rigmor, et al 2000), and e-learning has currently been particularly attractive for educational purposes in recent years because of its worldwide accessibility, capabilities, and interactive function (Li & Hart, 1996; Lin, 2006) The Sloan Consortium is a leading researcher of online learning In a survey of more than 2,500 US universities and colleges (2007), they find that over 3.9 million students were taking at least one

online course during the fall 2007 term - a 12 percent increase over the number reported the previous year They concluded: “online enrollments have continued to grow at rates far in excess of the total higher education student population, with the most recent data demonstrating no signs of slowing.” There has been a significant rise in the number of universities and colleges making online courses available in the last few years 96 percent of the very largest institutions (more than 15,000 total enrollments) have some online offerings, as outlined in the report „Making the Grade: Online Education in the United States‟ by the Sloan Consortium in 2006

1.1.2 Reasons for online learning

Because of the increasing popularity over the world, online learning proves many undeniable benefits These are the main reasons to use online learning provided by Shank and Sitze (2004):

First of all, it can improve access and flexibility People can log in everywhere, every time to complete a lesson or refer to learning materials

Secondly, the delivery speed is faster and it is cost-saving For organizations that need to convey targeted information that quickly becomes outdated, online modules are almost always faster and cheaper than flying trainers across the country and requiring learners to sit in a classroom for a set number of hours

Thirdly, control is improved and standardized Online learning presents a common, consistent message to large groups of learners regardless of location

Last but not least, communication and collaboration can be enhanced though online learning Certain software tools allow learners to communicate with each other, collaborate on projects, and share documents without the need for travel

Educationally, online learning can make class discussion richer with more ideas from those who are not really confident in class The interaction with learning content, therefore, will be deeper Students can rewind over lectures, or research points that the lecturer has made before moving on to their next point Different learning styles, visually and auditorily, can also be accommodated to satisfy learners‟ needs Because of more interaction among individuals, more friendships

can be built and the diversity of views can be boosted from them Other advantages include reinforced learning, privacy and wealth of information (Kung, 2005) Paudyal (2006) affirms the strength of e-learning: “the physical presence is becoming less significant and the earth is becoming a global village due to web technology and development in ICT (information communication technology).”

About the application of online learning, Moloudi (2011), in his report on Online and Face-to-face Peer Review: Measures of Implementation in ESL Writing Classes, finds that both online and face-to-face peer reviews have positive effects on students‟ writing progress; however, when transferred to a computer lab, participating students turn to be more motivated and productive in online peer review Trần Thanh Điện and Esichaikul‟s study (2007) aims at comparing students‟ learning achievements between traditional and online learning environment with the sample of thirty undergraduate students at Can Tho University

of Vietnam The results show a significant difference in learning outcomes between two learning environments, preferable to online learning Kung (2005) in his study also figures out that students found the assignments they did on the website worthwhile, interesting and beneficial for learning English

1.2.3 Challenges in online learning

It is clearly seen that online learning plays a crucial part in education; it is, however, not to say that there are no challenges faced by teachers and students who engaged in the course Current research has turned the attention from focusing on the linguistic and affective characteristics of computer-assisted discussion in single classrooms to long-distance collaboration which is challenging to arrange, because

it involves diverse learners with different cultural backgrounds, communicative expectations, and rhetorical frameworks (Kern, Ware, & Warschauer, 2004) As for Belz (2002), institutional differences in computer access, academic calendars and accreditation systems are also important factors for learners‟ participation levels and interaction O‟Down and Eberbach (2004) point out that the roles teachers play in organizing telecollaboration to promote greater learner autonomy or intercultural

competence has been generally neglected (as cited in Wang & Ip, 2010) Actually, the learning process of teachers, according to Richards (2005), to integrate ICT tools and programs effectively and confidently can be frustrating and often is not achieved without adequate support Wang and Ip (2010) also mentioned factors affecting online learning, i.e., regarding computer literacy, successful skills and strategies for face-to-face learning will not be suffice for online learning because learners need a basic-medium level of ICT competence (Hara & Kling, 1999) What‟s more, in order to have good outcomes, students should have social interaction, computer literacy, computer self-efficacy, and positive online learner qualities Otherwise, their learning will be poor with predetermined pace of learning, poorly functioning learner experiences, dissatisfaction, and negative online learner qualities They also emphasized that some of the aforementioned factors are related to student skills and attitudes while others are related to educational design which teachers should take good care of (Pillay, Irving & Tones, 2007) Faculty Focus (2009) stressed two of the most common frustrations of online learning which should be minimized, i.e., confusing course organization (how course elements are structured within the course) and unclear navigation (what links or buttons are used to access these elements) Wang and Ip (2010) in their study about tasks and challenges faced by teachers in handling on online project in which the participants are freshmen students from Macau and Taiwan (15 each) also find some challenges such as the different academic calendars of the two colleges, technical difficulties, motivating students, and the most outstanding is the constraints on the misalignment of academic calendars and the differences in student contact hours

1.2 Evaluation of online learning program 1.2.1 The importance of evaluation

According to Brill and Park (2011), evaluation “plays an essential role in almost all instructional development (ID) models and training cycles.” In the following figure (Gustafson & Branch, 2002), „evaluate‟ is positioned at the core of

the model, emphasizing its centrality to the ID process Evaluation must therefore

be embedded as a fundamental component serving all other components and approach it as an interactive part for any ID practice that is engaged in continuous improvement

Figure 1: Evaluation as central to practice, from Gustafson & Branch (2002)

As an interactive process, evaluation helps educators conceive of instruction not as an end point to ID but as part of a circle for ongoing improvement It thus can

be said that evaluation plays “the key role of quality control over the cycle.” (Bramley, 1996, as cited in Brill & Park, 2011)

Kirkpatrick (1998) identifies the reasons for evaluation:

 To justify the existence of the training department by showing how it contributes to the organizations objectives and goals

 To decide whether to continue or discontinue training programs

 To gain information on how to improve future training programs

Fisher and Ruffino (1996) mention some consequences in case the evaluation is not carried out:

 A participant may continue to take courses in which they fail to learn

 Course modifications may not be based on participant outcome data and changes may cause effective training activities to be replaced by ineffective activities

 Training may be viewed as a wasteful use of resources

To sum up, evaluation is a necessary process and an important feedback tool to ensure the development and improvement of a program

1.2.3 Evaluation of online learning program and the conceptual framework

Although the benefits as well as challenges in online learning are obvious, it

is not easy to evaluate an online course According to Son (2005) the online course which is carried out on a website is evaluated in terms of its purpose, accuracy, currency, authority, loading speed, usefulness, organization, navigation, reliability, authenticity, interactivity, feedback, multimedia, communication and integration Trần Thanh Điện and Esichaikul (2007) in their case study about evaluating online learning environment provide three models of evaluation: objectives-oriented evaluation, CIPP model, and Kirkpatrick‟s evaluation model Objectives-oriented evaluation is to compare learning results to online learning objectives From this approach, the objectives can be determined whether they are met and appropriate or not CIPP model stands for evaluations of context, inputs, processes, and products

It is considered a comprehensive framework using both formative and summative

evaluations Kirkpatrick‟s evaluation model was chosen for their study This is a four-level model developed by Kirkpatrick (1998) which is the most well-known and widely used for classifying areas of evaluation These four levels are the measurement to assess reaction, learning, behavior, and results In his study, Thair (2008) also mentions Kirkpatrick‟s model as the basis for his adaptive evaluation model On the reaction level, it is essential to find out whether the participants were pleased with the program and to assess the correlates of this pleasure In other words, it is to investigate what the participants thought about the program, including materials, instructors, facilities, methodology and content This can be done through reaction questionnaire The purpose of learning level is to examine the content of the material learned It is concerned with measuring the knowledge principles, facts, techniques and skills presented in a program, i.e., to see how they understood and absorbed the material On behavior level, it is the job performance that is to be measured to examine how affective the program is on participants‟ performance in their jobs The evaluation at result level, lastly, aims at organizational improvement

At this level, some of the results can be examined are cost savings, work output, improvement and quality changes

Although the Kirkpatrick‟s framework provides a useful basis for evaluation plan design, Aldito et al (2006) state that an evaluation methodology specifically for e-learning applications is necessary but not available They emphasize usability, accessibility and didactic effectiveness as important components of e-learning evaluation Horton (2001) recommends that the criteria for evaluating e-learning include: business issues, technical issues, content, instructional design, practice and feedback, usability, media, navigation and control, and motivation Lam and McNaught (2007) suggest that both technical and pedagogical elements be considered in evaluation Other scholars also suggest their own evaluation model sharing similar characteristics with Kirkpatrick‟s to some extent Thair (2008) mentions Treadway Parker‟s model with four types of evaluation: job performance, group performance, participant satisfaction and participant knowledge gained

According to Parker, most evaluation studies focus on the last two categories Jackson and Kulp (1979, as cited in Thair, 2008) develop a slightly different model with four levels of program results: reaction outcomes, capability outcomes, application outcomes and worth outcomes among which „reaction outcomes‟ presents the participants‟ opinions of the program as a whole or as specific components such as program content, materials, methods or activities, „capability outcomes‟ covers what participants are expected to know, think, do or produce by the end of the program, „application outcomes‟ refers to what the participants know, think, do or produce in the real-world settings, and „worth outcomes‟ represents the extent to which an organization benefits from training in terms of money, time, effort, or resources invested

In his study, Thair (2008) proposes an adaptation of Kirkpatrick‟s model which can be applied to e-learning environments The proposed model contains three areas:

+ Interaction + Learning + Results

The interaction phase investigates the utility of the e-learning interface, its aesthetic qualities, user satisfaction and interaction as well as the ease at which the interface facilitated learning

The learning phase measures the actual learning as the direct result of the course It is to assess whether the learner has learned the information or acquired the skills necessary to apply to the pertinent area

e-The results phase examines the benefit of the knowledge acquired, the ability for an employee to function effectively and efficiently after the training as well as the overall intrinsic and extrinsic benefits for both the employee and employer

In the study on evaluating online tutorials for university faculty, staff, and students, Brill and Park (2011) propose a conceptual framework applied to the evaluation of those online tutorials as follows:

Figure 2: Conceptual framework applied to the analysis of PBJ tutorial evaluation

This conceptual framework shares the same categories as those proposed by Thair (2008) including three levels of reaction (interaction), learning and transfer (results) Learning conditions, learning and learning application are the three dimensions of the evaluation In order to evaluate the evaluand‟s reaction, two components of learning conditions are examined including instructional design and user friendliness Seven categories in learning conditions play an essential role in determining the extent the users are satisfied with the tutorials, how these affect their learning and to what extent they can apply what they learned to their work

For online-learning program in blended learning course at HaUI, Thair‟s adaptation of Kirkpatrick‟s model and the above conceptual framework applied by Brill and Park provide a strong basis for the evaluation of its effectiveness Because

of the nature and the scope of the study, only two levels of reaction (interaction) and learning are chosen to evaluate

1.3 Online learning at Hanoi University of Industry

Faculty of Foreign Languages is a new faculty at Hanoi University of Industry Seeing the fact that the growth of English is accelerating at very high rate and becoming more and more important to students‟ future careers, the University has paid great attention to teaching and learning English New technological and educational achievements are always priorities to be applied to have better teaching and learning outcomes Therefore, online learning received considerate consideration from leaders of the Faculty to be adopted as part of two English courses for non-English major students with the hope that their English knowledge

as well as skills will be highly improved This is the first time online learning has been applied into teaching and learning English at Hanoi University of Industry in cooperation with GK Group-an online learning provider

CHAPTER 2: METHODOLOGY

2.1 The setting of the research

The research is conducted at the Faculty of Teacher Training and Tourism (FTTT) and Faculty of Mechanical Engineering (FME) at HaUI Students in the FTTT are said to be generally better at English in comparison with those from other departments in the university Many of them have a wider range of vocabulary and acquire better knowledge of grammar while most students in the FME are of lower levels Before they take online learning as part of their English course, they are assumed to be at elementary level as they have finished the two first terms learning English with the course book New Headway Elementary- third edition In the third term, they once again work with New Headway – third edition but at pre- intermediate level This is an integrated course book; therefore, the purposes of this course are enhancing students‟ knowledge of grammar, vocabulary, pronunciation and improving four skills of speaking, listening, reading and writing

2.2 Overview of the syllabus with the adoption of Online Learning Program

After students finished their first English course in the first term, two next courses were chosen to be learned in accordance with online program: they were General English 2 and General English 3 of which New Headway pre-intermediate was the course book The book was divided into two parts The first part consisted

of 7 first units and the second included 7 last units Each part of the book was learnt during one term This means that the content of the book was learnt during two terms when students finished their first year (General English 2) and began their second year (General English 3) There were, therefore, two parts of online learning program integrated to two courses For the General English 2, the online program had 7 units in consistence with 7 first units to be learnt during the term 7 last units for General English 3 were learnt in accordance with 7 units of online program which is under investigation The names of units in the course book and those in online program are exactly the same and the content of each unit in the program is related to the names of units Each online unit has three to five lessons of which two

are compulsory and the others are optional Each lesson is designed with activities

of different types (multiple choices, True/False, error correction…) to improve students‟ grammar, vocabulary and three skills: reading, listening and writing The program does not support speaking skill in forms of direct conversations or discussions Some activities of indirect forms and pronunciation activities are replaced instead

Each teacher in charge of teaching these courses and each student were given accounts to get access to the website: http://www.haui.vietnamlearning.vn When entering the website, after confirming registration information and some installations, students could start learning each unit Each time students learned on this website was recorded on another website to be only used by the teachers who taught them Because this program focused on individual‟s learning achievements, there was little interaction among learners Students finished the activities in each lesson and their answers were recorded to be automatically marked by the program Students then could know whether they had right or wrong answers and correct them until they are right Each time students finished the activities, they knew the results

Each teacher participated in these courses had 5 periods (equivalent to 225 minutes) to help students in their learning They could answer questions related to the lessons or knowledge related to the course Teachers and students could arrange time themselves provided that it is at their convenience Other technical troubles would be solved by GK Group via emails or on telephone Besides having an online account, teachers also had a common account to another website:

http://lms.haui.vietnamlearning.vn to check their students‟ learning progress Every week, students were assigned a lesson of each unit which was parallel with the unit learned in class One week later, teacher exported the report telling their students‟ learning process From this report, teacher could know which students had not finished their lessons Because this is an integrated part to the main course, the mark

students got from it was just one of two progress marks among four marks needed for the course

As for the syllabus, the blended learning course was carried out in 90 periods (65 of which were carried out face-to-face in class and 25 others were done online

by students) within 15 weeks of the term Every week, there was one face-to-face meeting of four periods (each one lasted for 45 minutes) in class which involved mainly with traditional teaching method Every two weeks, one unit was taught Therefore, 7 last units in New Headway - pre-intermediate and 7 units of Online Learning Program were evenly distributed to 15 weeks plus three tests

2.3 Subjects

2.3.1 Questionnaire respondents

The respondent group for survey questionnaire includes 165 second-year students among which are 55 students from class QTDLKS1, FTTT, 55 students from class CĐT2, and 55 others from class CK1, all of them come from FME, HaUI with the age ranging from 18 to 22 They come from different backgrounds and environments, varied from cities to mountainous areas of Vietnam The majority of students have learnt English for about 7 years, but a few of them started learning it when they attended the university Before learning General English 3 with online learning program, they had finished the first term with General English 1 at elementary level and second term with blended learning course at pre-intermediate level

2.3.2 Interview informants

10 teachers of English at Faculty of Foreign Languages are chosen to interview Their age ranges from 25 to 35 with at least 2 years of teaching experience at HaUI 5 of them have MA degrees, 3 are taking part in post-graduate program and 2 have BA degrees Most of them have very good command of English and are experienced in teaching English to students

2.4 Data collection instruments

2.4.1 Questionnaire

The purpose of the questionnaire is to explore students‟ attitudes and opinions towards online learning as part of the course, the benefits students get from online learning, the challenges they come up with during the course, and their suggestions for better use of online learning in their learning of English The survey questionnaire was translated into Vietnamese to help them understand the questions clearly and give correct answers

2.4.2 Interview

Structured interviews were carried out to 10 teachers so as to investigate their opinions, attitudes and the suggestions for maximum use of online learning The interviews were carried out when it was convenient, either at Faculty of Foreign Languages or during the break time between two-period classes in the staff room The interview lasted from 10-15 minutes for each teacher

2.4.3 Pre-test, post-test and students’ online learning reports

 Pre-test and post-test: To evaluate students‟ progress during the

implementation of the OLP in the term, two tests were designed The pre-test was taken at the beginning of the course (in the second week of the term), and the post-test was carried out by the end of the course (in the 14 th week of the term) The content of the two tests were mainly taken from the online materials with familiar types of exercises to students These tests include grammar and vocabulary, reading, listening, writing and speaking and were carried out offline (in class)

 Students’ online learning reports: Every week during the term, students‟

online learning reports were exported by their teachers from the website for learning management The data in each weekly report show students‟ completion of the lesson (%), the score they got (on the scale of 100) and the time they spent learning that lesson At the end of the term, the final report was exported to show students‟ completion of all lessons (%), the score they

got for the whole term ( also on the scale of 100), the whole amount of time they spent on learning all the lessons and the total entries (times they got access to the website to learn) These results accompanied with those of pre- test and post-test are the basis to evaluate students‟ online learning

2.5 Data analysis procedures

To analyze data, both quantitative and qualitative methods are employed Information collected from questionnaires will be quantified and displayed in charts and tables In order to figure out the common themes, data from interviews will be coded Data from online reports, pre-test and post-test will be assessed to evaluate students‟ learning

CHAPTER 3: DATA ANALYSIS AND DISCUSSION 3.1 Data analysis

3.1.1 Questionnaire

The questionnaire aims at investigating the students‟ opinions and attitudes

to OLP 19 questions were designed to uncover this purpose in which the first question and the second question aiming at examining the importance of learning English and the OLP The 14 following items were to discover students‟ opinions and attitudes toward the program in terms of three aspects of instructional design, user-friendliness and support In order to find out the difficulties they have during their learning related to these items, question 17 was addressed Question 18 searched for the benefits of the program to the students and their suggestions to improve its effectiveness were supposed to reveal in the last question Among 165 copies of the questionnaire delivered, the number of copies returned was 160 and they were all valid

3.1.1.1 The importance of learning English The first question was designed to examine how important learning English

is to the respondents As shown in the results, all of the students taking part in the survey highly appreciate the importance of English and the necessity to learn this popular language 76% agreed that learning English is strongly important and 24% thought that it is very important No one said that learning English is so so, not very important or strongly unimportant

Chart 1: The importance of learning English

76

24

0 0

0 strongly important

very important

so so not very important strongly unimportant

3.1.1.2 Students‟ opinions and attitudes toward the Online Learning Program

To seek for students‟ attitude to the OLP implemented at HaUI, the second question addressed the importance of this program No one chose strongly unimportant; therefore, the answers varied from strongly important to not very important 9% found that the OLP was strongly important, 49% said that it was very important while only 3% agreed that it was not very important 38% thought that it was so so, which means the importance of the OLP was neutral to them Among the respondents, those from CĐT2 were in the first position in their evaluation of the importance of the OLP with 66% agreed that it was strongly important and very important while those from QTDLKS1 and CK1 came with lower percentage of 56% and 54% respectively It can be said that to the respondents of CĐT2, they showed more appreciation to the necessity for OLP to be introduced into the course The following chart shows the details:

Chart 2: The importance of Online Learning Program

so so not very important strongly unimportant

In order to go deeper into the evaluation of OLP, the 14 following items were designed concerning three aspects: instructional design, user-friendliness and support In instructional design, students were expected to show their opinions about the clarity and suitability of the objectives of the program, the relevance of topics and content, the opportunity to practice and the appropriateness of the

exercises to their level To examine how friendly OLP was to users, students were asked about the interface, the design of the website, the accessibility and navigation, the instructions, the graphics and multimedia, the texts and the knowledge presented

in the website The aspect of support was to find out information about the time for learning and the support from the provider as well as teachers to students‟ learning

4 = disagree 5 = strongly disagree

No of items

1 (%)

2 (%)

3 (%)

4 (%)

5 (%)

Instructional design

Table 1: Students’ opinions on the three aspects of the OLP

As can be seen in the table, the opinions toward three aspects were quite positive The majority of the respondents showed that they strongly agreed or agreed with items provided For instructional design, the percentage ranks from 71- 77% strongly agree and agree, 4-24% undecided, 7-22% disagree and strongly disagree For user-friendliness, the percentage of strongly agree and agree accounts for 73-81%, undecided 9-18%, disagree and strongly disagree 7-21% The percentage for support are 76-82%, 6-12%, and 6-17% for strongly agree and agree, undecided, disagree and strongly disagree respectively So it can be said that those who showed their agreement outnumbered those with their disagreement

In instructional design, item 3 was designed to seek for the information about the objectives of the OLP 28% strongly agreed and 43% agreed that the objectives were clear and met their goals in learning while only 5% disagreed with it and 24% remained undecided Regarding the topics and the content of units in the website, 77% strongly agreed and agreed that they were relevant to what they wanted to learn That means the topics and the content met their needs However, there was 20% who disagreed and strongly disagreed indicating that the topics and the content might not be suitable to their needs or unfamiliar with them Students were also asked about the opportunity to practice in each lesson they learned in the website For this item, 29% strongly agreed and 46% agreed that practice part was available

to them in each lesson while 14% disagreed and 4% strongly agreed with it Concerning the exercises they had to do in each lesson, 75% strongly agreed and agreed that they were appropriate for their level This, however, is not true to 12%

of them

To decide the level of user-friendliness of the OLP, item 7 was designed to ask students about the interface of the website 81% strongly agreed and agreed that the interface was user-friendly while 7% disagreed with user-friendly characteristic

of the website With regard to the artistic feature of the website, 15% strongly agreed and 49% agreed that it was designed in an eye-catching way The percentage

of those who disagreed and strongly disagreed with this item was 18% Below are some pictures that show the interface and part of the website‟s design

The following item was designed to examine the accessibility of the website 69% strongly agreed and agreed that the website was easy to access while 21% disagreed and strongly agreed This shows that these students had difficulty in accessing the website to learn When learning on the website, the navigation and control are very important, so it is another item to be examined 73% strongly agreed and agreed that it was easy to move around without getting lost 17% showed their disagreement with it, which means they had troubles with navigation and control during their learning time The next item was to figure out the clarity of the instructions for tasks in each lesson 23% strongly agreed and 56% agreed that the instructions were clear and easy to understand This is very important to students as it affects the way they understand and do the exercises Item 12 was to ask about the help of media in their learning 73% strongly agreed and agreed that the graphics and multimedia were helpful to them The number of respondents who disagreed or strongly disagreed with the helpfulness of media used in the OLP accounts for 12% Similarly, the percentage of students who disagreed that the texts were written in an understandable level was 10% while that of respondents with their agreement was 75% This indicates that the texts used in the OLP were

suitable for them to read and understand In order to find out information about the way the knowledge was presented in the website, item 14 was designed 77% said that they could understand the knowledge presented in each lesson In contrast, 11% disagreed that the knowledge was presented in an understandable way implying that these students had problems understanding the knowledge because of its presentation

This is the first time an online learning program has been introduced into the syllabus, so the support for students plays an important role Item 15 was to ask students‟ opinions about learning time and item 16 asked students about the support they received from the service provider and their teachers during their learning process A large number of students highly evaluated the flexibility of learning time (82%); only 6% disagreed that the time was inflexible During the term, the support they received from the provider and their teachers was agreed to be always available (76% strongly agreed and agreed) while 17% of them disagreed with that

The purpose of question 17 was to uncover the difficulties students had with items 3-16 during their learning process Most of students‟ responses focused on problems with items 4, 5, 8, 9, 10, and 16 Item 4 addressed the relevance of topics and content to what students want to learn For this item, many respondents said that some topics and content were not familiar with students Sometimes, there were many topics in one unit leading to confusion in their learning Item 5 mentioned the opportunity to practice in each lesson For this item, students showed their expectation to have more tasks to practice 4 skills instead of many grammar exercises They said that although the number of exercises was large, they did not have many chances to practice their English skills because most of them were grammar exercises and they were repeated causing boredom to students Item 8 was about the design of the website 18% thought that it was not designed in an eye- catching way They said that the design is too simple, not beautiful as other online learning websites Concerning the accessibility, the reasons for 21% and 17% of disagreement for items 8 and 9 respectively were the difficulties to get access to the

website and the problems occurred frequently while they were learning or navigating in the website Most of these were stated to be technical problems In item 16, the support from the provider and teachers was mentioned 17% of respondents did not agree that the support was always available to them They did not mention the teachers‟ but the provider‟s They said that they hardly received any support from the provider when they had technical problems It can be implied that students really needed support from the provider to solve some technical problems while they were learning They did not mention their teachers maybe because they usually received the support from them for content-related matters

Although the problems during learning process are unavoidable, students can not deny the benefits the OLP brought to them as shown in their answers for question 18

Chart 3: The benefits of the OLP to students

0 10 20 30 40 50 60 70 80 90 Strengthen grammar

Broaden vocabulary Improve pronunciation Improve 4 skills

Improve self-study ability More motivated in class

More interested in English

Most of students (78%) found that the program helped them strengthen their grammar, 60% said that their vocabulary was broadened after the implementation of the program, but only 28% agreed that the program helped them improve their pronunciation Concerning the four skills, 68 students accounting for 43 % claimed that their skills were improved thanks to the OLP One interesting thing is that 72%

stated they improved their self-study ability This is very important as enhancing students‟ autonomy is one of the main purposes of the program One more benefit

of the OLP is that it made 61% respondents more interested in English and 36% more motivated in class though this number was not quite high They also commented that the OLP was generally good, useful, convenient and flexible in time

In order to improve the effectiveness of the program, question 19 was designed Number one in the list of suggestions from students was the necessity to use more audio-visual aids such as pictures, audios, video clips to make the lessons more interesting Another suggestion related to entertaining matter or students‟ motivation They need more songs, funny stories or game shows to be introduced so that they can relax after learning The topics were suggested to be more familiar with their daily lives so that they could understand them better and do exercises more effectively They also expected that there should be more exercises on pronunciation and the answers for speaking part should be recorded and marked For vocabulary and the answers for other parts such as reading, grammar, they need more detailed explanation so that they can have deeper understanding what they are learning Relating the answers for each exercise, they suggested the answers for the first time be saved or there should be no checking with available answer keys right from the beginning on the website They thought that it would be better if they got the keys in another way, from their teachers for example They also said that the provider should pay more attention to the keys as some of them are not correct resulting in demotivation to learners Because this program focused on students‟ autonomy, it was suggested that there should be stricter control on their learning to avoid their compulsory and negative attitude to the OLP The last suggestion concerned with the support and technical problems They thought that it would be necessary to have the help from the provider when they have troubles while learning They need high quality website with as few technical problems as possible Because many of them did not use Internet Explorer in their computers,