Duong Thi Hong An vd Dtg Tgp chi KHOA HQC & CONG NGHE 175(15) 103 10 A N E V A L U A T I O N O N T H E E F F E C T I V E N E S S O F T H E E N G L I S H S O F T W A R E S Y S T E M F O R N O N E N G L[.]
Trang 1A N E V A L U A T I O N O N T H E E F F E C T I V E N E S S O F T H E E N G L I S H S O F T W A R E
S Y S T E M F O R N O N - E N G L I S H M A J O R S T U D E N T S A T T H A I N G U Y E N
U N I V E R S I T Y O F I N F O R M A T I O N A N D C O M M U N I C A T I O N T E C H N O L O G Y
Duong Thi Hong An", Dang Phuong Mai
TNU University of Information and Communicalion Technology
S U M M A R Y
CALL (Computer Assisted Language Leaming) has proven its high effectiveness in the frend of information and communication technology (ICT) applications in education and fraining in general
and teachmg English in particular The project "Building the software system for supporting the
Project 2020 and Thai Nguyen University (TNU) was piloted at Universfty of Information and Communication Technology (ICTU) fi-om August 2015 to May 2016 The participants were 400 fi-om the lecturers and the students were sent to the project group Better study results made by the fu-st group of 200 students usmg the courseware compared with the ones made by the second group
of 200 students who did not use the courseware were also the proof of its effectiveness This bright perspective opened a broader implementation in the next stage ofthe project
Keywords: CALL; courseware; multimedia; evaluation, effectiveness
I N T R O D U C T I O N
The use of TCT is one of the factors that
changed the Vietnamese students' practice of
learning English as a foreign language Such
views are m synchrony with the official
directive from the Mmistry of Education and
Training, which urges EFL lecturers to
integrate multimedia and digital technology
into theu teaching At TNU, the project
"Buildmg the software system for supporting
the learning, teaching, reviewing and
self-assessing English" has been cairied out by
TCTU project group since 2015 The project
aims at supporting English teachmg and
learning at T N U and enhancing CALL abilities
of both lecturers and students In order to
examine the quality of this English language
teaching (ELT) multimedia courseware, it is
necessary to conduct quality evaluations on it
Thus, we examined and evaluated this
multi-media courseware piloted at ICTU, T N U
THEORETICAL REVTEW
Computer assisted language learning (CALL)
CALL has been defined as "the search for
and study of applications of the computer in
teaching and learning' [5]
Tel: 0904267 273, Email: dthan@lctu.edu.vi
Corresponding to the evolution of computer technology in the past decades, the development of C A L L can b e categorized into three stages: behaviouristic CALL, communicative CALL, and integrative CALL [10] The first stage of C A L L has been termed behaviouristic or sfructural CALL, which was characterized by route learning in the forms of pattern and drill exercises and explicit instructions of grammar Communicative C A L L emerged in late 1970s and sfressed that students should be encouraged to produce language in authentic communicative settings instead of manipulating prefabricated forms The mid 1990s witnessed the emergence of the integrative C A L L which sought to integrate both basic language skills and computer technology more naturally and fully into language learning Nowadays, the digitali-zation of language materials through information technology provides multiple chatmels for students to obtain information, which unobtrusively breaks the fraditional view of input of language learning N o t only can multimedia courseware redirect learner's attentional resources, but also it has the
Trang 2potential to enhance the salience of language
features Highlighting, colour-coding,
underlining, and font manipulation are typical
examples of visual input enhancement [6],
CALL was introduced into Vietnam in early
2000s and the research on English
instructional courseware in Vietnam lagged
behind that of their foreign peers who started
lo investigate courseware' in 1990s,
Furthermore, in the past decade, English
multimedia courseware has been widely used
in English teaching al tertiary level
Therefore, there is a need to conduct an
evaluation on the quality of courseware,
which could not only guide courseware
design but also facilitate the development of
future courseware evaluation
Courseware evaluation
Systematic evaluation of CALL courseware
generally fell behind the development efforts
[2] Courseware evaluation refers to the
process of measuring the appropriateness and
effectiveness of a particular instructional
system [1], [3], Hubbard sfressed that
judgments made by instructors and learners
should be at the core of CALL software
evaluation so that teachers' and learners'
considerations could be combined to yield
assessments of the appropriateness of a piece
of CALL courseware in a given context
Several research has been done to evaluate
multimedia learning materials from both
teachers' and students' perspective [4], [9],
METHODOLOGY
Contents ofthe courseware
The frame ofthe courseware includes five main
parts: Lecture Notes - Language Reference —
Practice - Mini-test - Trial Exam
The course books integrated in the system are
English Unlimited with 2 levels from A2 to
Bl [7], [8], Currently, three TNU university
members are using them as the main course
books These books aim at developing the
language knowledge and skills as in Common
European Framework Reference (CEFR) from Al to CI The four skills associated with Vocabulary, Grammar, Pronunciation, Cultures are integi'ated in each book The English curriculum at TNU has from 10
to 12 credits, which is equivalent lo 200 lecturing hours With the average input level
of English is Al while the English output standard is A2 and Bl This means one level raise of students' language competence Based on previous studies, in order to reach this goal, 300 to 600 class hours and self-study should be met However, due to time constraints, it is hard to exploit the couise books Therefore, the integration some parts
of these course books were integrated into the courseware so that students can study on their own, i.e., grammatical practice, reading, listening, writing, pronunciation and mini-tests Both TNU lecturers and students can exploit the learning and teaching recourses
The Grammar Reference is designed with
theme-based units in which learners can he supported online anytime when they want to check grammar theories They are in both English and Vietnamese so that student can understand easily
The Lecture Notes are lesson-based are
designed for teachers and students to use in class or at home since they are available online Teachers can actively share or upload their teaching materials with other teachers or students under the confrol of an professional administrator
The Practices are collected and processed lo
avoid copyright violation They follow the lecture contents supporting learners to review what thay have learnt in class for each unit Teachers also can assign further practice for students in the online system to assess their self-studying and participation Some audio and video are recorded by English native speakers who are our foreign students al ICTU The vividly-designed exercises can motivate learners to study online
Trang 3The Mini-tests are designed along with the
contents of each unit so learners can review
and assess their progress themselves
The Trial Exam level from A2 to B l is
collected, processed and mixed to created
different trial test for learners to practice and
self-assess before they take real English
output exams at TCTU
Participants
The first group of participants were 200
non-English major undergraduate E F L students
and 15 E F L teachers at ICTU, T N U The
questionnaire survey was administered in a
laboratory where the instructors taught the
students English skills using the courseware
The teacher questionnaire survey was carried
out in a meeting room after the teachers
delivered their lessons
The second group of participants were 200
non-English major undergraduate EFL
stu-dents who did not use the courseware Both
are at the same level of English language
competence ( A l ) based on their placement
test results After 10 months, both groups
took the s a m e final exams The results o f t h e
final semester exam results were compared
Instruments
The first group of participants were given the
survey questionnaires to evaluate the
courseware after 10-month-use
Eight courseware evaluation criteria include:
(1) The contents provided in the courscMwe are
appropriate with the standardized curriculum
(2) Users can use the system al the different
stages: building knowledge, reinforce and
improve skills, knowledge application, testing
(3)^ The courseware creates the better and
favourable teaching and learning environment
titan other traditional ones, which assists them
to be more active and creative
(4) The courseware ensures the learning
objectives evaluated by the teachers for the
(5) The system assists users with creative activities and with adjusting or adding information to the system
(6) The courseware is suitable for psychological of learners without overloaded knowledge and stress-free interface (colour, design, audio )
(7) The system helps users to solve problems emerging while using and give timely feedback (8) The system can archive the studying results and duration for learners and also for the purpose of evaluation or research
The evaluation based on eight criteria with a
five-point Likert-type scale: (1) Totally agree, (2) Agree, (3) Not sure, (4) Disagree, (5) Totally disagree
S u r v e y results
Table 1 Questionnaire survey results
Criteria Grit i Grit 3 CritS Cnt 7 CritS Average Percentage
Totally ^ ^ ^ agree "
133 65
125 72
117 75
131 65
123 69
112 70
134 75
196 91,2
Not sure
9
7
12
8
14
IS
4 9.75 4,5
Dis-5
6
S
6
2
Totally
e disagree
9,25 4,3
3
2
3
3
0
From the table above, we can see that majority of the participants totally agree/
a g ' e e (91,2%) w h h eight courseware evaluation criteria while the number of participants w h o are not sure, disagree
or totally disagree was minor (4,3-4,5%),
Compared final e x a m r e s u l t s
Table 2 Compared final exam results of two groups
Grade A B Group 1 ( Use tht Semester 1 15% 48%
Semester 2 17% 45%
Group 2 (Do not use Semester 1 5% 20%
Semester 2 7% 21%
G D
• courseware) 22% 12% 25% 11% the courseware) 49% 19% 45% 22%
F 3% 2%
7% 5%
Trang 4From the study results collected fi-om
Department of Academic Affairs of TCTU, we
can easily see that the percentages of student
getting grade A, D, F from both group are
almost the same The remarkable difference
can be shown in the percentage of students
getting grade B and C Tn the first group,
nearly half can get grade B while in the
second one, only 20% of them can reach B,
Courseware analysis
Strengths
More importantly, first group of participants
assessed the strengths and weaknesses of the
online system and gave some useful
suggestions for the project group The
strength of the system can be noted as
follows Firstly, the contents are evaluated to
be appropriate with the cirieulum and
asociated sfrictly with the course book
English Unlimited Every lesson has 4
contents: Lecture Notes Grammar
Reference — Practice — Minitest Practice part
is really useful for learners with vivid design
and connected with the grammar focus and
vocabulary It helps to develop students's
listening, reading, writing and pronunciation
skills, Minitests is applied after learners finish
each unit so they can have quick reflection
from what they have learnt From this, they
can adjust their learning method suitably
A2-Bl trial exam bank in the software system is
designed on Cambridge ESOL format, which
assists students to get accustomed to test
sample with different skills and levels They
can self-assess their own progress and
actively practice for the output exam by TNU
when they are in their first year Secondly,
learners' studying results are achieved during
the learning process so lecturers can easily
supervise their student learning progress
through Course Management, Class
Management They can also know the
self-studying hours by students via Online
learning duration This can count
automatically students self-learning duration
through their practice, review and revise for exam With the parts that need teacher's evaluation and marking, students can send their works directly to their teachers and get feedback via SMS and email system Students are managed in class according to their levels
of English profieney and are assigned homework individually During theu onlme study, students can send MSM, email or they can share their learning experiences via our forum or Facebook, Students can make their own study plans and save their study results, which make teachers easier to evaluate their students and viee versa This software system
is connected to Intelligent University (IU) so teachers can actively manage their own classes automatically Thus, students are more active in leaming English online because both participation in class and online are assessed
by teachers Thirdly, the intergrated system creates more favourable conditions for not only teachers but also students to teach and learn English online Teachers can design further practice or tests for their class under the control of professional admin They can also exploit the availble teaching resourses in the software system Moreover, e-Lessons with vivid and harmonious design make students feel more comfortable and sfress-free for students to study continuously in 1 or 2 hours a day
Weaknesses
Courseware users pointed out several weaknesses of the courseware Firstly, it does not have fimctions for developing speaking skill or voice recognition for learners lo improve their pronunciation Secondly, it does not have interactive activities or games for widening learners' vocabulary Thirdly, the evaluation and marking by teachers do not implemented immediately In fact, with the large amount of students' writing works sent
by students, teachers have difficulties in solving them at onee, normally within 24-48 hours Last but not least, the number of
Trang 5questions for revising for A2-B1 exam is not
adequate Within the scope of this project, we
provide 1000 questions for A2 exam and
1000 questions for Bl exam
CONCLUSION
After 10 month-piloting, the courseware
proved to be partially successfiil in supporting
ICTU lecturers and students in their teaching,
learning, reviewing and self-assessing
English Learners could be more active and
independent in their self-study while teachers
could save much time in theu class
management Moreover, the shortcomings of
the courseware were adjusted to cater for the
practical needs from both lecturers and
students The next stage ofthe project is using
the courseware for the whole undergraduates
at ICTU then if any TNU university members
find it useful, TCTU is willing to share
ACKNOWLEDGEMENTS
This paper Is the product ofthe project, code
T2017-07-21, funded by TNU-University of
Information Technology {Bdi bdo la sdn
phdm cita di tdi cd md sd T2017-07-21,
duac ldi trg b&i kinh phi cua tru&ng Dgi hgc
Cdng nghi thong tin & Truyin thong, Dgi
hgc Thdi Nguyen)
REFERENCES
1 Colpaert, J (2006), "Pedagogy-driven design for online language teaching and leaming",
CALICO Journal 23(3), pp 477-497
2 Flagg, B (1990), Formative evaluation for
Associates, New Jersey
3 Hubbard, P.(2006), "Calling in CALL: From
language teaching", CALICO 20(3), pp 313-338
4 Jamieson, L, & Chapelle, C (2010),
"Evaluating CALL use across multiple contexts"
System, 38(3), pp 357-369
5 Levy, M (1997), Computer-assisted language
University Press
6 Sharwood Smith, M, (1991), "Speaking to many minds: On the relevance of different types
of language information for the L2 leamer"
Second Language Research 7(2), pp 118-132,
7 Tibury, A, Clementson, T, Hendra LA and
Rea.D (201 \) English Unlimited Elementary A2 Course book with e-Portfolio, Cambridge
University Press, London
8 Tibury, A, Clementson, T Hendra, LA and
Rea,D (2011), English Unlimited Pre-inlermediale
Bl Course book with e-Portfolio: Cambridge
University Press, London
9 Van Doremalen,!., Boves, L., Colpaert, J,,
Cucchiarini, C , & Strik, H, (2016), "Evaluating
automatic speech recognition-based language
leammg systems: A case study", Computer Assisted Language Learning, 29, pp, 833-851
10 Warschauer, M„ & Healey, D (1998),
"Computers and language leaming: An overview"
Language Teaching, 3(\), pp 57-71
Trang 6T O M T A T
D A N H GIA H I E U Q U A SU" D U N G H £ T H O N G P H A N M E M T I E N G A N H
C H O SINH VIEN K H O N G C H U Y E N TIENG A N H T A I TRlTOfNG D A I HOC
C O N G N G H E T H O N G TIN V A T R U Y E N T H O N G - D A i H O C T H A I N G U Y E N
Duong Thj Hong An', Dang Phiro-ng Mai
Trudng Dgi hoc Cong nghe thong tin vd Ti uyen thong - DH Thdi Nguyen
Viec hoc ngoai ngii co su ho tro ciia may tinh da chung to hieu qua trong xu the ling dyng
CNTT&TT trong giao due va dao tao noi chung va giang day tieng Anh noi rieng Du an '•'Xdy dung phdn mim hoc liiu ho tra hoc lap, gidng dgy, on luyen vd iu ddnh gid mdn tiing Anh " tai trgr
Cong nghe thong tin & Truyin thong tu thang 8 nam 2015 den thang 5 nam 2016 Doi tuong nghien cuu la 400 smh vien nam thii nhdt va 15 giang vien tieng Anh cua Trucmg Sau giai dogn
thi diem, nhom du an nhgn duoc cac phan h6i tich cue tii phia giang vien va sinh vien thmg qua
voi nhom 200 sinh vien khong sir dung ph4n mem chiing to hiSu qua ciia hg thong Dilu nay mb ra
theo ciia duan
Til' khoa: Hoc ngoai ngd cd ho trg cua mdy linh, phdn mem hgc lap, da phuang tiin; ddnh gid; hieu qud
Ngdy nhgn bdi: 10/9/2017; Ngdy phdn biin: 09/10/2017; Ngdy duyit ddng: 29/12/2017
Tel: 0904 267 273 EmaiT dlhan@iclu.edu vn