A peer assessment approach to project based blended learning course in a Vietnamese higher education

A peer assessment approach to project based blended learning course in a Vietnamese higher education tài liệu, giáo án,...

A peer assessment approach to project based blended

learning course in a Vietnamese higher education

Viet Anh Nguyen1

# Springer Science+Business Media New York 2016

Abstract This article presents a model using peer assessment to evaluate students taking part in blended - learning courses (BL) In these courses, teaching activities are carried out in the form of traditional face-to-face (F2F) and learning activities are performed online via the learning management system Moodle In the model, the topics

of courses are built as a set of projects and case studies for the attending students divided into groups The result of the implementation of projects is evaluated and ranked by all course participants and is one of the course evaluation criteria for lecturers To assess learners more precisely, we propose a multi-phase assessment model in evaluating all groups and the group members The result of each student in the group based on himself evaluation, evaluations of the team members, the tearcher and all students in the course There are 107 students, who participated in the course entitled Bweb application development^, are divided into 20 groups conducting the course in the field of information technology is deployed in the form of blended learning through peer assessment The results of student’s feedback suggested that the usage of various peer assessment created positive learning effectiveness and more interesting learning attitude for students The survey was conducted with the students through the questionnaire, each question with scale 5-point Likert scale that ranged from 1 (very unsatisfied) to 5 (very statisfied) to investigate the factors: Collaboration, Assessment, Technology showed that students were satisfied with our approach Keywords Peer assessment Blended learning Many phases assessment

Learning activities

DOI 10.1007/s10639-016-9539-0

* Viet Anh Nguyen

vietanh@vnu.edu.vn

1

University of Engineering and Technology, Vietnam National University Hanoi, E3, 144 XuanThuy, CauGiay, Hanoi, Vietnam

1 Introduction

Blended learning (BL) model combines traditional teaching methods, teaching facilities and computer-based learning to improve the effectiveness of learning activities Currently, BL has proved its superiority in comparison with another learning method that finds out in published studies (Porter et al.2014; Tambouris et al.2014; Tayebinik and Puteh2012) Research of Osguthope & Graham (Osguthorpe and Graham2003) has pointed out six reasons to choose the design or usage of a BL system, including the abundance of pedagogical content; approaches to understanding, social interaction, personal agency, cost effectiveness and easy to modify The effective implementation

of BL usually can be done in several stages: making acquaintance, designing and testing, sharing and application, evaluation, and adjustment

One of the crucial stages is the assessment of learners - how to effectively assess learners participating the course in the BL model Assessment is not only an important part of the content, but it should also be started before teaching activities or at least during the beginning weeks of the course Students should have the ability to show their competence to complete tasks in an online environment before learning content that will be assessed later (Gikandi et al.2011) Meyen and his colleagues (Meyen et al

2002) proposed several methods for assessing online courses, including researching documents, cooperation projects, exams, and reports Additionally, promoting online contact discussions may be helpful for building knowledge Teachers can use online interactions to assess students (Rovai2000) Recent studies have shown that there are two common forms of assessment: formative assessment and summative assessment (Strijbos and Sluijsmans2010) Summative assessment separated from the learning process with a personal evaluation; the assessment usually is made by the teacher, being less dependent on contextual factors and usually happens once when students finish the course Formative assessment associated with the learning process, especially learning activities, assessing the full academic progress of students through several evaluations, tied to contextual factors to make a broader assessment of different aspects of the learner rather than simply only evaluating learner’s knowledge For BL courses, formative assessment is very effective because it not only assesses and promotes the learners but also collects feedbacks from them that help to improve the quality of the course in this form (Gaylard Baleni2015) This kind of assessment shows its superi-ority increasingly upon the support of computers which diversify the methods of making an assessment, and online assessment allows students to communicate with other members of the course, receiving comments, feedbacks and editing their works (Yang and Tsai2010) With supporting tools, teachers easily monitor the participating progress of students and make a more favorable evaluation (Lin et al.2001)

Another effective assessment in this form is a peer assessment that is definedBas an arrangement in which individuals consider the amount, level, value, worth, quality, or success of the products or outcomes of learning of peers of similar status.^ (Keith

1998) Topping Keith has developed 17 attributes of peer assessment, and many forms

of using peer assessment have been deployed such as grading, analysis, feedback, conferencing and revision (Reinholz2015) In this study, we propose a model using peer assignment with following forms: grading, wikis, group work and feedback for BL courses which were designed in the form of project-based learning (English and Kitsantas2013; McDonald2008)

In this model, the topics of the courses are built as a set of projects and case studies, which are implemented by attending students divided into groups Results of the implementation of the projects are evaluated and ranked by all course and are used

as evaluating criteria for teachers We propose a multi-phase model for evaluating groups and members of the groups

We approached formative assessment to evaluate students, combining the two forms

of evaluation: evaluation of a group and among groups With a group, we have proposed several forms of assessment phases: Firstly, students give the self-assessment, then group discusses and assess their members Finally, the teacher gives a mark for the performance of each team With evaluation forms between the groups, all class members to evaluate the results of each group’s project

In this study, we use the LMS Moodle to support the implementation of learning activities and assessment by Moodle is one of the solutions for flexible and friendly opened learning management Moodle has very useful documents and strong supports for governance and security of learning system, and Moodle is developed according to the IMS/SCORM standards With a large number of modules and plugins, Moodle can support the deployment of a variety of learning activities and assessment forms 1.1 Problem statement

UET is a public four-year university specializing in technology with the following main majors: Information Technology, Electronics and Telecommunications, Engineering Physics and Nanotechnology, and Engineering Mechanics and Automation Each year,

550 specialized courses are available for about 2500 students to select In recent years, UET has been more interested in e-learning courses, especially beginning in the academic year 2010–2011 Up to now, there were about 442 courses (~20 %) in the Information Technology and Electronics and Telecommunications major that applied the teaching/learning process in the form of blended learning Although there were many difficulties in the application process, the initial results proved that the blended-learning approach could be expanded to all the courses in the university

During implementing the BL course, the application of the model and technology for evaluating students attending the course has become an urgent need for the university instead of using the summative evaluation at the end of each course, where students often have complaints about the results after knowing the score Besides, to ensure the fairness in the assessment of the members of the working group model is a matter of concern: how correctly to assess the contribution of each member in the group This study aims to find solutions to improve the efficiency of the assessment of learners participating in BL courses 1.2 Research objective

Since this research aims to build effective evaluation solutions of learners through learning activities in the BL course in UET in Vietnam, therefore, it strives for achieving following objectives:

1) Building a multi-phase evaluation model through peer assessment activities to help teachers to evaluate students more precisely focusing on collaboration and com-munication learning elements

2) Providing solutions to assess the contribution of each member in group working environment

3) Testing the multi-phase evaluation for participating groups and each group member

1.3 Significance of the study

The study aims to find out solutions which support teachers to evaluate learners in BL courses more precisely, especially in project-based learning course emphasizing the teamwork ability, cooperation and exchange between members Besides, the study also encourages teachers to use LMS tools in the design and implementation of learning activities in BL courses in Vietnam, where the traditional form of learning and assessment is still popular Also, the research aims to promote the use of classes that focuses on improving the skills of teamwork, cooperation and exchange between the participating students– all the weak skills of the majority of students Recommended improvements of the multi-phase assessment of peer assessment activities help teachers

to evaluate each learner more accurately, especially more efficiently for large classes and students feel more responsible for their group’s work

1.4 Related literature

Liu et al (Lu and Law2012) examined the effects of online peer assessment, grading in the form of peer and peer feedback Their research has shown the benefits of online peer review on learning performance of the students With project-based learning courses, Hou et al (2007) conducted a study analyzing the content discussed in the form of peer-assessment of students This experimental research showed that before the intervention of teachers, the students demonstrated a certain level of discussing behav-ior related to the formation of knowledge, which promoted their self-study ability Study of Falchikov pointed out that student implemented peer-assessment activities in teamwork in higher education relevant to their future careers These activities played a role in promoting the lifelong learning skills, including reflection, autonomy, commu-nication, problem-solving and responsibility (Falchikov2007) However, teachers often have difficulties in assessing the contribution of each participating in group’s projects because they did not involve in the implementation process of the students and depended on the peer assessment Some forms of assessment are applied as marking/ grading, analysis, feedback, conferencing and revision With peer grading, teachers give students the criteria to evaluate other members Use of member’s feedback on the results of the work is the solution that many researchers believe that it is more effective than peer grading because of helping students to know clearly pros and cons of their work through specific suggestions instead of scores (Xiao and Lucking2008) Recent assessment with the use of the Wiki, suitable for the development of writing skills, is concerned The research results of Rekalidou et al (2015) showed that students not only applied their knowledge in the evaluation process but also developed skills in the course of constructing documents However, before conducting self-assessment process

by students, teachers should provide them examples, instructions of using evaluation tools as well as strategies to improve the reliability and accuracy of evaluating the

process (Thomas et al.2011) Many tools have been implemented, one of which is the Moodle system because of its superiority and it not only supports teachers in preparing learning plans, providing teaching materials and exercises but also effectively supports assessment activities such as peer assessment, workshops, test, forum, etc (Al-Ani

2013) Also, Moodle is an open sources product so that it is easy to be installed and deployed on a large scale

1.5 Research questions

With the goal of building multi-phase evaluation model through peer assessment activities to help teachers to assess students more precisely, focusing on the assessment

of learning collaboration and communication This study was conducted to answer the following questions: (1) How accurately and efficiently to assess the individual partic-ipation of the group in the BL course? (2) Evaluate the impact and influence of peer assessment to the students participating in the BL courses? (3) Whether the use of LMS tools to design learning activities in the BL courses encourages students to work in groups to solve the problem or not?

2 Method

2.1 Participants

One hundred seven students registered to participate in the case study The 3rd-year student of information technology took part in the course entitledBweb application development^ which consisted of 3 credits The students were randomly divided into

20 groups of at least five members up to six members to implement the projects and case studies relating to the subjects Tests were carried out between semesters, and the teacher evaluated final examination All component assessments during the course were conducted by using peer assessments

2.2 Design and procedure

2.2.1 Course design

The course was implemented in 15 weeks, including two forms of offline learning and online learning With offline learning, lecturers taught in class The theoretical lessons were implemented in compliance with the content and process in draft course plan approved by the scientific council of the university After the first class, teachers divided students into groups to implement group learning activities After participating in classroom activities in the form of F2F, students carried out online learning activities and implementation of group activities as depicted in Fig.1 Also, students must do a midterm test and a final exam, in which online activities implemented as follows:

Week 1: First individual assignment, submitted in week 2

Week 2: Group exercises Project 1 done in groups, submitted in week 5

Week 3: Group exercises Project 2 with making documents in the form of Wiki, submitted in week 8

Week 4: Second Individual assignment, submitted in week 5

Week 7: Mid-term exam, online test

Week 8: Group exercises Project 3, submitted in week 9

Week 13: Third individual exercise, submitted in week 14

2.2.2 Course grading

The evaluation of student’s completion of the course was done by giving scores Currently, according to the regulations of the university, student assessment results

by the score at the end of the course consists of two parts Part one, the learning outcomes of students making learning activities including individual implementation exercises, group exercises, tests, exams Part two, the attendance of the students involved in learning activities: there is full participation of learning activities online and offline deployed or not?

Students were evaluated according to a scale of 10, in which the proportion of component scores was calculated in Table 1 In particular, the necessary activities required students to attend were two of the three individual assignments, two of the three projects, 01 mid-term exams, and the final exam

Fig 1 Learning activities of the project based blended learning course

Table 1 Point value for grading

We build weighting of evaluation table based on two principles 1) The scientific training committee of university regulations weight for a final exam, it is from 50 % to

70 % of total grade 2) The teacher can himself construct the weight for the remaining points which are from 30 % to 50 % of total points In this study, we propose the weight

of final exam is 55 % With 45 % of evaluation for other activities, we evaluate students through the implementation of learning activities: personal assessment, assessed by the group Time to finished as well as the workload of each project as a basis for forming weighted points Project workload requires larger and need more time to complete the assessment points higher weights and vice versa For example, individual assignments

No 1, asked to complete after one week, with pure content, should have little weight (3 %) Project No 3, with duration of 4 weeks (accounting for 1/3 of the length of the course) should have higher weight (10 %)

2.3 Three types of peer assessment

2.3.1 Individual assignments for peer assessment

In the model, individual assignments were assignments for students to complete a period of one week Students must submit their work through the LMS system Individual exercises were evaluated by the teacher’s assistant and at the same time exercise of each student was assessed by other students randomly selected in the class with criteria made by teachers In this experimental course, individual assignments of week 1 required of students writing an essay about theBHistory of web development,^ each student was asked to write an essay about 500 words, then all members of group discussions and edits those, thereby forming an essay of the group to submitted the final version The participants of the course evaluated essays of 20 groups on a scale of 1–5, including their own group’s essay through the exploration of Google form

Evaluation scores of each member shall be based on the assessment of teachers and the group In this assignment, the team members had equal scores Individual assign-ments No 2 and No 3 tested programming skills of the students The students completed and submitted assignments through the LMS system Teachers gave criteria; each one had a different level of maximum points for evaluating the program code 2.3.2 Group exercises for peer-assessment

Exercises were done in groups; each group consisted of about 5 to 6 members selecting one project, the execution time of approximately 4 to 5 weeks Assessment of project performance was conducted in three stages: (1) The mem-bers of the group conducted their self-assessments based on its level of contribu-tion to the joint work, the team members classified the levels A, B, C, D respectively (2) The groups discussed and concluded a final assessment for each member of the group, with the consensus of the whole group in the minutes certified by each member The results of the peer assessment of group members were considered valid when the members had different levels of assessment, in other words, the assessments of groups at the same degree A, B, C or D were not accepted because almost there are no equal contributions of each member in the group (3) The results of group assignments were reported in class in offline form,

where teachers evaluated project performance through scores by the criteria in the project requirements Students in the other group filled out the rating forms of all groups after hearing all group presentations

2.3.3 Mapping group peer assessment to personal grade

The final scores of teachers consist of the evaluation score of teachers and equivalent score from rating forms of all members after hearing the reports presented by groups The basis of scoring for the group members according to the following principles: The final grade will be the average of individual scores of the members, allowing minimum deviation 0.25 point on a scale of 10 points For example, the discussion report of a group had the evaluation results for five students as follows: A, B, B, C, C The evaluation score of the teacher for the group project was 7.0 The group project was ranked the third in twenty groups by voting of group evaluation (the highest ranking team receives 1 point, the third place receives 0.8 points) As results, the average score

of members was 7.8 Therefore, the members received C had 7.6 points; B received 7.85 points, and A had 8.0 points respectively

In the experimental course, Project No.1 was an exercise in which groups learned about the common framework for developing web applications recently The groups had the task of building an instruction video of that framework In addition to the submission of the project through the LMS system, these videos were required to be uploaded to YouTube, then the number of viewers and likes were considered as a group ranking basis

With project No.2, groups were required to implement a project of building a small-scale software; the reports were applications installed on the server so that members of the class could use and evaluate Project No.3 asked groups to develop more advanced features of the project No.2 and to use workshop in Fig.2of the LMS to implement the mutual evaluation

Fig 2 Student ’s work is assessed by k students who are randomly selected in the class

2.4 Constructing documentation wiki and peer-assessment

Teacher’s assistants evaluate assignments and at the same time each student’s work is assessed by other k students who are randomly selected in the class by the criteria made

by the teachers Thus, each student will evaluate a minimum of k members in the class (k < = n) and be also evaluated by other k members The process of randomly selecting members to evaluate is carried out by the Moodle workshop as showed in Fig.2

In this course, students were required to prepare materials in the form of a wiki, learning the framework for the current development of the popular web Moreover, implementation of the assessment with the results showed in Project 3

3 Instrumentation

At the end of the course, the students were asked to answer the survey consisted of 18 quantitative questions, using Likert scale with five values in the range 1 to 5 These observation variables aim to assess the level of satisfaction of the students participating

in the experimental course

3.1 Research model and hypotheses

The model examines the factors affecting student’s satisfaction based on research results of Kuo et al (2014), with an approach that focuses on the learners’ satisfaction when they are participating in online learning activities through the addition and considering factors related to the interaction, collaboration, assessment, technology, and results The design of content and learning activities in the BL courses plays an important role in promoting the interaction between teachers and students The results

of recent studies showed that the course with diversified design in content and form encourages students to interact with the system (Havice et al 2010) Providing activities such as online question and answer, group exercises, peer group assessment, wiki documentation construction will facilitate learners to understand the course content better (Gulbahar and Madran 2009; Lim and Yoon 2008; So and Bonk

2010) In our study, to assess the effectiveness of the new evaluation model through learner’s satisfaction as described in Fig 3, we hypothesized the model including assumptions relating to the following factors: Collaboration, Assessment, Technology

Fig 3 The model examines the factors affecting student ’s satisfaction

Collaboration: This factor considers the impact of collaborative activities among students in the implementation of learning activities on the satisfaction of learners The deployment of group learning in courses at higher level of education is essential Work

in a team helps students to promote their skills: discussion, collaboration, scheduling, assigning works, communication Besides, learners’ satisfaction also is reflected in the ability of easy support and efficient tools for online teamwork environment

H1: Using the effective collaborative learning activities in group brings satisfaction

to the participants and vice versa

Assessment: One of the important steps of the learning process is to assess the learner The true assessment of learners at the end of a course affects their satisfaction Evaluation of test activities is appropriate for course content and the ability of the students The deployed evaluation forms also affect the promotion of learner partici-pation, encouraging students to learn How to combine assessment forms such as self-assessment, peer group self-assessment, peer evaluation in BL courses in order to make them become more efficient and exciting for students is a matter of concern

H2: Student’s satisfaction with a course that uses a variety of assessment forms to promote, encourage learners to participate in learning activities and vice versa Techonology:Technology is one of the important factors for effective deployment of

BL courses Many of the learning activities are deployed based on the supporting tools Smooth learning process during the entire course depends on the communications infrastructure: the student will not be satisfied with the course if the offline status occurs frequently For those students who need supports in learning process, technical staffs are required

H3: Learning activities environment which was easily, quickly, reliably deployed, brings satisfaction and vice versa

In order to know whether the evaluation of the working results of each students in the group in stages was reflected exactly or not (H1), we observed the variables: Self-assessment of the student about his contribution for group work (SA), level of agreement with the evaluation of the team members (GPA), degree of agreement with the assessment of the teacher for the group (TA), level of agreement with assessments

of other colleagues in the class for the results of group members (OSPA), level of student’s satisfaction with the scores of group exercises (RPA), degree of agreement with the assessment (TPA)

To assess the impact of used peer assessment forms (H2), we explored these factors from students: The level of interest in the course using a combination of evaluation forms (LCPA), in the used form of the assessment, knowing the level of satisfaction with the assessment (personal assignment for peer assessment - PPA, group assignment for peer assessment - GSPA, Wiki assignment for peer assessment -WPA) The degree

of the evaluation for other members (AOS), whether the evaluation form promoted student’s group work or not (GW), whether the combination of the evaluation forms accurately reflected learning results of students or not (ER) Satisfaction level of