AN INVESTIGATION INTO THE INFLUENCE OF FORMATIVE ASSESSMENT ON STUDENTS’ SATISFACTION TOWARDS ONLINE COURSES Le Thi Hoang Ha, Nguyen Phuong Vy (Faculty of Quality Management, VNU University of Education) Thanh Anh Minh (VNU University of Education) Abstract This study was conducted to explore formative assessment practice in a number of online courses of undergraduate programs at a university in Hanoi during a social distance period due to the Covid 19 pandemic The study’s focus was on assessmen[.]
Trang 1ON STUDENTS’ SATISFACTION TOWARDS ONLINE COURSES
Le Thi Hoang Ha, Nguyen Phuong Vy
(Faculty of Quality Management, VNU University of Education)
Thanh Anh Minh
(VNU University of Education)
Abstract: This study was conducted to explore formative assessment practice in a number of online courses
of undergraduate programs at a university in Hanoi during a social distance period due to the Covid-19 pandemic The study’s focus was on assessment strategies that had been said to be effective in facilitating and improving students’ engagement in learning and the authenticity of the learning A quantitative method was used to analyze survey data of 286 respondents who had experienced a period of three months (February to April 2020) as online students Research findings showed that the practice of formative assessment strategies had been available in online courses, possibly be categorized into “facilitating engagement” and “improving authenticity” strategies, which did have positive impacts on students’ satisfaction.
Keywords: assessment strategy, formative assessment, online course, engagement, authenticity.
1 INTRODUCTION
Nowadays, online learning has become an inevitable trend as a result of science and technology developments and the increase in the need for lifelong learning An online course is understood as the learning organized and implemented in an online platform through a learning management system (LMS) In online classes, teaching activities can occur in real-time through a system that supports meetings/webinars (virtual classroom) Online courses allow students to proactively make and implement learning plans according to their own needs, abilities, and conditions However, the lack of face-to-face instruction makes some challenges to build the teacher-student and student-student interactions That also leads to the problem of promoting learning motivation, the active participation of the student Assessment activities bring many advantages to students’ learning process Assessment results give teachers information about the student’s achievement at some particular periods and help them collect data about their student’s learning and thinking activities Therefore, they can make an appropriate and effective teaching plan Nitko & Brookhart (2014) showed that in a classroom, the teacher needs to make a lot of important decisions such as what to teach, how to teach, how to evaluate students’ learning outcomes All such decisions must be made based on high-quality information, which the teacher only has through high-quality assessments (page 24)
With the various purposes, assessment should not only be conducted by teachers with the tests to score students’ learning outcomes but also become regular, flexible learning activities with the proactive participation of both teachers and students Hence, classroom assessment needs to be a part of the learning process, vigorously promote the role of assessment for learning and assessment as learning instead of focusing on assessment of learning (Earl, 2007)
Trang 2Because assessment plays such a crucial role in teaching process, the online classroom assessment should confirm the role of promoting students’ learning motivations, engagement, and proactive learning This study was conducted at a university in Hanoi, where online learning has been applied for almost one semester because of the Covid19 pandemic This university’s training method, which had been operated in blended learning mode previously, was then shifted to online, applied to all undergraduate courses, using MOODLE – a platform for learning management system in combination with virtual classrooms on Zoom software The study aims to explore the current landscape of applying formative assessment techniques in online classes and the relationship between the use of these techniques and student satisfaction Quantitative methods were used in this study A survey questionnaire was sent to collect data from students The questionnaire collected information on how often formative assessment techniques had been applied and how satisfied they were with the online courses
2 A LITERATURE REVIEW
In 1971, Bloom, Hastings, and Madaus formally categorized “assessing students’ learning activity” into two categories: summative assessment and formative assessment A summative assessment appears
at the end of a teaching cycle/stage, used for summary purposes such as classifying students, determining students’ development, or the teaching’s effectiveness The authors emphasized that the summative assessment time is at the end of a particular learning stage In contrast, the formative assessment is highlighted by the “just-in-time” characteristics It can happen at any time from curriculum development
to learning practice and aims to improve the training process (Guskey, 2005) Moreover, at the first time Bloom et al explained deeply and distinguished “summative assessment “ - “ formative assessment’’, they emphasized some striking characteristics of formative assessment For example, it helped find evidence relevant and meaningful to teaching and learning, improve academic performance by giving formative feedback, and minimize negative impacts of assessment by keeping negative judgement limited Thus, the focus of formative assessment lies in collecting and using assessment information as evidence of learning for giving feedback to students Discussing as building a theory for formative assessment, Black & Wiliam (2009) mentioned three important questions that William & Thompson (2007) had set out in the teaching and learning process They are “Where am I now?” (i.e determining the level of the student at present); Where am I going? (identifying and communicating learning goals to students); “How do I close the gap”? (help students choose the most appropriate strategy to reach their goals) Thanks to the continuous formative assessment techniques used during the teaching process, teachers can answer the above questions: it is possible to determine the current level of individual student’s competence, learning demand, learning skills, thereby providing appropriate guidance to help learners adjust and develop The two authors conclude that, with proper formative assessment strategies used, learning becomes an evolved loop of knowledge and skills Koka et al (2017) showed that formative assessment must be added in higher education programs by reducing formal lectures, where students are passively involved in the learning process However, formative assessment should be organized purposefully with self-reflection and communication activities Those are all critical features to make the lesson meaningful and help students understand deeply
William & Leahy (2016) still used the 5-strategy model proposed by (Leahy et al., 2005) to describe practical classroom formative techniques The five well-known strategies are:
1 Clarifying and sharing learning intentions and criteria for success
2 Engineering effective classroom discussions and other learning tasks that elicit evidence of student understanding
Trang 33 Providing feedback that moves learners forward.
4 Activating students as the owners of their own learning
5 Activating students as instructional resources for one another
(p 11)
Each of them has proved to be effective in online training, too Firstly, the goals and criteria sharing strategy is a practical approach to enhancing student engagement in learning Formative assessment
is not just an assessment, it offers opportunities to promote the interaction of teachers – students and the participation and engagement of students with the learning process Sharing and discussing with students to establish learning goals or success criteria gives students a sense of mastery their own learning activities (Black & William, 2009) When students take responsibility for their learning, they can nurture self-directed learning skills that have been widely agreed to be one of the essential skills needed for lifelong learning
The second strategy emphasized the power of feedback Hattie & Timperley (2007) though not exclusive to the online teaching environment, confirm that feedback based on learning goals will motivate students to develop effective learning strategies such as setting goals, plan personal learning, self-monitor and assess the learning process Feedback, in this situation, should be understood as techniques of responding formatively to students’ learning, based on information of learning collected during a assessment process
The last two strategies specifically deal with self and peer assessment When joining in self-assessment or peer-self-assessment, “learners might reflect their experience of having done what they’ve done, having presented what they’ve presented, having moderated what they’ve facilitated online, and
so on” (Conrad & Openo, 2018) Topping (2017) value, or quality of a product or performance of other equal-status learners, then learn further by giving elaborated feedback and discussing their judgements with peers to achieve a negotiated agreed outcome.” It is organized in elementary (primary indicated that when students join in peer assessment, both evaluators and receivers can benefit because, in that process, students can learn from other’s mistakes and understand criteria and standards very well Furthermore, learners’ engagement in self-assessment tasks and online tasks have the same feature: they should do it
at their own pace and time, so they need to make a plan by themselves to complete it (Conrad & Openo, 2018)
Currently, the shift in higher education’s goal is leading to the change in the focus and types of assessment Educational goals now centered on individualization and personal development (what does it mean to be alive and human?), cultivating informed and active citizens, developing intrinsically valuable knowledge, and serving society through the public interest have been limited The perceived purpose of educational attainment has since narrowed to serving the community through economic development Wall et al., (2014) defined assessment as “a set of activities that seeks to gather systematic evidence
to determine the worth and value of things in higher education” (p 6), including the examination of student learning They strongly argued that assessment “serves an emerging market-focused university” (p 6) Besides the change of assessment focus, the assessment types need to change to adapt well to new requirements and demands The fact that students were expected to perform a profession at their university graduation properly drove the assessment toward authenticity That is a form of assessment
where students can provide direct evidence of applying their learning (Goff et al., 2015) They have to apply their learned skills, knowledge, values, and attitudes in “the performance context of the intended
Trang 4discipline” (Goff et al., p 13) This would help build students an ability to simulate real-world problems
in their profession or future work
Online learning is widely believed to be created for adult education Adults have appropriate characteristics and competencies for online learning Besides a variety of advantages such as flexibility, diverse alternatives for students, online learning still raises the problem about the quality of assessment (Conrad & Openo, 2018) Assessment should focus much more on students’ learning than other training modes in online courses because lecturers have no other way to observe students It is ‘an activity assigned
by the professor that yields comprehensive information for analyzing, discussing, and judging a learner’s performance on valued abilities and skills’ (Huba & Freed, 2000, p 12) It is used for formative purposes with some missions like “multiple drafts of written work in which faculty provide constructive and progressive feedback; oral presentations by students; peer assessment; team projects that produce a joint product related to specific learning outcomes, and service-learning assignments that require interactions with individuals, the community or business/industry” (Webber, 2012, p203) When conducting those activities, the instructors have to give feedback promptly to students, along with students need to engage
in a learning process to collaborate, peer-assess, self reflection… that provide more evidence to prove students’ development However, to ensure the quality of evidence, the formative assessment activities must focus on their authenticity and engagement (Conrad & Openo, 2018)
In summary, formative assessment is a necessary component of training, whatever mode of training
is In online courses, to ensure the role of enhancing students’ learning, formative assessment should focus on two purposes: facilitating students’ engagement in learning and improving the authenticity of learning tasks Formative assessment has been empirically studied for investigating its influence on students’ achievements This research explored the influence of the two above focuses of online formative assessment
on students’ satisfaction with the courses
3 RESEARCH QUESTIONS AND HYPOTHESIS
The study aims to answer the research question: How do online formative assessment strategies
of authenticity improvement and student engagement facilitation influence online learning students’ satisfaction?
Hypothesis: Both strategies of authenticity improvement and student engagement facilitation
positively influence students’ satisfaction towards online courses
4 METHOD
4.1 Research design
The study uses quantitative methods with data collected from a student survey questionnaire
described in the Research instrument below The survey was conducted in late May 2020, which is the
last week of a semester At that time, the university has switched to entirely online training for three months (February-April 2020) due to the effects of COVID19
Statistical Package for the Social Sciences (SPSS) was used in this for data analysis Data was analyzed with a stepwise multiple regression method to measure the influence of formative assessment strategies on students’ satisfaction
The study data included 286 responses, of which female students accounted for 82 percent Gender bias in the survey respondents is available as a popular characteristic of pre-service teacher education programs in Vietnam Of the 286 participants, 49% were third-year students, while the first-year
Trang 5accounted for 25%, and the rest were divided equally by the second-year and fourth-year (Table 1)
Table 1 Participants’ Demographic Information
*Note Frequencies are the number of students
4.2 Research instrument (the questionnaire)
A student questionnaire of assessment activities in online learning is built, consisting of 2 main parts The first main part has 20 items describing the activities and indicators of formative assessment that should appear in online courses A 5-level Likert scale (1 – Very rarely or never, 2 – Rarely, 3 – Occasionally, 4 – Frequently, 5 – Very often) was used for students to choose how often they meet the actions or indicators relating to formative assessment in their online courses The items were designed based on the theoretical framework for online learning presented in the literature review, consisting of
two constructs: the first comes with activities that can facilitate students’ engagement in the course, and the second are those that can improve the authenticity of assessment
Twelve variables were defined in the second main part to measure students’ satisfaction with online courses This part focus on typical aspects of online training, such as the communication and interaction among instructors and students, the availability of learning resources and supports, the LMS itself, time allocation for online lectures, assessment and teamwork in general, and instructors’ feedback The scale used the 5-level Likert, too (i.e 1 – Very unsatisfied, 2 – Unsatisfied, 3 – Neutral,
4 – Satisfied, 5 – Very satisfied)
5 DATA ANALYSIS AND RESULTS
5.1 Test of factor analysis and measure scale reliability
EFA (Explore Factor Analysis) was conducted into two phases The first phase dealt with formative assessment variables, which had been defined into two factors: engagement facilitation and authenticity improvement The second phase was devoted to students’ satisfaction variables
In the first phase, EFA with principal components analysis and Varimax rotation was used The KMO value equaled 939 (≥ 5), and the significance was 000 (≤ 05), indicating that Bartlett’s test was statistically significant, indicating that the data is suitable for EFA (Keiser, 1974) (Tabachnick & Fidell, 2007)
Trang 6Table 2: KMO and Bartlett’s Test
Kaiser-Meyer-Olkin Measure of Sampling Adequacy .939
Bartlett’s Test of Sphericity Approx Chi-Square 2754.190
After Varimax rotation with Kaiser normalization, 17 items related to formative assessment
techniques were categorized into two factors Factor 1 (named FA1) consisted of 10 variables representing a formative assessment strategy that was expected to be able to improve the authenticity
of the assessment Factor 2 (FA2) was made up of 7 variables, expressing a scale that could measure the possibility of formative assessment to facilitate students’ engagement in online learning Three variables
(F8, F12, and F14) were eliminated from the two scales due to their factor loading not reaching 5 (F12) or the difference in the factor loadings of items was less than 3 (F8 and F14) (Hair et al., 2014) Cronbach’s alpha values of engagement facilitation and authenticity improvement were 84 and 92, respectively, proving the scales adequately reliable for the measure
Table 3: Rotated Component Matrixa (Formative assessment factors)
Items AuthenticityFA1 EngagementFA2
F18: Students are instructed to create a portfolio by gathering learning products
F16: Students have the opportunity and enough time to complete their
products based on the feedback of instructors and classmates to improve their
F10: The instructor promptly responds to student comments and questions on
F20: In the course, rubrics are used to assess students’ performance 729
F9: When you have an inquiry, you use the forum on the LMS system to ask for
F17: At some stages in the learning process, such as the end of the chapter/
lesson or the middle semester , students have the opportunity to review their
learned issues such as mindmap, reflection, survey about the learning process,
or self-assessing their level of achieving learning outcomes
.710
F19: In general, through each assessment, you know what you need to do to
F15: Students are instructed/asked to self-assess their products/assignments
F11: Instructor feedback on the LMS helps you understand the problem
F13: Students are informed about the assessment criteria of essays or projects (if any) 601
Trang 7F7: Students are given the opportunity to comment, assess the exercise/product of
other classmates or groups (In the virtual classroom or on the LMS system). .717
F4: In the virtual classroom, the instructor asks questions and calls students to
F5: In the virtual classroom, the instructor uses many quick assessment
activities such as short tests, surveys, self-reflection about the problem students
F3: The specific assessment plan (clear regulations on the form and time of
F1: Course outline/syllabus is published on the LMS system or emailed to
F2: The learning objectives/learning outcomes of each lesson/ chapter are
F6: When performing an assessment in the virtual classroom, the instructor
In the second phase, a similar analysis was conducted with 12 satisfaction variables The KMO value was 939, which is ≥ 5, and the sig was 000 (≤ 05) The Bartlett’s test was statistically significant (Table 4), the data is suitable for EFA
Table 4: KMO and Bartlett’s Test (Satisfaction)
Kaiser-Meyer-Olkin Measure of Sampling Adequacy .939
Bartlett’s Test of Sphericity
Approx Chi-Square 2116.559
Varimax rotation with Kaiser normalization analysis applied to 12 items of student satisfaction brought all the items into 1 group, as having attentively been developed Cronbach’s alpha value of this scale is 934, with all corrected item-total correlations are greater than 3 (Table 5)
Table 5: Rotated Component Matrixa (Satisfaction)
S10: Discussions operated in LMS forums were useful to learning 808 S2: Online learning strategies and schedules were explained to students clearly .787 S12: You feel comfortable when learning online because you received companionship and support from
S11: Feedback you received from instructors on LMS is on time and useful 766
Trang 8S6: Instructors’ use of technology to support teaching was useful .765 S1: Learning objectives were clarified and helpful in monitoring your learning .762 S5: The operation of online lectures helped engage students in learning 760 S3: Students were provided/introduced with learning resources that met the course’s demand .745 S4: Time allocated to online lectures was relevant to the course’s contents 722 S9: Team members in the course worked hard, effectively, and were fairly assessed 964
5.2 Regression analysis to identify the influence of formative assessment strategies on online learning students
5.2.1 Hypothesis test of the model
The adjusted R square was 523 meant that the two independent variables could explain 52.3% the dependent variable of satisfaction (Table 6) The statistical F value is equal to 157.041 with sig value
is 000 (< 05) (Table 7) could help to conclude that the multiple regression was relevant to the data The Durbin-Watson statistic was used to detect serial correlation The value of 1.815 (approximate to 2) indicated that there was no autocorrelation problem having existed in the data (Gujarati, 2003 as cited
in Sun et al., 2008)
Table 6: Model Summaryb
Model R R Square Adjusted R Square Std Error of the Estimate Durbin-Watson
a Predictors: (Constant), FA2, FA1
b Dependent Variable: Satisfaction
Table 7: ANOVAa
1
Regression 57.789 2 28.895 157.041 000 b
a Dependent Variable: Satisfaction
b Predictors: (Constant), FA2, FA1
Trang 95.2.2 Assessment of multicollinearity
The research used the Variance Inflation Factor (VIF) to assess the existence of multicollinearity between the two independent variables in the model (i.e engagement facilitation and authenticity improvement) The VIF value of 1.984 (less than 2) indicated that no multicollinearity existed between the two variables (Table 8) Also, with sig values were 000 (less than 05), all three variables are significant in the standardized regression model (Allison, 2012)
Table 8: Coefficientsa
Model Unstandardized Coefficients Standardized Coefficients t Sig.
Tolerance
Collinearity Sta-tistics
1
a Dependent Variable: Satisfaction
5.2.3 Regression analysis results
Table 8 shows regression coefficients of independent variables in the model The β (standardized coefficients) of both FA1 and FA2 were positive, indicating that both the facilitation of engagement and improvement of authenticity strategies had a positive influence on the students’ satisfaction The constant B equaled to 855; the β between FA1 and S was 498; the β between FA2 and S was 283; and sig values of all three coefficients were less than 05 Symbolically, the model can be presented through
a prediction formula as: S = 855 + 498*FA1 + 283*FA2
Table 9 is the descriptive statistics of the three studied variables Although FA1 (consisting of techniques helping to improve the course’ authenticity) had a greater influence on students’ satisfaction towards online courses than FA2 (those facilitating students’ engagement in learning), the earlier actually appeared less frequently in online courses than the latter (FA1 mean = 3.99 with SD = 69 compared to FA2 mean = 4.31 with SD = 51) In general, students were satisfied with the online courses (S mean = 4.14 with SD = 62)
Table 9: Descriptive Statistics
Trang 106 CONCLUSION AND RECOMMENDATIONS
Although formative assessment is widely known with five strategies proposed by a group of UK authors, applying to online training, it should appear to be techniques that could help engage students more in the learning process and improve the authenticity of a course Based on these arguments, a model
of online formative assessment with two categories of assessment techniques (coded as FA1 and FA2) has been developed to measure the frequent use of formative assessment in online courses The research findings revealed that both two groups of assessment techniques contributed to making students satisfied with their online courses In the surveyed courses, FA1 (authenticity improvement techniques) was used less frequently than FA2 (learning engagement facilitation), as evaluated by students However, the impact that FA1 makes on students’ satisfaction is greater than FA2 These results lead to some recommendations that: (i) more assessment techniques for enhancing the authentic aspects of online courses should be planned and implemented by lecturers; (ii) further qualitative research should be conducted to explore what kind of techniques is suitable and beneficial for individual course, and how the techniques should be implemented to maximize students’ benefits from them
References
1 Allison, P (2012) Logistic regression using SAS: Theory and application SAS Institute.
2 Black, P., & Wiliam, D (2009) Developing the theory of formative assessment Educational
Assessment, Evaluation and Accountability, 21(1), 5–31 https://doi.org/10.1007/s11092-008-9068-5
3 Conrad, D., & Openo, J (2018) Assessment Strategies for Online Learning: Engagement and
Authenticity Athabasca University Press https://doi.org/10.15215/aupress/9781771992329.01
4 Earl, L (2007) Assessment - A Powerful Lever for Learning Brock Education Journal, 16(1),
1–15 https://doi.org/10.26522/brocked.v16i1.29
5 Goff, L., Potter, M K., Pierre, E., Carey, T., Gullage, A., & Gaste, V (2015) Learning Outcomes
Assessment A Practitioner’s Handbook Higher Education Quality Council of Ontario https://
scholar.uwindsor.ca/ctlreports.https://scholar.uwindsor.ca/ctlreports/6
6 Guskey, T (2005) Formative Classroom Assessment and Benjamin S Bloom: Theory, Research,
and Implications Paper Presented at the Annual Meeting of the American Educational Research
Association (Montreal, Canada, April 11-15, 2005) https://eric.ed.gov/?id=ED490412
7 Hair, J F., Black, W C., Babin, B J., & Anderson, R E (2014) Multivariate Data Analysis
In Pearson New International Edition (7th ed.) Pearson Education Limited https://doi.
org/10.4324/9781351269360
8 Hattie, J., & Timperley, H (2007) The power of feedback Review of Educational Research,
77(1), 81–112 https://doi.org/10.3102/003465430298487
9 Huba, M E., & Freed, J E (2000) Learner-Centered Assessment on College Campuses: Shifting
the Focus from Teaching to Learning Allyn & Bacon.
10 Keiser, H F (1974) An index of factorial simplicity Psychometrika, 39(1), 31–36.