A Model to Forecast Learning Outcomes for Students in Blended Learning Courses Based On Learning Analytics44977

A Model to Forecast Learning Outcomes for Students in Blended Learning Courses Based On Learning Analytics Viet Anh Nguyen VNU University of Engineering and Technology E3, 144 Xuan Thu

A Model to Forecast Learning Outcomes for Students in Blended Learning Courses Based On Learning Analytics

Viet Anh Nguyen

VNU University of Engineering and

Technology

E3, 144 Xuan Thuy, Cau Giay, Hanoi

vietanh@vnu.edu.vn

Quang Bach Nguyen VNU University of Engineering and

Technology E3, 144 Xuan Thuy, Cau Giay, Hanoi 14020652@vnu.edu.vn

Vuong Thinh Nguyen Vietnam Maritime University

484 LachTray - NgoQuyen - Haiphong thinhnv@vimaru.edu.vn

ABSTRACT

One of the difficulties experienced by online learners is the lack

of regular supervision as well as the need to provide instructions

to support the learning process more effectively The analysis of

the learning data in the online courses is not only becoming

increasingly important in forecasting learning outcomes but also

providing effective instructional strategies for learners to help

them get the best results In this paper, we propose a forecast

learning outcomes model based on learners’ interaction with

online learning systems by providing learning analytics dashboard

for both learners and teachers to monitor and orient online

learners This approach is mainly based on some machine learning

and data mining techniques This research aims to answer two

research questions: (1) Is it possible to accurately predict learners'

learning outcomes based on their interactive activities? (2) How to

monitor and guide learners in an effective online learning

environment? To answer these two questions, our model has been

developed and tested by learners participating in the Moodle LMS

system The results show that 75% of students have outcomes

close to the predicted results with an accuracy of over 50% These

positive results, though done on a small scale, can also be

considered as suggestions for studies of using learning analytics in

predicting learning outcomes of learners through learning

activities

CCS Concepts

Applied computing➝ Education➝ E-learning

Keywords

Learning analytics, learning activities, learning outcomes,

predictive modeling, forecast model

1 INTRODUCTION

Online learning support systems are being increasingly invested

and developed There are not only huge numbers of online

learning websites such as the edx.com, coursera.com but also a lot

of online learning platforms such as Moodle, and Blackboard

Besides the advantages, online learning also has many defects

Learners often do not have information about the learning

outcomes in each stage during the learning process to adjust the

learning method properly In addition, a learner does not receive timely support from his instructor These things can make an online training process less effective Teachers also do not have accurate information about learners such as the interaction level, the comprehension level Therefore, they can not track the learning process of their learners to suggest and orient in time to help learners get the desired result Moreover, teachers also lack the feedback of learners on the content of the lesson and the learning activities do not have the content adjustments to help students get the highest results

To improve the quality of online learning platforms, we need additional tools to help teachers and learners track the learning process, interactions, suggestions, and feedback This means that

it offers recommendations for effective learning These tools must address the shortcomings of the online learning system: suggesting and regulating the learner interaction with the course is essential and should be prompted in time to help learners adjust the interaction level as well as learning methods to achieve the desired results An instructor also needs to have an overview of the interaction of trainees with the course to remind them Furthermore, teachers can rely on learners' interactions to adjust the way they give lessons to learners? In addition, to forecast the learning outcomes of learners in each stage of the course plays a more and more important role Based on the forecast results, a learner can imagine the final result and change his own study to achieve better results in the next time In this study, we focus on the predictive model over time to forecast student outcomes The basis for making predictions about future learner outcomes is based on the data of learners’ interaction with the system, along with the learning outcomes of other students who have taken the previous courses and learning analytics techniques Learning Analytics (LA) is a topic of increasing interest in the educational research communities [1] Learning data is generated from more and more learning activities but most of them are not being used effectively LA creates tools for data mining and creates models that help improve online learning systems LA includes five steps: data collection, reporting, prediction, action, and improvement [2]

LA provides not only useful information about the learning process, the relationship between learners but also specific actions, suggestions, and warnings for learners to improve their learning outcomes

LA will provide the ability to collect and analyze data from various sources to provide information on what works and what does not involve teaching and learning LA allows individuals and organizations to understand the learning and make informed decisions about the allocation of resources and interventions needed to promote learner success [3]

In this paper, we propose a model to predict learning outcomes over time and alert learners based on learning analytics We use machine learning algorithms to evaluate these interactions to

Permission to make digital or hard copies of all or part of this work for

personal or classroom use is granted without fee provided that copies are

not made or distributed for profit or commercial advantage and that copies

bear this notice and the full citation on the first page Copyrights for

components of this work owned by others than ACM must be honored

Abstracting with credit is permitted To copy otherwise, or republish, to

post on servers or to redistribute to lists, requires prior specific permission

and/or a fee Request permissions from Permissions@acm.org

ICSET 2018, August 13–15, 2018, Taipei, Taiwan

© 2018 Association for Computing Machinery

ACM ISBN 978-1-4503-6528-4/18/08…$15.00

DOI: https://doi.org/10.1145/3268808.3268827

predict learners' learning outcomes and help teachers give

suggestions and warnings to learners about learning progress In

addition, this study also focuses on clarifying the influence of

interactions between learners and online learning systems on

learning outcomes of learners To better understand the research

results, the following contents of the paper are presented in the

following structure:

- The next section presents some researches related to predicting

and forecasting results based on learning analytics

- In section 3, we describe our forecasting results model based on

the interactive activities of the learners

- Some results of the research and model testing are presented in

section 4, some of the exchanges and implications are presented in

section 5, and finally the conclusions

2 LITERATURE REVIEW

To deal with the variety of collected data, LA has many

techniques to achieve the users’ goals The most used technique is

building a predictive model which has the ability to forecast

uncertain future events [4] The predictive model helps learners to

understand more deeply about their learning process and provides

solutions for learners to do next Administrators can use

forecasting models to predict learning trends, forecast the number

of courses, which are needed for learners [5] There are some

commonly used algorithms for forecasting models such as Linear

Regression, Logistic [6], Decision Trees [7], Nạve Bayes

Classifiers… But, which algorithm is the most appropriate?

Comparing algorithms can help to find out the optimal algorithm

for forecasting models I Babic's [3] compared three algorithms:

SVM, Classification Tree (CART) and Artificial Neural Networks

(RBF model) The classification models were divided into two

groups according to SDI (Self Determination Index) with four

attributes V1 - assign view, V2 - forum view discussion, V3 -

questionnaire view, V4 - resource view The RBF model achieved

the highest result of 76.92% so that the RBF model is the most

suitable model for dynamic learning prediction Along with the

development of machine learning algorithms, forecasting models

are increasingly developed, so the reliability of the model is

increasing

Content which is taught by teachers becomes more and more

diversified These data contain a lot of information about teaching

content, interactive content… They can help the analysts know the

trader's trends, the quality of the material that helps to improve

teaching and learning LA definite content analysis technique is an

automated method to test, evaluate, index, filter, present and

visualize different types of digital learning content, regardless of

the manufacturer such as instructors, students with the goal of

understanding learning, improving practice and educational

research [9]

Recently, many methods and algorithms have been proposed and

successfully implemented to predict learning outcomes Bovo [10]

and colleagues created a tool to get data from Moodle and they

used clustering algorithms to split learner's data into groups The

research has shown that using clustering algorithms can yield

clusters, but it is difficult to distinguish between clusters, which

are not enough to describe data Consequently, it is not enough to

use the clustering algorithm to predict student learning outcomes

We need to describe the differences between clusters to evaluate

and make predictions Another model to predict is the use of

logistic regression suggested by Barber and Sharkey [6] In this

study, the user data, which were the input data of Logistic

regression, were divided into two groups of low- risk and

high-risk multiple times with different properties The discrimination rate of the two groups was about 98% With Logistic regression algorithm, we can predict the possibility of passing or slipping and providing timely information to the instructor, but learners still can not know theirs ability to learn in order to adjust their learning To classify learners, we can use many other classification algorithms such as Decision Tree, Naive Bayes,

In his research, Romero [11] compared different classification algorithms to evaluate the performance of algorithms when predicting learning outcomes through user interaction on the online learning system The author concludes: It is not the only algorithm that achieves the best classification accuracy in all cases The accuracy of algorithms is not high, only about 65% so it is difficult to predict by classification algorithms

It is also the performance forecast of the learner, the study of Ali Daud and colleagues [12] provide another perspective The authors argue that family financial factors have a great impact on job performance and excitement in learning, so the paper focuses

on attributes related to spending and financial ability of the family Research uses classification algorithms to predict the attributes such as academic performance, family income, family assets, student personal information, and family expenditures SVM algorithm is the most suitable algorithm for the given problem Some studies rely on prior time to predict the outcome of the present Thakur and his colleagues [13] used the Autoregressive model, which is based on the idea of present values based on past values, to analyze time series to explore and describe relative variability in classes The authors proposed predictive models based on Logistic Regression and Feed-Forward Neural Network The Feed-Forward Neural Network algorithm gave better results, especially week 1, averaged over the three weeks is 84% Through their observations, they found that the homework attribute most influenced the results and added the "post" attribute in the forum When adding this attribute, the predicted accuracy rate was higher with both algorithms The study by Althaf Hussain Basha and colleagues [14] suggested that the percentage of students passing the course was dependent on the factors of the previous school year by using the formula OP = Cr * SVOP * 1.05 where OP is the number of students, Cr is the number of students involved, SVOP is the ratio of student passing/number of students taking part of the previous year The predicted number of students passing the course with the error rate was 7-8%

Zacharis [15] used a linear regression model to calculate the student's point value The author found that the four attributes with the highest correlation coefficient with the point to create regression model were reading and posting messages, Content creation contribution, quiz effort and file viewed Next, the author used Binary Logistic Regression to predict students who were at risk of slipping and achieve an accuracy of 81.3%

From the above studies, we can see some problems that need to be resolved: (1) Algorithms cannot accurately predict the learning outcomes of learners (2) Using only a single algorithm, it is difficult to predict the learning outcomes of learners (3) Data selection, pre-processing (cleaning, standardization, etc.) is very important, having a great impact on the results achieved

3 METHODS 3.1 Participants

Participating in this course prediction model is 290 students in the second and third year of the 4-year course, taking part in three online courses built on Moodle All students are students in information technology, studying at the Faculty of Information

Technology of the VNU-University of Engineering and

Technology, so they are proficient in using computers and using

online courses Each course is implemented in 15 weeks in the

form of blended-learning, specifically 81 students enrolled in the

first course, 150 students enrolled in the second course and 59

students enrolled in the third course

3.2 The forecast learning outcomes model

Our learning outcomes model focuses on some of the key

elements of student engagement when participating in online

learning The number of views/posts of learners on the materials

is used to measure the degree of interactivity, frequency of use,

level of focus on the materials, at what times To assess the

interaction of learners with one another and learners with teachers,

we need to get more information about the course discussions A

number of viewings/postings of learners in course forums can

help us in this case That is typical for the interaction, which is

viewed, answered in the course topics Submission deadline:

Assessment of homework assignments is also quite important

Learners who have mastered the lesson will usually do the

assignment and submit the papers early and on time, however

those who do not understand the lesson often take more time to

complete so they will be able to submit later or not to submit

These can be considered as a basis for forecasting results

The results are forecasted at some point throughout the course for

each learner to help the learner adjust his / her learning behavior

In addition, the overall student performance forecast is provided

for the teachers Particularly, they are provided with an overview

of student involvement as well as their results in some phases

throughout the course

The prediction model, described in Figure 1, consists of the

following main components:

•Log Analysis Module: The module will retrieve data directly

from the tables in the Moodle database, which will collect some

data on the number of views/post of student, course information,

student information, submitted assignments, progress of

assignments

Learner

Forecast Module

Notification Module

Teacher

Log Analysis Module

UET Analyitics Model

Figure 1 The forecast learning outcomes model

•Predictive Module: This module can be viewed as the core

component of the predictive model We use machine learning

techniques to analyze the collected information, classify learners

and predict outcomes for learners

In this model, we apply some specific techniques including classifying, clustering, regression to forecast learning outcomes for learners The basic steps for making the resulting forecast are described in Figure 2

This can be considered as a two-stage process (1) A regression model is used to predict the amount of learners' interaction to the time they need to anticipate learning outcomes, for example at week 7th or week 15th of the semester (2) Classify learners into classes labeled A, B, C, D, F

Log Database

Predict value of interaction factors

Data Trainning

Clustering

Filtering &

Labeling

Classifying Forecast Results

Figure 2 The predictive module workflow

To perform the first stage (1), we collected data from 3rd, 6th, 7th,

10th, 13th, 15th weeks, then created regression models for 3rd to 6th week and 6th to 7th week, 7th to 10th week, 10th to 13th week, and

13th to 15th week We found that the data interaction from 3rd week

to 7th week and the 15th week had much lower accuracy than predicted between successive weeks In the second phase (2), our study data have not been labeled, if manual labeling will take a lot

of time and effort so we use the K-means algorithm to clone the original data and label each cluster automatically The K-means algorithm is used because after checking the Silhouette coefficient, this algorithm has a larger value, as described in Table 1, as the clusters are dense and distinct

Table 1 Compare the Shihoutte correlation coefficient result

of some algorithms

Algorithm KMeans Birch agglomerative clustering Silhouette

cofficient 0.6808 0.6442 0.644205

In this step, we grouped learners into three clusters: a cluster of learners who were at risk of not achieving the desired learning outcomes, a high-impact cluster and a medium-scale cluster We distinguish them based on the cluster average Labeling student who is at risk of not completing the course is F With the remaining two clusters we continue to use a clustering algorithm

to split each cluster into two small clusters With the highly interactive cluster, we divide it into two small clusters labeled A and B With the intermediate cluster, we divide it into two clusters labeled C and D In both cases, the labels are based on point’s average of the cluster After assigning labels to all clusters, we continue to filter the actual point values that do not match the

scores for A from 8-10, B from 6.5-7.9, C from 5.5-6.5, D from 4

to 5.5, F <4.0

Once the learning data has been labeled, we include this data in

the classification algorithm to create a forecasting model for the

new data

•Notification Module: Provides a dashboard with graphs that

display visual information to each student for an overview of their

interaction with the course The module also offers predicted

grade along with real grades so that learners can compare and

improve learning methods For the instructor, the module provides

information about students' predictions, warns students who are at

high risk of failure, and supports the integration of existing

Message modules Pre-installed on LMS systems to issue alert

messages to learners

3.3 Instrumentation

When attending the course, in addition to attending classroom

lectures in the traditional way, students must access the LMS

system to carry out the learning activities required by the

instructor Student interactions are stored in the log system

including login, logout, view, post, comment, and submission

information To complete the course, students must undertake

some online learning activities The prediction model is integrated

as a plug-in to the Moodle LMS system, to provide information in

the form of a dashboard for students and faculty to monitor the

results of their learning during the 3rd, 6th, 10th, and 13th week

4 RESULTS

In this test, we used data collected from 284 students' systems for

training data for clustering, classifying and forecasting results

Test data from 59 students have completed the course and have

specific learning outcomes In this section, we present the results

of our research to answer our research questions

4.1 Results of an experiment to forecast

learning outcomes

4.1.1 Forecasting interactions in different stages

Linear regression results for a number of interactive activities

indicate that when forecasting interactions for the following

weeks from the 3rd week (for example, 7th week and 15th week),

the correlation was not high This means that predicting the level

of interactivities of the learners for the coming weeks based on

their interaction data in the first few weeks is not exactly accurate

However, when there is a lot of interactive data (starting from the

sixth week of the semester), the predicted interaction for the next

weeks is based on a strong correlation between the attributes

Table 2 The correlation coefficient in predicted interactive

activities over the weeks

R2 MSE MAE

W3 – W7

Linear

Regression

forum view 0.578 2.077 0.993

forum post 0.935 0.0002 0.005

success submission 0.496 0.0059 0.058

W3 – W15

Linear

Regression forum view 0.291 1.629 0.860

forum post -2.66 0.002 0.032

success submission 0.144 0.031 0.161

W6 – W7 Linear Regression

forum view 0.809 0.9395 0.269

forum post 0.977 9.305 0.002

success submission 0.939 0.0007 0.025

W6 – W15 Linear Regression

forum view 0.408 1.358 0.700

forum post -0.633 0.0010 0.027

success submission 0.468 0.0193 0.129

W10 – W15 Linear Regression

forum view 0.637 0.833 0.550

forum post 0.340 0.00043 0.0187

success submission 0.652 0.012 0.0968

W13 – W15 Linear Regression

forum view 0.834 0.379 0.3517

forum post 0.551 0.0002 0.0076

success submission 0.912 0.0032 0.0462

As shown in the table above, the prediction of attribute values is quite difficult, especially in the first weeks of the course As for the last weeks, the value of the prediction attributes is more accurate

For each student, we forecast the level of interaction with the system through learning activities starting from the third week to the last week of the semester The graphs in Figures x, y, z predicts a degree of interaction with the system of a student (user 34) for learning activities from a 3rd week up to a 15th week

Figure 3 Compare the forecast and actual results of the view

activity interactions

Figure 4 Compare the forecast and actual results of the

post-activity interactions

Figure 5 Compare the forecast and actual results of the

assignment activity

4.1.2 Results of forecasting the learning outcomes

Predictive models of learning outcomes are applied at 3rd, 6th,

10th and 13th weeks respectively The predicted results are

compared with the actual scores of each student (the prediction

point is denoted by the blue circle, the actual point is denoted by

the black circle) At timelines, we forecast the midterm and final

results respectively The graphs in Figure 6, 7 and 8 illustrate the

predicted score versus the real score of 59 students starting at

week 3 taken as test data In this test, we evaluate the degree of

deviation of the predicted score and the student's actual score at

the midterm and the end of the semester We assume that, if there

is a difference between the predicted and actual scores within the

range of a 5-point scale (A-F), it is acceptable For example, if the

student's predicted score is F, the actual result is D, this is

considered to be the correct prediction

The test results show that if the student's academic

performance is predicted from the first week of the course (3rd

week), the results will be quite wrong Figure 6 shows the actual

mid-term and mid-term forecasts deviated at the F to A, B scores

(more than 70% of students have deviations of more than two

point scales) If the student's final grade results are forecasted

from the 3rd week (as shown in Figure 7), the same result is

expected from the previous weeks The final-grade predicted from

the 13th week (Figure 8) shows that the value of the actual point

has a smaller difference than the predicted results (more than 50%

of the students have deviations in one scale)

Figure 6 The predicted mid-term grade result (7 th week) from

3 rd week

Figure 7 The predicted final grade result from 3 rd week

Figure 8 The predicted final grade result from 13 th week

4.2 Results of an experiment to Moodle LMS

We deployed the model as a plug-in integrated with the Moodle

system Once installed, the plugin will also provide a dashboard

for both students and teachers

Figure 9 Dashboard for student

Figure 9 depicts information for students, in which students are

provided with information about their interaction with the learning

system from the beginning to the moment of viewing the report

together with the learning result that predicted at the viewing time

For lecturers, the function provides not only a statistical report on

the data interactions of all students attending the course but also

the results of each member's forecast This information is also

provided for the faculty to send instructive notices as well as

alerts to students with poor academic results

5 DISCUSSIONS

A course consists of a mid-term test (about 7th weeks) and the

final exam at the end of the course (15th week) to assess the

learning outcomes of the learner Therefore, we need to forecast

results for students in the weeks prior to week 7th and week 15th to

help learners make timely adjustments to achieve the desired

results There are many ways to evaluate a learner's performance,

but the most common is based on grades, which are usually

grouped into different groups A, B, C, D, F When the model

provides the exact value of a score such as 6.5 or 7.0, it loses the

generality of the score and is likely to confuse the learner The

predicted results we inform the learner are the points which are

labeled

One problem is how can label a new data set without training data

set With a new forecasting system, it is not possible to assess

which class of learners In machine learning, clustering algorithms

are possible to solve this problem When using clustering

algorithms, we have not yet defined which groups have a point level, but we can divide the data into the groups that have the most similarity After data clustering, we need to re-filter the data according to the actual score achieved, thereby assigning labels to each data stream for training data sets for the classification algorithms

In addition, we rely on the values of the 7th and 15th week grades

to categorize and evaluate, but we need to forecast the results before these weeks, so how the values of the attributes of the 7th week and 15th week can be derived from the previous week to forecast results for learners If learners regularly access to use the materials on the system, the learner's interactions will increase gradually over the week Therefore, we can use linear regression algorithms to derive the predictions of the learning process We retrieve data from the week of forecasting, predict the values of the properties at 7th week and 15th week, sort and give forecast results We can predict the interaction of students in the first weeks of the course to help the learners prepare better

Throughout the experiment, the interactions between learners from 3rd week to 15th week are often inaccurate, but the interactions between the two weeks are quite accurate (e.g., between 3rd week and 6th week) Therefore, intervals should not be predicted at 3rd week with 15th week but forecasts are in order of

3rd week to 6th week… 13th week to 15th week With such sequential work, the predicted accuracy rate is 70%

One of the difficulties in implementing our forecasting model is that it depends entirely on the degree to which the learner interacts with the system The predictive model will be limited if the learner does not perform the learning activities required by the instructor, or is lacking regular interaction with the system Courses which cover a variety of activities and spread over weeks will have a sufficiently large number of interactions to produce accurate prediction results and vice versa In addition, in order to

be able to form an accurate forecasting system, the course needs

to be implemented over and over again to form a training data file large enough for clustering, stratification, and regression

At present, our model is based only on online activities that do not cover all the activities of the course In this way, the forecast results are not high accuracy, we will improve in the future In addition, because we have considered only online activities, the data collected only include the interaction of students with the system such as views, posts, comments, submissions and may not reflect well the forecast results

6 CONCLUSIONS

Learning outcomes of learners are becoming increasingly important, especially in the online learning model Forecasting results are a good reference for students and teachers Based on the forecast results, the students can visualize their final results at the end of the course if they continue to study at such so that they can adjust themselves to better academic results Moreover, based

on the forecast results, the teachers can receive a comprehensive picture of the learning outcomes of the students, on which the teachers can issue alerts to the students and may tailor the teaching activities accordingly

In this study, we proposed a model using learning analytics techniques that predict learning outcomes over time as students participate in online learning activities based on interactive data of students with the online learning system The empirical results show that the forecast of students at risk of failing to meet the requirements of the course is high (84%), although the results of the study results by the score are not effective (accuracy of 50%)

However, this model can be considered as a solution to predict

student learning outcomes, combined with other student

assessment criteria in the online learning

In the next study, to improve the efficiency of forecasting, we will

focus on model improvement and identifying factors that have a

direct or indirect impact on the learner's performance as well as

improving forecasting algorithms

7 REFERENCES

[1] Chiappe, A 2017 Learning Analytics in 21st century

education: a review Ensaio: aval.pol.públ.Educ 25, 97

(June.2017), 971–991.DOI=

http://DX.DOI.ORG/10.1590/S0104-40362017002501211

[2] Campbell, J P., DeBlois, P B., and Oblinger, D G.2007

Academic Analytics Educ Rev 42 (October,2007), 40–57

[3] Brown, M 2011 Learning Analytics: The Coming Third

Wave Educ Learn Initiat Br., (April,2011), 1–4

[4] Brooks, C and Thompson, C 2010 Chapter 5: Predictive

Modelling in Teaching and Learning Handbook of Learning

Analytics., 61–68

[5] ECAR.2015.The Predictive Learning Analytics Revolution:

Leveraging Learning Data for Student Success ECAR Work

Gr Pap., 1–23

[6] Barber,R and Sharkey, M.2012.Course Correction: Using

Analytics to Predict Course Success Proc Second Int Conf

Learn Anal Knowl - LAK ’12, (May,2012), 259–262

[7] Xing, W., Gou, R., Petakovic,E., and Goggins, S 2015

Participation-based student final performance prediction

model through interpretable Genetic Programming:

Integrating learning analytics, educational data mining and

theory Comput Human Behav 47,168–181

[8] Babić,I Đ.2017 Machine learning methods in predicting the

student academic motivation Croat Oper Res Rev 8, 443–

461

[9] Ali, L., Hatala, M., Gašević, D., and Jovanović, J.2012.A

qualitative evaluation of evolution of a learning analytics tool

Comput Educ 58,1, 470–489

[10] Bovo,A., Sanchez, S., Heguy, O., and Duthen, Y 2013 Clustering moodle data as a tool for profiling students In

Proceedings of 2nd International Conference on E-Learning and E-Technologies in Education, 121–126

[11] Romero, C., Espejo, P., Zafra, G A., Romero, J R., and Ventura, S.2013 Web usage mining for predicting final

marks of students that use Moodle courses Comput Appl

Eng Educ 21, 1, 135–146

[12] Ali Daud,J S A., Naif Radi Aljohani, Rabeeh Ayaz Abbasi, Miltiadis D Lytras, Farhat Abbas.2017.Predicting student

performance using Advanced Learning Analytics Pakistan

Saudi Arab C, 415–421

[13] Thakur, G S., Olama, M M., Mcnair, A W., Sukumar,S R and Studham, S 2014 Towards Adaptive Educational Assessments: Predicting Student Performance using Temporal Stability and Data Analytics in Learning

Management Systems in Proceedings 20th ACM SIGKDD

Conference on Knowledge Discovery and Data Mining

[14] Althaf Hussain Basha, S., Ramesh Kumar, Y., Govardhan, A., Zaheer Ahmed, M., and Professor, A.2012 Predicting Student Academic Performance Using Temporal Association

Mining * 1 Int J Inf Sci Educ 2,1, 2231–1262

[15] Zacharis, N Z 2015 A multivariate approach to predicting student outcomes in web-enabled blended learning courses

Internet High Educ 27, 44–53