Relationship Between Learner Engagement and Performance based on The User Behavioral Factors in E-Learning Environments

Relationship Between Learner Engagement and Performance based on The User Behavioral Factors in E-Learning Environments

Ranasinghe N. K

Department of Software Engineering

Sri Lanka Institute of Information Technology, Sri Lanka

Dr. Lakmal Rupasinghe

Department of Cyber Security

Sri Lanka Institute of Information Technology, Sri Lanka

Abstract:- Adaptive E-learning systems make a personalized e- learning environment according to the identified learning styles. The traditional way of detecting those learning styles is mainly based on asking learners to fill in a questionnaire or survey related to the learning process. This approach has some limitations including the excess time taken to fill the forms and the lack of self-awareness of learners regarding their own interests in learning. Furthermore, those traditional approaches lead learners to get bored when asking them to fill out some questions. Even if they are willingly fill out the questionnaire, they may give some irrelevant answers without knowing the importance of filling out those forms. Therefore, the results might be inaccurate by processing those irrelevant answers.

Besides that, traditional approaches assume that learning styles cannot be changed over the time. But learning styles are not fixed. They are dynamic because, learning styles are derived from the learners preferences and those preferences can be evolved due to some psychological and physical reasons. This paper explains an experimental study which has been taken place with the use a real-world dataset which is obtained from a e-learning platform in order to identify the relationship between the engagement of learners and the performance based on learners behavioral factors in e-learning environments. Three machine learning algorithms Decision tree, Random Forest and Support Vector Machine have been used to obtain the best predictive model.

Keywords Adaptive E-Learning, Web usage mining, learning style, Academic Performance Prediction, Learning Analytics

1. INTRODUCTION

   Nearly, the advancement of technologies has helped to change teacher centered education to student centered education. Students have different types of ways to engage in learning. Either they may be aware of their learning patterns or they may not be aware of their learning patterns. Therefore, every student should have an ability to identify their learning styles to make a good learning plan. Besides that, knowing the learning styles help students to observe, process, distinguish and comprehend learning materials. Many technological mechanisms have been introduced to detect and categorize the learning styles of students. The aim of many adaptive e-learning platforms is to identify the learning patterns of each student and adapt the deliverables accordingly.

   Traditional way of getting aware of the learning behaviors of students basically depends on questionnaires and surveys. Although, filling in questionnaires have many drawbacks.

   Firstly, it may be an exhaustive task for students which also wastes time. Secondly, many learners do not know about their learning styles and they might not know the importance of those surveys and questionnaires, that may encourage them to give incorrect answers. Therefore, the results received through these kinds of traditional ways can be incorrect and might not exactly represent the learning patterns of learners.

   To overcome the above-mentioned limitations, many automated approaches have been proposed which expects to determine learning behaviors of each student according to their activities while they are collaborating with the e- learning platforms. These automatic approaches have many benefits over the traditional approaches. Therefore, by using automated approaches, excess time usage, which was for filling in questionnaires will be reduced, since the user behavioral patterns can be detected from the user collaboration with the system. On one side, as the learning styles, which are identified by traditional approaches are static. In other words, those learning styles do not change over the time. But there is a chance that students behavior in learning may change over the time. Therefore, students behaviors should also be detected regularly. Previous automatic approaches have used LS (Learning Styles) models to detect learning styles of students. A learning style model assists to categorize the learners based on the specific behavior patterns in learning. It consists of some special variables related to the medium of the deliverables and the type of user interaction with the learning materials. Eventually, LS models give an idea about the way of each learner prefers to learn with. There are many LS models in literature [1] [2] [3].

   Based on the literature [4] [5], the FSLSM can be recognized as the most common and the most applicable for implementing an adaptive e-learning platform. According to the FSLSM, each student can be uniquely identified based on their learning patterns. It also helps teachers, evaluators, and examiners to provide personalized learning materials and to use a personalized evaluation plan.

   This work does not depend on learning styles model as there are some identified limitations in those models according to literature. As most of the previous research were carried out only according to one aspect of measurement such as the user behavior (Learning styles) with the system, this approach emphasizes the value of user behavior analysis for predicting the learners performance. A machine learning

   model is being used to predict the results of students based on their collaboration with the e-learning platform. The results will be predicted considering some important features which have affected the most to the performance of students.

   Learning Behaviors

   The term Learning behaviors is the awareness of that each learner learns differently and each of them have their own preferred way of absorbing, processing, comprehending, and preserving their knowledge. For example, some learners can understand a lesson by doing it practically by themselves while some learners just gain the knowledge by listening to the lesson and the instructions. This logic leads to a point that each learner is created equally and differently. Because everyone has a natural preferred way of learning, more research have gone into discovering the different learning styles. The individual learning patterns depend on many factors. According to [6], learning styles may vary according to cognitive, psychological, and environmental factors as well as learners prior experience in learning. The understanding of different learning behaviors of learners is important for educators to understand the different learners, so that they can focus upon implementing a learner-friendly learning environment.

2. LITERATURE REVIEW

   Various approaches using machine learning techniques have been proposed to automatically identify learners learning patterns. Most of them are data-driven approaches which use data mining algorithms on existing learners behaviors to construct a model. Then that model has been used to detect the learning pattern of a new learner. Based on [7], the BN classifier which is known as Bayesian Network can be recognized as one of the most common classifiers to presume the learning patterns. Authors of [8] have used Bayesian Networks to illustrate the relationship between the learner model and the features of a learning profile. To evaluate the link between above mentioned models, they have used the collaboration of Bayesian Network and the Overlay Model.

   Decsion tree is another most adopted classification algorithm which is used for automated detection of learning styles. Aijaz in [9] proposed an automatic approach for detecting learning styles of e-learners by using a decision tree as the classification algorithm. Through his solution, he has addressed the One size fits all issue of e-learning systems. Kolbs learning style theory has been used to understand learning styles from web logs of learners using data mining techniques. In [10], Pantho and Tiantong have addressed a solution to classify learning styles according to VARK [10] model of learning styles by using Decision Tree C4.5 algorithm. A questionnaire had been used to collect data from 1205 people. The gathered data were then analyzed using the Decision Tree C4.5 algorithm.

   Addition to that, NW which is known as Neural Networks, is also one of the often used algorithms in automatic learning styles detection. [11] Proposed an approach to detect and track students learning patterns in order to provide recommendations of relevant learning resources. That model was based on Neural Networks and Felder Silverman

   Learning Style model. Hmedna also introduced an approach in his work [11], to detect the learning styles of students in learning management system. That work was incorporated with fuzzy cognitive maps FCMs (Fuzzy c-means) which is a colloboration of Fuzzy logic and neural networks.

   Apart from the above-mentioned algorithms, KNN (K- nearest neighbors) is often used to automatically identify the learning. [12] proposed a solution to detect learning styles by improving K-nearest neighbor (KNN) classification and combining it with genetic algorithms (GA).

   Similarly, Mohomed Alloghani [13], carried out a research to develop predictive models for detection of learning styles based on the decision tree, neural network and naïve Bayes algorithms. But the dataset was previously collected not from an e-learning system.

   Based on the literature, most of the works which were carried out to detect learning styles, relied on a specific learning style model. Consequently, most of the systems used FSLSM considering that there are 8 learning styles. In this component, the relationship between user engagement and performance is analyzed according to various user behavioral factors. This approach considers two aspects of learner attributes such as: learner attention towards the learning materials and the learner real intention in the learning platform.

   The learning style of each learner is identified using data mining techniques and machine learning algorithms on the e-learning platforms log file. Consequently, by analyzing the log file each learners performance level can be obtained. Higher the engagement level of learners, Greater the grades they obtain for the final examination is a theory. This model is based on that theory.

3. METHODOLGY

   Data and Sources

   The Open University is the one of the largest universities in the United Kingdom. VLE delivers the learning materials for each module (subject), and each users activities per day are recorded in the VLE logs. Those learning materials in that VLE are delivered through HTML, PDFs and videos. Activity types can be identified as dataplus, forumng, glossary, oucollaborate, oucontent, resource, subpage, homepage, and URL and demographic data of users.

   Since the teacher-student interactions in the VLE are limited when the number of users is getting increased, it leads to a difficulty for the academic instructors to support to all users real-time. The research focused on this approach involves reducing the dropout rate of students and optimize their performance in final examinations by predicting the results beforehand.

   Predictive Models

   Three machine learning models which were used to obtain the most suitable predictive model for predicting learner performance are Random Forest Classifier, Decision Tree classifier and Support Vector Machine classifier. Those algorithms were chosen because they are applicable for both categorical and domain attributes. Data preprocessing is an important step in developing a model. The selected dataset which has been used to develop the model has 797 learner records. It also contains 12 variables including the target variable: Result. Three of them are categorical while rest of the variables are numeric. The target variable consists 4 values: Pass, Fail, Distinction and Withdrawn. Data preprocessing consists the steps of cleansing the dataset to obtain more accuracy in a predictive model. It also includes feature extraction and taking care of the categorical data. From 12 variables 5 variables were chosen based on their impact to the target variable. The chosen variables are homepage, url, subpage foruming and he_level. The variables he_level and final_results are categorical. Therefore, those variables have been encoded.

   Decision Tree Classifier

   Decision tree algorithm is based on entropy to classify the variables which is known as a mathematical technique. Decision tree has a tree-like design. Rectangles and ovals

   represent the internal nodes and leaves, respectively. Those nodes represent the features of dataset while each leaf has a class according to the dataset. This classifier has been developed by deciding which variables best split the input variables. As mentioned earlier, for this case, theory of information gain was used which means the node that has minimum entropy (Highest Information Gain) is used as split node.

   Random Forest Classifier

   Random forest classifier is an ensemble machine learning algorithm. It consists of individual decision trees which work together as an ensemble. Each of those individual decision trees gives a prediction and among those predictions most voted value becomes the output of the model.

   Support Vector Machine

   The concept behind the support vector machine is to find out the optimal hyperplane in an N-dimensional space which classifies the data points. In other words, hyperplanes represent as decision boundaries.

4. RESULTS AND DISCUSSION Evaluation Measures

   1. Decision Tree Accuracy by Class

      C. SVM Accuracy by Class

   2. Random Forest Accuracy by Class

5. CONCLUSION

Considering the descriptive statistics, Random forest Classifier was chosen to develop prediction model.

REFERENCE

1. L. K. S. R. M Felder, "Learning and Teaching styles," vol. 78, pp. 674-681, 1988.

2. H. K Laschinger, M. W Boss, Learning Styles of nursing students and career choices, 1984.

3. Biggs, "Study Process Questionnaire Manual Student Approaches to Learning and Studying," Australian Education Research, 1987,

   p. 53

4. F. L. J. Kuljis, "A Comparison of Learning Style Theories on the Suitability for elearning," in Web technologies, Applications and services, 2005.

5. K. S. Graf, "Advanced adaptivity in learning management sysems by considering learning styles," in IEEE/WIC/ACM International Conference on Web Intelligent Agent Technology – Workshops, 2009.

6. J. Keefe, "Learning style: An overview. NASSP's Student learning styles: Diagnosing and proscribing programs," pp. 1-17, 1979.

7. F. J, M. A and A. A, "Automatic detection of learning styles," in

   State of the art, Artificial Intelligence, 2015, pp. 157-186.

8. M. T. A, R. N. C and M. K, "The determination of learning styles in a learner model using the combination of bayesian network and the overlay model," 2019.

9. Aijaz Ahmed Kalhoro, "Detection of E-Learners' Learning Styles: An Automatic Approach using Decision Tree," International Journal of Computer Science and System Analysis, vol. 14, no. 8, pp. 420-425, 2016.

10. O. Pantho and M. Tiantong, "Using decision tree C4.5 algorithm to predict VARK learning styles," International Journal of the Computer, the Internet and Management, vol. 24, no. 2, pp. 58-63, 2016.

11. B. HMEDNA, A. E. Mezouary, O. Baz and D. Mammass, "Identifying and tracking learning styles in MOOCs: A neural networks approach," International Journal of Innovation and Applied Studies, vol. 19, no. 2, pp. 267-275, 2017.

12. C.-H. Chiu, C.-P. Chua, W.-Y. Kao and Y.-C. Chang, "A learning style classification mechanism for e-learning," Computers & Education, vol. 53, no. 2, pp. 273-285, 2009.

13. M. Alloghani, D. Al-Jumeily, A. Hussain, J. Aljaaf and E. Petrov, "Application of Machine Learning on Student Data for the Appraisal of Academic Performance," in 11th International Conference on Developments in eSystems Engineering , 2018.

14. J. E. Villaverde, D. Godoy and A. Amandi, "Learning styles' recognition in e-learning environments with feed-forward neural networks," J. Comp. Assisted Learning, vol. 22, pp. 197-206, 2006.

15. O. E. AISSAOUI, Y. E. A. EL MADANI, L. OUGHDIR and Y. E. ALLIOUI, "Combining supervised and unsupervised machine learning algorithms to predict the learners learning styles," Procedia Computer Science, vol. 148, pp. 87-86, 2019.