A Comparative and Explainable Machine Learning Approach for Student Dropout Prediction

A Comparative and Explainable Machine Learning Approach for Student Dropout Prediction

Polepally Navaneetha

Assistant Professor Dept. of AI & ML Vignan Institute of Technology and Science Deshmukhi, Pochampally, TG, India – 508284

Ijjana Prasanth Kumar

Assistant Professor Dept. of AI & ML Vignan Institute of Technology and Science Deshmukhi, Pochampally, TG, India – 508284

Dr. M. Sreeramu

Assistant Professor Dept. of MBA Vignan Institute of Technology and Science Deshmukhi, Pochampally, TG, India – 508284

Abstract – Now a days student dropout has become major challenge for educational institutions. This is impacting on

students academic performance , their career development. It is affecting educational reputation also. Existing student dropout prediction approaches primarily focus on achieving high predic- tion accuracy using individual machine learning models, while providing limited model comparison and insufcient explanation of the factors inuencing dropout decisions This study proposes a comparative and explainable machine learning approach for predicting student dropout. A publicly available dataset con- taining academic, behavioral, and socio-economic attributes was utilized for analysis. Data preprocessing techniques, including missing value imputation and categorical encoding, were applied to ensure data quality. The dataset was divided into training and testing subsets in an 80:20 ratio for model evaluation. Several machine learning algorithms, such as Logistic Regression, Decision Tree, Random Forest, and Support Vector Machine, were implemented and compared.

Index TermsMachine Learning, Student Dropout Prediction, Random Forest, Classication

1. INTRODUCTION

   Student dropout is a signicant challenge faced by edu- cational institutions, affecting both student success and insti- tutional performance. Early identication of students at risk of dropping out is essential to implement timely intervention strategies. However, traditional methods often fail to accurately predict dropout due to the complexity of inuencing factors such as academic performance, behavioral patterns, and socio- economic conditions.

   With the advancement of machine learning techniques, it has become possible to analyze large volumes of educational data and identify patterns that can help predict student dropout. Machine learning models can effectively handle complex relationships among multiple variables and provide better prediction accuracy compared to traditional approaches.

   This study aims to develop a comparative and explainable machine learning approach for student dropout prediction. The objective is to evaluate multiple machine learning models and identify key factors inuencing student dropout, thereby assisting educational institutions in making informed decisions and reducing dropout rates. Education analytics has emerged as an important research eld due to the rapid growth of digital educational platforms and student-related data. Educational

   institutions continuously generate data related to attendance, academic performance, assessments, and student behavior. Proper analysis of this data can help institutions improve learning outcomes, identify at-risk students, and enhance in- stitutional decision-making processes.

   Student dropout is considered a major educational and social problem worldwide. High dropout rates negatively affect insti- tutional reputation, nancial stability, and overall educational quality. In addition, students who discontinue their education may face difculties in career growth and employment oppor- tunities. Therefore, accurate prediction of student dropout has become an essential task in modern educational systems.

   Despite the availability of educational data, predicting stu- dent dropout remains challenging because dropout behavior is inuenced by multiple interconnected factors. Academic performance alone is not sufcient for accurate prediction, as behavioral and socio-economic conditions also play a critical role. Traditional statistical methods often struggle to handle these complex relationships effectively.

   Machine learning techniques provide efcient solutions for analyzing educational datasets and extracting hidden patterns from the data. Algorithms such as Logistic Regression, Deci- sion Tree, Random Forest, and Support Vector Machine have shown promising results in predictive analytics applications. These techniques can improve prediction accuracy and support early intervention strategies for educational institutions.

   In recent years, explainable articial intelligence has gained signicant attention in educational analytics. In addition to accurate predictions, it is important to understand the factors inuencing student dropout. Explainable machine learning models improve transparency and help educators identify the key attributes contributing to student dropout risk.

   1. Problem Statement

      Educational institutions face signicant challenges due to increasing student dropout rates, which negatively affect aca- demic performance, institutional reputation, and student career development. Identifying students at risk of dropping out at an early stage is difcult because multiple academic, behavioral, and socio-economic factors inuence student performance. Traditional approaches are often unable to effectively analyze

      these complex relationships. Therefore, there is a need for an efcient machine learning-based system capable of accurately predicting student dropout and identifying the key factors contributing to it.

   2. Objectives

      • To preprocess and analyze student-related data.

      • To implement multiple machine learning models for dropout prediction.

      • To compare the performance of different algorithms.

      • To identify important factors inuencing student dropout.

2. LITERATURE REVIEW

   Several studies have explored the use of machine learning techniques for predicting student dropout. Commonly used algorithms include Logistic Regression, Decision Trees, Ran- dom Forest, and Support Vector Machines. These models have demonstrated the ability to identify students at risk based on academic and behavioral data. Some research has reported high prediction accuracy using advanced techniques such as ensemble methods and neural networks. However, many of these studies focus primarily on improving accuracy and do not provide sufcient insights into the underlying factors contributing to student dropout. This lack of interpretabil- ity limits the practical application of these models in real- world educational settings. Recent works have emphasized the importance of explainable machine learning approaches to better understand the factors inuencing student dropout. Identifying key attributes such as academic performance, atten- dance, and socio-economic conditions is essential for effective intervention strategies. Despite these advancements, there is still a need for a comprehensive study that not only compares multiple machine learning models but also provides clear insights into the factors affecting student dropout. Therefore, this study adopts a comparative and explainable machine learning approach to address these challenges. In recent years, educational data mining and learning analytics have become important research areas due to the increasing availability of student-related data. Educatinal institutions generate large amounts of data related to student attendance, assessments, learning behavior, and academic performance. Analyzing this data can help institutions identify at-risk students and improve retention strategies.

   Logistic Regression is widely used in student prediction systems because of its simplicity and interpretability. It per- forms effectively when relationships among variables are linear and provides probability-based predictions. However, its performance may decrease when handling highly complex and non-linear datasets.

   Decision Tree algorithms are popular because they provide clear visualization and easy interpretation of decision-making processes. These models can identify important attributes affecting student dropout. However, Decision Trees are prone to overtting and may produce unstable results when trained on noisy data.

   Random Forest, an ensemble learning algorithm, overcomes some limitations of Decision Trees by combining multiple trees to improve prediction accuracy and reduce overt- ting. Due to its robustness and capability to handle high- dimensional datasets, Random Forest has become one of the most effective techniques for educational prediction tasks.

   Support Vector Machine is another widely used classica- tion algorithm in educational analytics. SVM performs well in high-dimensional spaces and can create optimal decision boundaries for classication tasks. Nevertheless, the algo- rithm requires careful parameter tuning and feature scaling to achieve better performance.

   Recent studies have also emphasized the importance of explainable articial intelligence in educational prediction systems. In addition to achieving high prediction accuracy, researchers focus on understanding the factors inuencing student dropout predictions. Explainable machine learning techniques improve transparency and help educators identify critical factors such as attendance, GPA, and study behavior that contribute to dropout risk.

3. 3.Proposed work:

   Fig. 1. Student Framework Methodology

   The proposed work depicted in Figure 1 entails a com- prehensive analytics-driven framework for student dropout prediction and educational risk analysis aimed at extracting meaningful insights to improve student retention, academic performance, and institutional decision-making.

   3.1.1 Data Collection and Preprocessing Data collection and preprocessing are the foundational stages of the proposed student dropout analytics framework, enabling the extraction of meaningful insights from large educational datasets.

   1. Descriptive and Diagnostic Analytics With the prepro- cessed dataset, descriptive and diagnostic analytics techniques were applied to gain detailed insights into student dropout- related factors.

   2. Predictive Analytics

      Building upon the insights obtained from descriptive and diagnostic analytics, robust machine learning models were developed using predictive analytics techniques to forecast student dropout probability.

   3. Prescriptive Analytics Finally, leveraging the ndings obtained from descriptive, diagnostic, and predictive analytics,

   prescriptive analytics techniques were utilized to formulate effective intervention strategies for reducing student dropout risk and improving academic performance.

   Through this holistic approach, the proposed work aims to reduce Students dropouts rates with data-driven insights and actionable recommendations derived from advanced analytics techniques.

   1. 4. Dataset Analysis

      Fig. 2. Dataset Overview

      The effectiveness of the proposed framework is demon- strated using a student dropout dataset downloaded from Kag- gle.com, illustrating how educational institutions can leverage analytics techniques for data-driven decision-making and early student intervention. The dataset comprises several important academic, behavioral, and socioeconomic attributes related to student performance and dropouts, as depicted in Figure .

   2. 5. Data Preprocessing

      Data preprocessing was performed to improve the quality and reliability of the dataset before model development. The preprocessing steps included missing value imputation, dupli- cate removal, and categorical data encoding.

      Initially, missing values were identied and handled using multiple imputation techniques. Among the evaluated meth- ods, the K-Nearest Neighbors (KNN) imputation technique produced comparatively better predictive performance, indicat- ing its effectiveness in preserving the underlying data distri- bution. The comparative analysis of the imputation techniques is presented in Figure 3.

      After selecting the most suitable imputation method, du- plicate records were identied and removed to eliminate re- dundant observations. Subsequently, categorical features were transformed into numerical representations using label encod- ing, making the dataset suitable for machine learning algo- rithms. The preprocessing workow and the resulting cleaned dataset are illustrated in Figure ??.

      Fig. 3. Comparative Analysis of Missing Value Imputation Techniques

      Fig. 4. Dataset after missing value imputation and duplicate removal.

      After handling missing values and performing deduplica- tion, label encoding is applied in order to convert the categor- ical into numerical ones to make the data ready to efciently apply predictive analytics. The dataset after label encoding is depicted in Figure.

      Fig. 5. Dataset after label encoding.

   3. 6. Descriptive Analytics and Diagnostic Analytics: Com- prehensive Insight Discovery

      Fig. 6. Architecture of the proposed descriptive and diagnostic analytics framework.

      Descriptive Analytics: Further analysis aims to identify the major academic and behavioral factors inuencing student dropout and low GPA. In particular, the study investigates the impact of attendance rate, study hours per day, stress level, and academic risk on student performance and dropout probability.

   4. Diagnostic Analytics:

      GPA, Attendance Rate, and Stress Index are identied as the three most inuential features through the correlation analysis depicted in Figure 11, and the target attribute Dropout is found to be highly correlated with GPA. The analysis indicates that students with lower GPA values exhibit a signicantly higher probability of dropout compared to students with better academic performance.

      Several machine learning models were implemented for comparative analysis, including Logistic Regression, Decision

      Fig. 11. Correlation heatmap of the student dropout dataset.

      Fig. 7. Student Dropout Distribution.

      Fig. 8. Distribution of Grade Point Average (GPA) for dropout and non- dropout students.

      Fig. 9. Attendance rate distribution for dropout and non-dropout students.

      Fig. 10. Study hours per day distribution for dropout and non-dropout students.

      Fig. 12. GPA risk analysis for dropout and non-dropout students.

      Fig. 13. Attendance risk analysis and Acedemics risk analysis for dropout and non-dropout students.

      Fig. 14. Stress level and backlog risk analysis for dropout and non dropout students.

      Tree, Random Forest, and Support Vector Machine. These algorithms were selected because they represent different learning approaches and are widely used in classication and predictive analytics tasks.

      Logistic Regression is a supervised machine learning algo- rithm used for binary classication problems. It predicts the probability of a target class using the sigmoid function and performs effectively when relationships among variables are linear. Logistic Regression is widely preferred because of its simplicity, computational efciency, and interpretability.

      The Logistic Regression model is mathematically repre- sente as:

      1

      P (Y = 1) = 1+ e(b0 +b1 x) (1)

      where P (Y = 1) represents the probability of the positive class, b0 represents the bias term, and b1x represents the weighted input features.

      Decision Tree is a non-linear supervised learning algorithm that represents decisions in a tree-like structure. The model splits the dataset into multiple branches based on feature conditions and predicts the target class using decision rules. Decision Trees are easy to interpret and visualize but may suffer from overtting when trained on complex datasets.

      Random Forest is an ensemble learning algorithm that com- bines multiple Decision Trees to improve prediction accuracy and reduce overtting. Each tree is trained using random subsets of the dataset and features. The nal prediction is obtained through majority voting among the trees. Due to its robustness and ability to handle high-dimensional data, Random Forest often provides better performance compared to individual decision trees.

      Support Vector Machine (SVM) is a supervised classi- cation algorithm that identies the optimal hyperplane for separating different classes. SVM performs effectively in high- dimensional feature spaces and is widely used for classication tasks. However, the performance of SVM depends on appro- priate feature scaling and parameter tuning.

      These machine learning models were trained using the pre- pared training dataset and evaluated using the testing dataset to compare their prediction accuracy and effectiveness for student dropout prediction.

   5. Model Evaluation

   The performance of the machine learning models was evalu- ated using accuracy as the primary evaluation metric. Accuracy measures the proportion of correctly predicted instances to the total number of instances in the dataset. The evaluation process was carried out using the testing dataset to ensure an unbiased assessment of model performance on unseen data.

   Accuracy is mathematically represented as:

   TABLE I Model Comparison

   Model	Accuracy
   Random Forest	81.2%
   Logistic Regression	81.0%
   Support Vector Machine	77.25%
   Decision Tree	74.1%

4. Results and Discussion

   1. Model Performance Analysis

      The performance of different machine learning models was evaluated using accuracy as the metric. Among the models, Random Forest achieved the highest accuracy of 81.2

      The comparable performance of Logistic Regression and Random Forest suggests that both linear and non-linear rela- tionships exist in the dataset. Random Forest performs slightly better due to its ability to capture complex feature interactions and reduce overtting. The lower performance of Decision Tree may be due to its tendency to overt the training data. Support Vector Machine shows moderate performance, indicating sensitivity to feature distribution and scaling.

   2. Graphical Analysis

      Comparison of Machine Learning Models

      Fig.15 illustrates the performance comparison of different machine learning models used for student dropout prediction. Among all the models, Random Forest achieved the highest accuracy of 81.2

      TP + TN

      Accuracy =

      TP + TN + FP + FN

      (2)

   3. Feature Importance Analysis

      where TP represents True Positives, TN represents True Negatives, FP represents False Positives, and FN represents False Negatives.

      In addition to accuracy, confusion matrix analysis was also performed to evaluate the classication performance of the models. The confusion matrix provides detailed information regarding correctly and incorrectly classied instances for both dropout and non-dropout classes.

      The machine learning models were compared based on their prediction accuracy to identify the most effective algorithm for student dropout prediction. Experimental results demonstrated that Random Forest achieved the highest prediction accuracy among all implemented models, followed closely by Logistic Regression.

      Fig. 15. Feature Importance for Student Dropout Prediction

      Fig. 16 shows the importance of different features in predict- ing student dropout. Academic factors such as GPA and atten- dance have higher importance, indicating their strong inuence on student retention. Behavioral attributes such as study hours also contribute signicantly. These ndings provide useful insights for early intervention strategies.

   4. Confusion matrices Analysis

      Fig. 16. Confusion matrices of the evaluated machine learning models for student dropout prediction.

      Fig. 17. Confusion matrices of the evaluated machine learning models for student dropout prediction

      Fig. 18. Confusion matrices of the evaluated machine learning models for student dropout prediction

      Fig. 19. Confusion matrices of the evaluated machine learning models for student dropout prediction

      The confusion matrix was used to analyze the classication performance of the Random Forest model in detail. The matrix

      Fig. 20. Confusion matrices of the evaluated machine learning models for student dropout prediction

      provides information about correctly classied and misclassi- ed instances for both dropout and non-dropout classes.

      The results indicate that the Random Forest model cor- rectly classied the majority of students in both categories, demonstrating strong predictive capability. The number of false predictions was comparatively low, which indicates that the model can effectively identify students who are at risk of dropping out.

      The confusion matrix analysis also shows that the model achieved balanced classication performance without signif- icant bias toward a particular class. This is important in educational prediction systems because both dropout and non- dropout students must be identied accurately for effective intervention strategies.

   5. Comparative Discussion

      The comparative analysis of machine learning models demonstrates that ensemble learning methods provide better performance for student dropout prediction. Random Forest achieved the highest accuracy because it combines multiple decision trees and reduces the risk of overtting. The algo- rithm can effectively capture complex relationships among academic, behavioral, and socio-economic factors.

      Logistic Regression also achieved competitive performance, indicating that the dataset contains signicant linear relation- ships among the features. Due to its simplicity and inter- pretability, Logistic Regression remains a useful model for educational prediction tasks.

      Support Vector Machine achieved moderate performance and required proper feature scaling for effective classication. Although SVM performs well in high-dimensional spaces, its performance depends heavily on parameter tuning and feature distribution.

      Decision Tree produced the lowest accuracy among the eval- uated models. This may be due to overtting and sensitivity to variations in training data. Individual decision trees often fail to generalize effectively on unseen data compared to ensemble methods such as Random Forest.

   6. ROC Curve:

      ROC Curve Analysis: Figure X illustrates the ROC curves of the top-performing machine learning algorithms. Logistic Regression achieved the highest Area Under the Curve (AUC) of 0.819, demonstrating the strongest capability to distinguish

      between dropout and non-dropout students. AdaBoost and CatBoost followed closely with an AUC of 0.812, while Random Forest achieved 0.804. SVM obtained the lowest AUC (0.773) among the selected models. Since all AUC values are signicantly greater than 0.5, the evaluated models exhibit effective pedictive performance, with Logistic Regression providing the best balance between sensitivity and specicity for the student dropout prediction task.

      Fig. 21. ROC curve comparison of the top-performing machine learning models for student dropout prediction.

   7. Practical Implications

   The results of this study can help educational institutions identify students who are at risk of dropping out at an early stage. By analyzing important factors such as GPA, attendance, and study behavior, institutions can develop targeted interven- tion strategies to improve student retention.

   Machine learning-based prediction systems can support ed- ucators and administrators in making data-driven decisions. Early identication of at-risk students enables institutions to provide academic counseling, mentoring, and additional support services to improve student success rates.

   The explainable nature of the proposed approach also im- proves transparency and trust in educational analytics systems. Understanding the factors inuencing dropout prediction can help educators implement more effective policies and student support programs.

5. LIMITATIONS AND FUTURE ENHANCEMENTS

   Although the proposed machine learning approach achieved promising results for student dropout prediction, the study has certain limitations. The prediction models were trained and evaluated using a single dataset, which may limit the gener- alization capability of the models across different educational environments.

   The dataset mainly focused on academic, behavioral, and socio-economic factors. Additional attributes such as psy- chological factors, emotional well-being, family support, and extracurricular activities were not included in the analysis. Incorporating such features may further improve prediction

   performance and provide deeper insights into student dropout behavior.

   Another limitation is that the study primarily used tra- ditional machine learning algorithms. More advanced deep learning techniques and ensemble boosting methods such as XGBoost and LightGBM may provide improved prediction accuracy for larger and more complex datasets.

   Future work can focus on implementing explainable ar- ticial intelligence techniques in greater depth to improve transparency and interpretability of prediction systems. Real- time educational analytics systems can also be developed to continuously monitor student performance and provide early intervention recommendations for educational institutions.

6. CONCLUSION

   This study presented a comparative and explainable ma- chine learning approach for student dropout prediction using educational data. Multiple machine learning algorithms, in- cluding Logistic Regression, Decision Tree, Random Forest, and Support Vector Machine, were implemented and evaluated to identify the most effective model for predicting student dropout.

   The experimental results demonstrated that Random Forest achieved the highest prediction accuracy of 81.2%, followed closely by Logistic Regression with 81%. Support Vector Machine and Decision Tree showed comparatively lower per- formance. The ndings indicate that ensemble learning tech- niques are more effective in capturing complex relationships among academic, behavioral, and socio-economic factors.

   Feature importance analysis revealed that attributes such as GPA, attendance, and study behavior signicantly inuence student dropout prediction. These insights can help educational institutions identify at-risk students at an early stage and implement effective intervention strategies to improve student retention and academic success.

   The proposed approach also emphasizes explainability in machine learning-based educational analytics. In addition to achieving strong prediction performance, the study provides meaningful insights into the factors contributing to student dropout. This improves transparency and supports data-driven decision-making in educational institutions.

   Overall, the results conrm that machine learning tech- niques can play a signicant role in educational prediction sys- tems and assist institutions in reducing dropout rates through early identication and targeted support mechanisms.

7. FUTURE WORK

Future work may focus on improving predictive perfor- mance by incorporating additional student-related attributes, including socioeconomic factors, extracurricular activities, learning management system interactions, and psychological indicators. Advanced deep learning architectures, hybrid en- semble models, and explainable articial intelligence (XAI) techniques such as SHAP and LIME can also be explored to further enhance both prediction accuracy and interpretability. Additionally, deploying the proposed framework as a real-time

early warning system for educational institutions represents a promising direction for future research.

REFERENCES

1. L. Breiman, Random Forests, Machine Learning, vol. 45, no. 1, pp. 532, 2001.

2. C. Cortes and V. Vapnik, Support-Vector Networks, Machine Learning,

   vol. 20, no. 3, pp. 273297, 1995.

3. F. Pedregosa et al., Scikit-learn: Machine Learning in Python, Journal of Machine Learning Research, vol. 12, pp. 28252830, 2011.

4. J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques, 3rd ed., Morgan Kaufmann, 2012.

5. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed., Springer, 2009.

6. I. H. Witten, E. Frank, M. Hall, and C. Pal, Data Mining: Practical Machine Learning Tools and Techniques, 4th ed., Morgan Kaufmann, 2017.

7. T. Mitchell, Machine Learning, McGraw-Hill, 1997.

8. S. B. Kotsiantis, Machine Learning: A Review of Classication and Combining Techniques, Articial Intelligence Review, vol. 26, no. 3,

   pp. 159190, 2006.

9. S. Kotsiantis, K. Pierrakeas, and P. Pintelas, Predicting Students Per- formance in Distance Learning Using Machine Learning Techniques, Applied Articial Intelligence, vol. 18, no. 5, pp. 411426, 2004.

10. S. B. Kotsiantis, Use of Machine Learning Techniques for Educational Purposes: A Decision Support System for Forecasting Students Grades, Articial Intelligence Review, vol. 37, no. 4, pp. 331344, 2012.

11. F. Del Bonifro, M. Gabbrielli, L. Lisanti, and S. Zingaro, Student Dropout Prediction, Proceedings of the International Conference on Articial Intelligence in Education, Springer, 2020.

12. J. Kabathova and M. Drlik, Towards Predicting Student Dropout in University Courses Using Different Machine Learning Techniques, Applied Sciences, vol. 11, no. 7, 2021.

13. J. Berens et al., Early Detection of Students at Risk: Predicting Student Dropouts Using Administrative Student Data and Machine Learning Methods, International Journal of Intelligent Systems and Applications in Engineering, vol. 10, no. 1, 2022.

14. H. Dasi and S. Kanakala, Student Dropout Prediction Using Machine Learning Techniques, International Journal of Intelligent Systems and Applications in Engineering, vol. 10, no. 4, pp. 408414, 2022.