Design and Implementation of an Intelligent System Based on Machine Learning for Delivering Healthcare

Design and Implementation of an Intelligent System Based on Machine Learning for Delivering Healthcare

Lara Alpeer Bahdi

Web Science and Artificial Intelligence Syrian Virtual University

Al-Qamishli, Syria

Dr. Issam Salman

Web Science and Artificial Intelligence Syrian Virtual University

Damascus, Syria

Abstract – The integration of machine learning into primary healthcare offers significant potential for early diagnosis and interactive medical support. However, a major methodological challenge in medical datasets is severe class imbalance, particularly the presence of numerous rare diseases with insufficient samples for reliable statistical learning. This paper proposes an intelligent diagnostic system that addresses this challenge through a novel rare disease aggregation strategy. By conducting a sensitivity analysis, an optimal threshold of fifty samples was identified, effectively reducing the target classes from 773 to 528, thereby enabling robust stratified learning. Four machine learning models were developed and evaluated. The XGBoost model achieved the highest overall accuracy of 84.06% and demonstrated superior weighted precision. Furthermore, explainable artificial intelligence techniques, specifically feature importance analysis, confirmed that the model learns medically logical patterns, assigning high weights to primary diagnostic symptoms while utilizing rare symptoms as precise confirmatory signals. The optimized model was successfully deployed as a responsive web application using the Django framework, featuring efficient memory management. This system provides a reliable, interpretable, and scalable triage tool suitable for underserved and remote healthcare environments.

Keywords – machine learning, class imbalance, rare disease aggregation, explainable artificial intelligence, primary healthcare, XGBoost.

1. ‌INTRODUCTION

   The global burden of disease has shifted significantly, with non-communicable diseases and mental health disorders now accounting for the majority of the global disease burden [1], [2]. This epidemiologic transition, driven by aging populations and improved control of infectious diseases, presents profound challenges to healthcare systems worldwide [3], [4]. Furthermore, socioeconomic factors and health inequalities exacerbate these challenges, particularly in low- and middle- income countries where access to specialized care is limited [5], [6], [7]. A critical issue compounding this burden is poor medication adherence and diagnostic inaccuracy, which lead to suboptimal clinical outcomes and adverse drug reactions [8], [9], [10].

   To mitigate these risks, healthcare systems are increasingly turning to predictive analytics and big data [11]. The integration of artificial intelligence (AI) and machine learning

   (ML) into clinical decision support systems (CDSS) offers unprecedented capabilities for early diagnosis, personalized treatment, and proactive health monitoring [12], [13]. However, deploying ML models in real-world primary healthcare settings presents a significant methodological challenge: severe class imbalance. In many public health repositories, a small number of common diseases dominate the data, while hundreds of rare diseases are represented by only a handful of samples. This disparity makes reliable statistical learning and stratified data splitting mathematically impossible for these minority classes.

   To bridge this gap, this research proposes an intelligent primary healthcare diagnostic system capable of simulating initial medical reasoning. The main contributions of this paper are:

   • Developing a systematic threshold-based aggregation strategy to resolve extreme class imbalance, reducing 773 unique diseases to 528 statistically learnable classes.

   • Conducting a comprehensive comparative evaluation of four distinct ML algorithms on a large-scale symptom- disease dataset comprising 246,945 samples.

   • Applying explainable artificial intelligence (XAI) via feature importance analysis to validate that the model's decision-making process aligns with clinical medical logic.

   • Designing and deploying an efficient, responsive web application using the Django framework, optimized for low-resource environments.

2. ‌RELATED WORK

   1. ‌Public Health and Socioeconomic Determinants

      Foundational studies have long established the bidirectional relationship between poverty and chronic disease [14], [5]. The epidemiologic transition theory highlights how populations shift from infectious to chronic diseases as healthcare improves [3]. Addressing these shifts requires population-based strategies that target social determinants of health [15], [6], [7]. Specific burdens, such as cardiovascular diseases [16], obesity [17], and allergic diseases [18], require continuous monitoring and early intervention to prevent systemic healthcare collapse.

   2. ‌Electronic Health Records and Predictive Analytics

      The digitization of health data through Electronic Health Records (EHRs) has been a cornerstone of modern health informatics [19], [20]. Standards such as HL7, DICOM, and ISO/TC 215 ensure interoperability and data security [21], [22]. Leveraging this data, predictive analytics enables critical interventions, from real-time monitoring to diagnostic assistance and population health management [23], [12], [24]. Big data analytics promises to transform healthcare delivery by identifying hidden patterns and optimizing resource allocation [11], [13], [25].

   3. ‌Foundations of Clinical Decision Support and AI

      The evolution of CDSS relies on robust knowledge acquisition and representation [26],[27]. Traditional approaches utilized ontologies and semantic networks to structure medical knowledge [28], [29]. With the advent of ML, probabilistic reasoning and fuzzy logic have been employed to handle the inherent uncertainty and imprecision of clinical observations [30], [31], [34]. Furthermore, data mining techniques have shifted the paradigm from rule-based systems to data-driven knowledge discovery [32].

   4. ‌Recent Machine Learning Applications in Disease Prediction

   Recent studies have demonstrated the efficacy of ML in specific diagnostic tasks. For instance, compression- complexity measures have been applied to analyze and classify coronaviruses [35]. Other research has focused on predicting diseases from various symptoms using traditional ML pipelines [36], while newer approaches explore efficient genetic algorithm-based AutoML for classification and regression tasks [37]. Additionally, specialized models for heart disease prediction classification continue to show high accuracy, validating the use of ML in cardiovascular diagnostics [38]. Despite these advancements, few studies address the extreme class imbalance of rare diseases in large-scale, multi-symptom datasets, which this research aims to resolve.

3. ‌PROPOSED METHODOLOGY

   1. ‌Dataset and Initial Preprocessing

      The study utilizes the disease-symptom dataset [33], comprising 246,945 patient samples and 377 binary symptom features (0 for absent, 1 for present), mapping to 773 unique diseases. Initial data quality checks confirmed the complete absence of missing values, eliminating the need for imputation straegies. The binary nature of the data inherently negates the requirement for feature scaling.

   2. ‌Rare Disease Aggregation Strategy

      The primary methodological challenge was that 246 out of 773 diseases had fewer than 50 samples, with some having only a single instance. To resolve this, a threshold-based aggregation strategy was developed. A sensitivity analysis was conducted across five thresholds (30, 40, 50, 75, and 100 samples). For each threshold, the number of rare diseases to be aggregated, the number of common diseases to remain separate, the total final classes, and the minimum number of

      samples in the 30% test set were calculated. The results of this sensitivity analysis are presented in Table I.

      TABLE I. SENSITIVITY ANALYSIS RESULTS FOR RARE DISEASE AGGREGATION THRESHOLD

      Threshold	Rare Diseases (Aggregated)	Common Diseases (Separate)	Final Classes	Min. Samples in Test Set
      30	191	582	583	9
      40	224	549	550	12
      50	246	527	528	15
      75	287	486	487	22
      100	330	443	444	30

      The threshold of 50 was selected as optimal based on three methodological criteria. First, it guarantees a minimum of 15 samples in the test set for every class (50 × 0.30 = 15), which is statistically sufficient for reliable calculation of evaluation metrics (accuracy, recall, F1-score). Second, it achieves an ideal balance between coverage (retaining 527 distinct diseases for direct classification) and reliability (ensuring sufficient samples for learning). Third, the acceptable loss ratio of 31.8% (246 diseases aggregated into a single "Rare Case / Other" category) still covers the vast majority of diagnostic needs in primary healthcare environments, while rare cases are referred to specialized physicians. Diseases below this threshold were grouped into a single category, reducing the total classes from 773 to 528.

   3. ‌Model Development and Training

      The preprocessed data was split into 70% training and 30% testing sets using stratified sampling to maintain proportional class representation. Four models were selected to ensure diverse learning mechanisms: Decision Tree as an interpretable baseline, Random Forest to mitigate overfitting via bagging, XGBoost as an advanced boosting algorithm with built-in regularization, and K-Nearest Neighbors (KNN) as a distance- based lazy learning approach. All models were configured with a fixed random state of 42 to ensure reproducibility.

   4. ‌Diagnostic Workflow

   The diagnostic workflow of the proposed system follows a systematic three-stage process, as illustrated in Fig. 1. The flowchart demonstrates the complete diagnostic pipeline from initial user interaction through data validation, machine learning analysis, and final prediction output.

   The process begins with symptom input, where users can search or select from 377 available symptoms through an interactive interface. The system then validates the input data to ensure completeness and correctness. If the input is invalid, the user is prompted to re-enter the symptoms.

   Upon successful validation, the encoded symptom vector is analyzed using the trained XGBoost model to generate probabilistic predictions for all 528 disease classes. The system then ranks the possible diseases by confidence score and

   displays the top 5 predictions exceeding a 1% confidence threshold to the end user.

   Fig. 1. Flowchart of the proposed disease prediction system

4. ‌RESULTS AND DISCUSSION

   1. ‌Model Evaluation Metrics

      The models were evaluated using quantitative metrics derived from the confusion matrix, including accuracy, precision, recall, and F1-score. Accuracy represents the ratio of correct predictions, precision measures the proportion of true positives among all positive predictions, recall quantifies the proportion of actual positives correctly identified, and F1-score provides the harmonic mean of precision and recall for balanced classifier comparison. To ensure fairness in the imbalanced context, both macro average (equal weight to all classes) and weighted average (weighted by class support) were calculated.

   2. ‌Performance Evaluation

      The evaluation results for the four models are summarized in Table II.

      TABLE II. PERFORMANCE EVALUATION OF THE FOUR MODELS

      Model	Acc	Macro Avg			Weighted Avg
      		Macro Precisi on	Macro Recall	Macr o F1	Weig hted Preci sion	Weig hted Recal l	Weig hted F1
      Decision Tree	0.813	0.82	0.83	0.82	0.82	0.81	0.81
      Random Forest	0.839	0.86	0.86	0.85	0.84	0.84	0.84
      XGBoost	0.8406	0.86	0.85	0.85	0.85	0.84	0.84
      KNN	0.8167	0.83	0.84	0.83	0.82	0.82	0.82

      While Random Forest and XGBoost showed closely matched performance, XGBoost was selected for final deployment for four critical reasons: it achieved the highest overall accuracy (84.06%), demonstrated superior weighted precision (0.85) minimizing false positives, produces a compact file size (approximately 20 to 40 MB) in the native

      .ubj format, and requires no decompression time during application loading.

      To contextualize these results within the broader literature, a quantitative comparison with recent related studies is instructive. Munagala et al. [35] achieved 98% accuracy in coronavirus classification using compression-complexity measures, while Keniya et al. [36] reported 93.5% accuracy for disease prediction from symptoms across more than 230 disease classes. Spandana et al. [37] obtained 98% accuracy using genetic algorithm-based AutoML for binary classification and regression tasks, and Dubey et al. [38] achieved a maximum accuracy of 87.78% (using AdaBoost- J48) for heart disease prediction. While these figures appear higher than the 84.06% accuracy achieved by the proposed XGBoost model, a direct comparison must account for fundamental differences in problem complexity. Most of these studies, including [35], [37], and [38], address binary classification tasks, which are inherently less complex than multi-class problems. Even Keniya et al. [36], who addressed a comparably large number of classes, did not report facing the severe class imbalance that necessitated the aggregation strategy proposed in this study. In contrast, the proposed system distinguishes among 528 distinct disease classes derived from 773 original diseases, using a large-scale dataset of 246,945 samples with pronounced class imbalance. Achieving 84.06% accuracy in this high-dimensional, imbalanced, multi-class setting represents a substantial contribution, demonstrating the effectiveness of the proposed rare disease aggregation strategy and the robustness of the XGBoost classifier in realistic primary healthcare scenarios.

   3. ‌Explainable Artificial Intelligence

      To validate the model's clinical logic, a feature importance analysis was conducted on the selected XGBoost model. The analysis extracted the relative weights assigned to each of the 377 symptoms during the diagnostic process, as illustrated in Fig. 2.

      Fig. 2. Top 20 diagnostic symptoms feature importance

      The feature importance analysis reveals two distinct patterns that reflect the model's logical alignment with medical reasoning:

      • Primary Diagnostic Keys (Ranks 1-7): The top seven symptoms collectively account for approximately 84% of the total feature weight. These symptoms, including hip stiffness or tightness, breathing fast, and joint swelling, represent common, general symptoms that appear across a wide range of diseases. The model learns these as primary diagnostic keys for initial differentiation between disease categories. The high concentration of importance in a limited number of symptoms is a positive indicator that the model has successfully identified the most significant signals in the data.

      • Precise Confirmatory Signals (Ranks 8 and below): Symptoms from rank 8 onwards possess relatively small weights (less than 0.01 each). However, low weight does not imply lack of diagnostic value. Symptoms such as eyelid lesion or rash, infant feeding problem, and itching of the anus are rare in the dataset but critical when present, serving as precise confirmatory signals to distinguish between clinically similar diseases. The model's ability to capture these rare symptoms and assign them weight, however small, proves it learns nuanced patterns reflecting the complexity of medical reality rather than merely memorizing the data.

   This graded distribution of importance, from common primary symptoms to rare specialized symptoms, provides strong evidence of the model's credibility and efficiency, enhancing confidence in its use as an assistive tool in primary healthcare environments.

5. ‌SYSTEM IMPLEMENTATION

   The trained XGBoost model was integrated into a production-ready web application using the Django framework, adhering to the Model-View-Template (MVT) architectural pattern. The project structure follows Django best practices, with dedicated directories for templates, static assets (CSS and JavaScript), and the serialized model artifacts.

   1. ‌System Architecture

      The application was designed according to the Model- View-Template (MVT) pattern provided by the Django framework, which separates the three application layers: data (Model), business logic (View), and user interface (Template). As illustrated in Fig. 3, the application consists of four main layers:

      • Frontend Layer: Comprises three main HTML templates (home page, diagnosis page, and result page), supported by CSS files for styling and JavaScript for dynamic interactions such as symptom search.

      • Backend Layer: Contains three core modules: urls.py for routing requests to appropriate handlers, views.py for processing user requests and preparing responses, and utils.py as a utility module responsible for loading the model and executing predictions.

      • Data Layer: Stores the trained model files and intermediate files in the models/ directory, produced from the separate training phase.

      • Training Phase: Represents the model training stage conducted in Jupyter Notebook using the Anaconda environment, where the model and intermediate files used by the data layer were generated

        Fig. 3. System architecture diagram

   2. ‌Model Loading Strategy

      To optimize performance, a singleton design pattern was implemented for model loading. Since loading the model from disk for every diagnostic request is computationally expensive, the system loads the model and intermediate files into memory only once upon the first request. The same instance is then reused for all subsequent requests, drastically reducing RAM consumption and response latency.

   3. ‌Diagnostic Workflow and Data Processing

      The user selects symptoms from 377 available options through an interactive interface featuring both text-based search with autocomplete suggestions and a complete alphabetical browsing list. The selected symptoms are sent to the server via HTTP POST request, where they are converted into a one-hot encoded vector of 377 elements. This vector is fed to the trained XGBoost model to calculate probabilities for all 528 disease classes, and the system displays the top 5 diseases exceeding a 1% confidence threshold.

   4. ‌User Interface Design Principle

   The frontend was designed following five key principles: simplicity and clarity, immediate feedback via visual progress bars for confidence scores, responsive design for all screen sizes, full Arabic language support with right-to-left (RTL) text direction, and visual hierarchy to guide user attention.

   The application consists of three main interfaces that reflect these principles. The main application page, shown in Fig. 4, features a clean navigation bar, a welcome section with a system description, and three feature cards highlighting early diagnosis, remote area support, and platform security.

   Fig. 4. Main application page showing navigation bar, welcome section, and feature cards.

   For symptom selection, the input page (Fig. 5) provides an interactive search functionality with autocomplete, alongside an alphabetical browsing list. Selected symptoms are immediately displayed as removable tags to provide visual feedback to the user.

   Fig. 5. Symptom input page showing interactive search functionality and alphabetical browsing list.

   Finally, the diagnosis results page, illustrated in Fig. 6, displays the top 5 predicted diseases with their respective confidence scores (e.g., cranial nerve palsy at 49.8%, tic disorder at 45.6%, Presbyopia at 2.6%). It also includes a mandatory disclaimer at the bottom clarifying that the output is a preliminary consultation and not a substitute for professional medical advice

   Fig. 6. Diagnosis results page displaying top 5 predicted diseases with confidence scores and a preliminary consultation disclaimer.

6. ‌CONCLUSION AND FUTURE WORK

This research successfully developed and deployed an intelligent primary healthcare diagnostic system. The proposed rare disease aggregation strategy effectively resolved severe class imbalance, enabling the robust training of ML models. The XGBoost model emerged as the optimal choice, achieving 84.06% accuracy while offering superior deployment efficiency and interpretability through XAI. Future work will focus on validating the model on real-world clinical data, collecting additional samples for rare diseases, and integrating an interactive, natural language processing (NLP)-based medical chatbot for dynamic patient consultations.

‌ACKNOWLEDGMENT

The author would like to express her sincere gratitude to her supervisor, Dr. Issam Salman, for his continuous guidance, valuable insights, and unwavering support throughout the course of this research. The author also extends her appreciation to the faculty members of the Web Science Department at the Syrian Virtual University for their encouragement and for providing the necessary academic environment to complete this work.

‌REFERENCES

1. World Health Organization, The world health report 2002: Reducing risks, promoting healthy life. WHO, 2002.

2. World Health Organization, Preventing chronic diseases: A vital investment. WHO, 2005.

3. A. R. Omran, "The epidemiologic transition: A theory of the epidemiology of population change," Milbank Memorial Fund Quarterly, vol. 49, no. 4, pp. 509538, 1971.

4. A. Boutayeb and S. Boutayeb, "The burden of non communicable diseases in developing countries," International Journal for Equity in Health, vol. 4, no. 1, p. 2, 2005.

5. A. Wagstaff, "Poverty and health sector inequalities," Bulletin of the World Health Organization, vol. 80, no. 2, pp. 97105, 2002.

6. M. Marmot, "Social determinants of health inequalities," The Lancet, vol. 365, no. 9464, pp. 10991104, 2005.

7. M. Marmot, S. Friel, R. Bell, T. A. Houweling, and S. Taylor, "Closing the gap in a generation: Health equity through action on the social determinants of health," The Lancet, vol. 372, no. 9650, pp. 16611669, 2008.

8. E. Sabaté, Ed., Adherence to long-term therapies: Evidence for action. World Health Organization, 2003.

9. L. Osterberg and T. Blaschke, "Adherence to medication," New England Journal of Medicine, vol. 353, no. 5, pp. 487497, 2005.

10. J. Lazarou, B. H. Pomeranz, and P. N. Corey, "Incidence of adverse drug reactions in hospitalized patients: A meta-analysis of prospective studies," JAMA, vol. 279, no. 15, pp. 12001205, 1998.

11. W. Raghupathi and V. Raghupathi, "Big data analytics in healthcare: Promise and potential," Health Information Science and Systems, vol. 2, no. 1, p. 3, 2014.

12. T. Davenport and R. Kalakota, "The potential for artificial intelligence in healthcare," Future Healthcare Journal, vol. 6, no. 2, pp. 9498, 2019.

13. H. C. Koh and G. Tan, "Data mining applications in healthcare," Journal of Healthcare Information Management, vol. 19, no. 2, pp. 6472, 2011.

14. K. Annan, We the peoples: The role of the United Nations in the 21st century. United Nations, 2000.

15. T. R. Frieden, "A framework for public health action: The health impact pyramid," American Journal of Public Health, vol. 100, no. 4, pp. 590 595, 2010.

16. A. S. Go, D. Mozaffarian, V. L. Roger, E. J. Benjamin, J. D. Berry, W.

    B. Borden, et al., "Heart disease and stroke statistics2013 update: A report from the American Heart Association," Circulation, vol. 127, no. 1, pp. e6e245, 2013.

17. M. Ng, T. Fleming, M. Robinson, B. Thomson, N. Graetz, C. Margono, et al., "Global, regional, and national prevalence of overweight and obesity in children and adults during 19802013: A systematic analysis for the Global Burden of Disease Study 2013," The Lancet, vol. 384, no. 9945, pp. 766781, 2014.

18. R. Pawankar, "Allergic diseases and asthma: A global public health concern and a call to action," World Allergy Organization Journal, vol. 7, no. 1, p. 12, 2014.

19. A. K. Jha, C. M. DesRoches, E. G. Campbell, K. Donelan, S. R. Rao, T.

    G. Ferris, et al., "Use of electronic health records in U.S. hospitals," New England Journal of Medicine, vol. 360, no. 16, pp. 16281638, 2009.

20. D. Blumenthal and M. Tavenner, "The 'meaningful use' regulation for electronic health records," New England Journal of Medicine, vol. 363, no. 6, pp. 501504, 2010.

21. HealthIT.gov, "Electronic health records (EHRs)," Office of the National Coordinator for Health Information Technology, 2023.

22. ISO/TC 215, "Health informatics standards," International Organization for Standardization, 2023.

23. A. Haleem, M. Javaid, R. P. Singh, and R. Suman, "Telemedicine for healthcare: Capabilities, features, barriers, and applications," Sensors International, vol. 2, p. 100117, 2021.

24. L. Ricciardi, F. Mostashari, J. Murphy, J. G. Daniel, and E. P. Siminerio, "A national action plan to support consumer engagement via e-health," Health Affairs, vol. 32, no. 2, pp. 376384, 2013.

25. Agency for Healthcare Research and Quality, "Medical expenditure panel survey (MEPS) data," AHRQ, 2023.

26. E. H. Shortliffe and J. J. Cimino, Eds., Biomedical informatics: Computer applications in health care and biomedicine, 4th ed. Springer, 2014.

27. M. A. Musen, B. Middleton, and R. A. Greenes, "Clinical decision- support systems," in Biomedical informatics, pp. 643674. Springer, 2014.

28. T. R. Gruber, "A translation approach to portable ontology specifications," Knowledge Acquisition, vol. 5, no. 2, pp. 199220,

    1993.

29. A. L. Rector, "Modularisation of domain ontologies implemented in description logics and related formalisms including OWL," in Proc. 2nd Int. Conf. Knowledge Capture, pp. 121128, 2003.

30. J. Pearl, Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, 1988.

31. S. Russell and P. Norvig, Artificial intelligence: A modern approach, 4th ed. Pearson, 2020.

32. U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, "From data mining to knowledge discovery in databases," AI Magazine, vol. 17, no. 3, pp. 37 54, 1996.

33. R. K. Dhivyesh, "Disease-symptom dataset," Kaggle, 2023. [Online]. Available: https://www.kaggle.com/datasets/dhivyeshrk/diseases-and- symptoms-dataset

34. L. A. Zadeh, "Fuzzy sets," Information and Control, vol. 8, no. 3, pp. 338353, 1965.

35. N. V. T. S. Munagala, P. K. Amanchi, K. Balasubramanian, A. Panicker, and N. Nagaraj, "Compression-complexity measures for analysis and classification of coronaviruses," Entropy, vol. 25, no. 1, Article 81, 2023, doi: 10.3390/e25010081.

36. R. Keniya, et al., "Disease prediction from various symptoms using machine learning," SSRN, 2020, doi: 10.2139/ssrn.3661426.

37. S. Chereddy, S. I. Venkata, A. Nandhini, P. K. Rangarajan, B. M. Gurusamy, and P. Srinivasan,"An efficient genetic algorithm based Auto ML approach for the classification and regression," in Proc. 2023 Int. Conf. Intelligent Data Communication Technologies and Internet of Things (IDCIoT), 2023, doi: 10.1109/IDCIoT56793.2023.10053442.

38. S. K. Dubey, S. Sinha, and A. Jain,"Heart disease prediction classification using machine learning," International Journal of Inventive Engineering and Sciences, vol. 10, no. 11, pp. 16, Nov. 2023, doi: 10.35940/ijies.b4321.11101123.