Medical Diagnosis System Using Artificial Intelligence and Machine Learning

Medical Diagnosis System Using Artificial Intelligence and Machine Learning

Mrs. RAJESHWARI. M (1)

ASSISTANT PROFESSOR

(rajimecse90@gmail.com)

Mohammed Saifulla(2) Mohammed Ayaan R(3) Mohammed Faraz(4) Kashinath(5)

(mohammedsaifulla2207@gmail.com)

Department of Computer Science & Engineering, ACS College of Engineering

Affiliated to VTU Belagavi | NAAC 'A' Grade | Kambipura, Mysore Road, Bangalore-560074, India

Abstract Modern healthcare demands faster, more accurate diagnosis and better patient support. This paper presents MedAI Hub, a unified AI- powered Medical Diagnosis and Healthcare Assistance System built on a modular web architecture. The platform integrates Convolutional Neural Network (CNN)-based image classification using TensorFlow/Keras for detection of skin diseases, brain tumors (MRI), and pneumonia (chest X-ray), achieving classification accuracies between 89% and 95%. A React.js frontend communicates with a FastAPI backend, while an SQLite database manages patient records securely. Additional modules include a Wellness Hub for personalized diet and fitness plans, a Responder Hub for emergency first-aid and CPR guidance, an AI-powered medical chatbot, and a role-based doctorpatient communication interface. The system demonstrated reliable diagnostic predictions with confidence scores, improved early detection capability, and a streamlined clinical workflow. Experimental results confirm an average API response time of 460 ms and a confidence threshold mechanism that flags predictions above 75% as clinically reliable.

Keywords Medical Diagnosis, Artificial Intelligence, Machine Learning, CNN, Deep Learning, FastAPI, React.js, TensorFlow, Wellness Hub, Telemedicine.

1. INDRODUCTION

   Healthcare is undergoing a profound transformation, driven by the rapid advancement of Artificial Intelligence (AI) and Machine Learning (ML). The exponential growth of medical data from electronic health records, diagnostic imaging, wearable sensors, and laboratory systems has far exceeded the capacity of traditional manual analysis methods. Clinicians and healthcare providers increasingly require intelligent tools capable of analyzing diverse datasets, identifying

   complex patterns, and delivering timely, accurate diagnostic insights to support clinical decision- making.

   Despite this progress, existing healthcare systems remain fragmented. Diagnosis often relies on manual interpretation of medical images, personalized wellness support is rarely available at scale, emergency first-aid knowledge is inaccessible in critical moments, and doctor patient communication platforms are disconnected from diagnostic tools. This paper presents MedAI Hub, a comprehensive AI-based Medical Diagnosis and Healthcare Assistance System that addresses these limitations through a unified, modular web- based platform.

   The key contributions of this work are: (1) an end- to-end web platform combining AI-driven image analysis with wellness and emergency features; (2) CNN models trained on domain-specific datasets for multi-disease classification; (3) a FastAPI backend efficiently serving ML predictions via RESTful APIs; and (4) a role-based user management system ensuring secure, authenticated access for patients and doctors.

2. RELATED WORKS

   Modern healthcare systems face several interconnected challenges that motivate this work:

   • Early disease detection is difficult for image- based conditions (skin diseases, brain tumors, pneumonia) requiring specialized expertise not universally available.

   • Personalized wellness guidance such as diet and exercise plans is rarely integrated into diagnostic platforms, leaving patients without proactive health support.

   • Emergency response awareness (CPR, first aid) is limited among the general public, increasing risks and delays in critical situations.

   • Doctorpatient interaction is fragmented across multiple disconnected applications, reducing clinical efficiency and continuity of care.

   • No unified digital platform currently integrates AI-powered diagnosis, personalized wellness, emergency guidance, and clinical data management in a single accessible interface. Medical Health Diagnosis System, utilizing artificial intelligence and machine learning in the early detection and precise diagnosis of a patient's medical condition. Based on the analysis of the patient's data and symptoms, this system should be able to render dependable preliminary assessments that will help health professionals make informed decisions. In general, this is going to improve diagnosis accuracy and enhance patient outcomes with advanced AI and ML technologies. The project also aims to connect advanced technologies with the real needs in healthcare. Including machine learning models in the system, it will keep improving and learning, thus being able to give better diagnoses each day that passes.

3. METHODOLOGY

   The proposed MedAI Hub is a modular, web-based AI healthcare platform comprising six core modules as illustrated in Fig. 1. The system integrates a React.js frontend, FastAPI backend, TensorFlow/Keras CNN models, and an SQLite database into a cohesive architecture.

   1. Frontend Design (React.js)

      A responsive, component-based UI developed in React.js supports image uploads, form inputs, chatbot interaction, and role-based navigation for patients and doctors across modules: HomePage, Analysis Page, Wellness HubPage, Responder HubPage, Login Page, Registration Page, Patient Profile Page, and Doctor Dashboard.

   2. Backend Integration (FastAPI)

      A high-performance FastAPI backend handles API requests, manages Base64/JWT authentication, and coordinates communication between the frontend, ML models, and SQLite database with automatic Swagger UI documentation and asynchronous processing.

   3. CNN-Based Diagnosis Module

      Three specialized CNN models are trained using TensorFlow/Keras for: (1) skin disease classification using the HAM10000 dataset, (2) brain tumor detection from MRI scans (meningioma, glioma, no tumor), and (3) pneumonia classification from chest X-rays. Each model returns a predicted label with a ranked confidence score. The inference pipeline follows: image upload FastAPI validation preprocessing (resize, normalize) CNN inference SQLite storage React UI display.

   4. Wellness Hub

      Users provide dietary preferences and fitness goals to receive AI-generated personalized daily diet

      plans and workout routines (Men's Plan / Women's Plan) adapted to individual profiles.

   5. Responder Hub

      The Emergency Responder Hub provides curated video content on CPR, first aid, snakebite treatment, and other emergency procedures. Users search by emergency type and access step-by-step visual guides sourced from trusted health organizations.

   6. Medical Chatbot and Doctor Interface

   An AI chatbot (Gemini 2.5 Flash API) assists users with navigation, symptom questions, and health FAQs. Doctors securely log in, search patient records by ID, and annotate clinical notes to maintain continuity of care.

   Fig 3.1 fastAPI routing algorthm

   Use FastAPI for lightweight and performance model serving, and automatic API documentation. Machine Learning Pipeline image FastAPI + Frontend architecture: React FastAPI DB – Use the FastAPI-React diagram. FastAPI serving + monitoring – endpoint, predictions, monitoring storage; use the image of the FastAPI + Evidently architecture.

   The development followed a five-phase methodology:

   Phase

   Description

   1. Requirements Analysis	Identify healthcare problems, define system needs for AI- driven diagnosis, wellness, emergency, and doctorpatient management.
   2. System Design	Design modular architecture: React frontend FastAPI backend CNN models SQLite DB.
   3. Implementation	Develop frontend pages, backend API routes, CNN model training and integration, and database schemas.
   4. Testing	Functional, integration, performance, security, UI, and model accuracy testing across all modules.
   5. Result Analysis	Evaluate CNN accuracy, API response time, confidence scores, and overall system performance.

   The system was implemented as a full-stack web application. Table II summarizes the six implementation modules:

   Module	Description
   M1: Frontend	React.js responsive UI image upload, navigation, dashboards.
   M2: Backend	FastAPI routes: /auth, /predict, /chatbot, /emergency api.
   M3: CNN Diagnosis	TensorFlow/Keras CNN models (.h5) for skin, MRI, X-ray analysis.
   M4: Wellness & Responder	Personalized diet/exercise plans + emergency CPR video hub.
   M5: Additional Features	Gemini chatbot integration, doctor search, patient history.
   M6: Testing	Functional, integration, performance, security, UI testing.

   Table I. Development Methodology Phases

4. SYSTEM DESIGN

   The system design follows a hybrid ML architecture combining stream processing for real- time diagnosis and batch processing for offline model training. Fig 5.1 illustrates the patient workflow from image upload to result display.

   Fig.4. 1 Patient Workflow: Image Upload to ML Prediction

   The input layer handles medical images (X-ray, MRI, skin photos) submitted via REST API as JSON or image streams. The API validates file type, extracts metadata, and routes data into the ML pipeline. The stream ML module performs real-time classification using CNNs, returning predictions with confidence scores. The storage layer uses SQLite to maintain patient login records, diagnosis histories, and doctor annotations. Batch learning modules handle offline model retraining on updated datasets.

   Table II. Implementation Modules

   The FastAPI backend (app.py) uses Pydantic schemas for data validation. Patient and doctor accounts are managed with role-based access control. CNN models are loaded at startup and serve predictions via the /predict endpoint. Confidence thresholds classify predictions as reliable (75%), uncertain (5074%), or inconclusive (<50%).

5. EXPERIMENTAL RESULTS

   The system was tested across all six testing dimensions. The CNN models achieved strong classification performance on their respective test sets as shown in Fig. 5.1.

   Fig. 5.1. (a) CNN Model Accuracy (b) System Response Time (ms)

   The Skin Disease CNN achieved 89.4% accuracy on the HAM10000 test partition, the Brain Tumor MRI CNN achieved 91.2%, and the Pneumonia

   Chest X-ray CNN achieved 94.6%. A representative diagnosis result showed Pneumonia classified with 98% confidence, while a brain

   tumor MRI scan returned meningioma_tumor at 84% (glioma_tumor 15%, no_tumor 1%).

   The FastAPI backend demonstrated average API response times of 460 ms (preprocessing: 45 ms, CNN inference: 380 ms, DB write: 35 ms) under standard single-user load. The medical chatbot (Gemini 2.5 Flash) responded accurately to health- related queries within 1.2 seconds. The Wellness Hub generated gender-specific workout plans and the Responder Hub returned accurate emergency video results for CPR and snakebite queries.

   Fig. 5.2. MedAI Hub Home Page and Analysis Result Interface

6. CONCLUSION

   MedAI Hub, used for Medical Diagnosis and Healthcare Assistance System that integrates CNN- based image diagnosis, personalized wellness support, emergency response education, and doctorpatient management into a single accessible web platform. The system demonstrated that modern AI and web technologies can be effectively combined to address real-world healthcare challenges, reducing diagnostic time, improving early disease detection capability, and enhancing healthcare accessibility.

   CNN models achieved accuracy between 89% 95% across three disease domains. The full-stack architecture (React + FastAPI + SQLite) delivered sub-500 ms average response times. Role-based authentication ensured secure data access for both patients and doctors. Future enhancements include: training on larger and more diverse datasets, adopting EfficientNet/Vision Transformer architectures, integrating Grad-CAM explainability maps, extending disease coverage (diabetic retinopathy, COVID-19), and deploying on cloud infrastructure with HTTPS and GDPR-compliant data handling for real-world clinical use.

7. REFERENCES

1. Javaid, M., & Haleem, A. (2023). Significance of Machine Learning in Healthcare: Features, Pillars, and Applications. Journal of Healthcare Innovation, 14(1), 45-55.

2. Barodiya, V. K. (2022). A Study of Disease Diagnosis Using Machine Learning. Journal of ML in Medicine, 11(2), 89- 101.

3. O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional networks for biomedical image segmentation, Proc. MICCAI, pp. 234241, 2015.

4. K. He et al., Deep residual learning for image recognition, IEEE CVPR, pp. 770778, 2016.

5. P. Rajpurkar et al., CheXNet: Radiologist-level pneumonia detection on chest X-rays, arXiv:1711.05225, 2017.