Real-Time Driver Monitoring System using Face recognition and Head pose detection

Real-Time Driver Monitoring System using Face recognition and Head pose detection

Sumana

Student, St. Joseph Engineering College, Mangalore

Nishmitha J

Assistant Professor, St Joseph Engineering College , Mangalore

Abstract – Distractions, tiredness, and unsafe driving are some of the leading causes of road accidents worldwide. This paper shows a real-time Driver Monitoring System (DMS) that uses both head pose detection and behavioural moni toring to make driving safer. The system uses MediaPipe Face Mesh to extract landmarks and keep track of the drivers head position all the time. A Support Vector Machine (SVM) classifier that was trained on a labelled dataset can tell if a persons head is straight, left, or right with 98.82% accuracy. Long periods of time spent not facing straight ahead mean that you are distracted, which sets off voice alerts right away. The system also has more detection modules, such as one that can tell when someone is drowsy by looking at their Eye Aspect Ratio (EAR) and using an SVM classifier with 95.55% accuracy. Using a fine-tuned YOLOv8 object detection model, it can detect when someone is using their phone and how they are rubbing eyes. For every unsafe behaviour that is detected, a voice alert is sent to the driver in real time, telling them to take action. This proposed system works well with standard webcams and lightweight deep learning models, quickly and accurately spotting driver distractions, fatigue, and dangerous behaviours. This shows that the system could make driving safer by using smart, non-intrusive, real-time monitoring.

Keywords Driver Monitoring System (DMS), Head Pose Detection, Drowsiness Detection, Distraction Detection, YOLOv8, Eye Aspect Ratio (EAR), MediaPipe Face, Support Vector Machine (SVM), Real-Time Monitoring, Driver Safety.

1. INTRODUCTION

   Driver distraction and tiredness are two of the main reasons for accidents around the world. These collisions pose a serious risk to other road users as well as drivers. According to studies, many auto accidents occur as a result of drivers becoming distracted, losing concentration, or falling asleep at the wheel. Advanced Driver Monitoring Systems (DMS), which continuously monitor driver behaviour and issue alerts when needed, have emerged as successful strategies to increase driver safety.This study presents a real-time DMS that uses head pose detection in conjunction with multiple

   behavioural monitoring modules to detect signs of distraction, drowsiness, and risky behaviours like cell phone use. Head posture is a good indicator of driver attention because prolonged deviations from the straight-ahead position frequently signify distraction. The system employs MediaPipe Face Mesh to extract facial landmarks and a Support Vector Machine (SVM) classifier trained on a labelled dataset to classify head orientation as straight, left, or right. When a driver has been distracted for a long time, real-time voice alerts are triggered to get their focus back on the road. In addition to detecting head poses, the system also employs an SVM model with a 95.55% accuracy and uses eye rubbing detection using hand landmark tracking and drowsiness detection via Eye Aspect Ratio (EAR), both of which are indications of fatigue. Additionally, mobile phone usage is detected using an improved YOLOv8 object detection model. The system instantly voice-alerts the driver of any identified unsafe behaviour, reducing the likelihood of collisions and enabling the driver to take corrective action.

   The proposed system operates in real-time, ensuring efficient performance suitable for practical implementation, by utilising lightweight deep learning models and standard webcams. By integrating multiple detection modules and real- time alert mechanisms, the system offers a reliable, non- intrusive method of enhancing driver safety through intelligent, continuous monitoring.

2. LITERATURE SURVEY

   Multiple past research has contributed toward developing driver monitoring systems using machine-learning, deep- learning, computer-vision, and the like. The following works were referred to for insights and technical knowledge:

   1. A proposed ensemble driver anomaly detection system utilizes XGBoost as the meta-learner, which is combined with ResNet50, DenseNet201, and InceptionV3. The explanation technique for model prediction was SHAP (SHapley Additive Explanations). By noticing behaviors of the driver, such as yawning, sleeping, and head movement, the ensemble had an almost perfect accuracy with precision and recall scores nearing 100%.

   2. A real-time driver sleepiness detection model is

      Ref.No	Methodology	Performance
      [2]	Eye state detection using LBP, HOG, and template matching.	Accuracy Not specified; dependable and real-time.
      [3]	For contactless heart and breathing monitoring, a ToF camera was utilised.	RR error 1.4 breaths/min; HR success up to 71.9%.
      [4]	Eye closure detection and face recognition were combined to provide customised drowsiness tracking.	Over 95% f driver IDs are accurate, and drowsiness detection is also very successful.
      [5]	CNNs for facial expression analysis and YOLOv8 for object detection are used to	High, real-time accuracy (not stated explicitly,
      	track driver fatigue and distraction.	but evaluated as strong).
      [7]	Driver fatigue while wearing sunglasses can be detected using adaptive image processing and landmark tracking.	Under occlusion, performance was observed to be approximately 95% accurate.
      [8]	CNNs are used in a low-cost deep learning technique for real-time driver monitoring.	Designed for common car hardware, over 90% accuracy was attained.
      [9]	Behaviour monitoring using edge detection and thresholding.	Real-time performance, approximately 90%.
      [10]	Eye closure monitoring at a threshold to identify drowsiness.	In real-world tests, an accuracy of 8590% was attained.

      implemented in this work using facial features and template matching to track the eyes. It used LBP and HOG for feature extraction. While the exact accuracy was not mentioned, the real-time system functioned well in embedded environments.

   3. This work proposes a contactless vital sign monitoring system, utilizing a near-infrared time-of-flight camera. Deep learning models were applied for signal analysis of heart rate and respiration. The implementation gave more importance to enhancing the driver's safety from physiological state tracking, though accuracy was not explicitly mentioned.

   4. A merged system for driver identification and drowsiness detection was established in this investigation using facial recognition and eye closure analysis. The system utilized deep learning techniques to extract facial features, resulting in an identification accuracy of over 95% and efficient drowsiness detection.

   5. Another monitoring framework was introduced in this paper, which is based on deep learning and employs the open- source YOLOv8 object detection algorithm and CNNs for facial expression recognition. High accuracy was achieved in detectin distractions, including mobile phone use, and prioritizing real-time performance and multi-modal input to improve driver monitoring.

   6. In this study, researchers introduced a lightweight and real-time drowsiness detection model called 4D, which utilizes a convolutional neural network (CNN). This innovative system boasts an impressive accuracy of 96.5% when distinguishing between awake and drowsy states, and its been fine-tuned for use on embedded platforms.

   7. Additionally, the paper tackled the challenge of detecting driver fatigue while theyre wearing sunglasses by employing adaptive image processing techniques. By leveraging traditional thresholding and landmark tracking, the model was able to reliably detect drowsiness even under occlusion, achieving a reported accuracy of 95%.

   8. In this study, researchers developed an affordable driver monitoring system that leverages CNN-based deep learning techniques. Impressively, the system achieved over 90% accuracy in spotting drowsiness and inattention, all while being computationally efficient and perfectly suited for real- time use in regular vehicles.

   9. The work also utilized standard image processing techniques such as edge identification, template matching, and thresholding. While it didnt delve into deep learning, the system still showcased practical accuracy for real-time drowsiness detection using these fundamental techniques.

   10. In this paper, we explored a drowsiness detection method that relies on monitoring eye closure patterns. While it sticks to traditional techniques, it highlights the need for low- complexity solutions and the importance of keeping drivers alert, achieving a balance with moderate yet practical accuracy levels.

   Table 1:Literature Survey

3. METHODOLOGY

   The methodical approach taken in the creation and deployment of the suggested driver monitoring system (DMS) is described in this section. Data collection, performance, functional development, model training, evaluation, and real-time integration are some of the steps that make up this feature. The objective is to use a light machine learning model in conjunction with data vision techniques to monitor drowsiness in real time and correct the driver's distraction.

   1. Data Collection

      For a head position detection, a label dataset with three classes was created: Straight, left-turn and right-wing head orientation. Images were arranged in separate folders for each class and occupied under different circumstances to improve the strength of the model. To detect drowsiness, a dataset where two classes- drowsy and natural- was prepared by taking the driver images after the actual driving scenarios. Medapipe Face Mesh was used to filter all images and only individuals with clearly defined faces were kept for model quality.

   2. Data Preprocessing

      All the images were converted in OpenCV from BGR to RGB format for use with medapipe face mesh. The system extracted 468 facial landmarks per picture, which provides wide facial geometry. Images without a valid landmark detection were automatically removed to maintain data set quality. To detect drowsiness, the invalid images were removed to avoid complete anomalies. (X, Y, Z) Landemark Coordinates were leveled into a dimensional functional vector for both head currency and drowsiness. All these features were employed to train the machine learning model. Ultimately, the dataset was partitioned into 20% test sets for training and evaluation.

   3. Feature Extraction

      For each valid image, 468 the three-dimensional (x, y, z) discovered the coordinates of facial places were extracted using a Mediapipe Face network. These coordinates were leveled into a dimensional functional vector, which captures the general structure and spatial arrangement of the driver's face. To detect head position, a comprehensive set of landmark coordinates was utilized to categorize head orientation as left or right. Particular focus was placed on regions near the eyes, where the variation in these coordinates provides important clues to close the eye canal and fatigue.

      Figure 1: Data distribution for the drowsiness test: 45.6% drowsy, 54.4% natural.

   4. Model Training

      After removing the functional vector from the facial landmark, an 80:20 distribution of data was used to evaluate the model's performance by dividing it into training and test kits. The selection of a linear core-based supporting vector (SVM) was made to identify head position detection and sleepiness, as it is capable of handling high-dimensional data and producing precise results with minimal computational complexity. For a head position detection, the SVM model was trained to classify the driver's head position into three categories: Straight, Left-Mandatory or Right. This model gained an accuracy of 98.82%, and showed its high credibility in identifying when the driver ignores the road. Similarly, to detect drowsiness, the model was trained to classify whether the driver is in a awake or drowsy state using facial national label functions, especially focusing on the pattern in the eye area. The model for detecting trained drowsiness gained an accuracy of 95.55%and confirmed the efficiency of identifying signs of fatigue. Both models were stored using joblib so that they could be integrated into the real -time system for continuous monitoring of the driver and immediate notice during the vehicle operation.

   5. Real-Time Integration

   The trained model was incorporated into the real time monitoring system during the last performance stage. The video frame was caught with a standard webcam, and the frame of the frame was pulled out of the frame using a media feature face net. Save SVM model classified the condition of the guide's position and real -time drowsiness. If the head swings for a long time, drowsiness or other insecure behavior

   was detected, the system produced immediate voice warning using PYTSX3 to accelerate corrective driver functions. The system is operated effectively with an integrated approach that ensures safe operation and provides precise, non-Guspath monitoring.

4. RESULTS

   The experimental outcomes of the head pose detection and drowsiness detection models using Support Vector Machine (SVM) classifiers are presented in this section. Classification accuracy, precision, recall and F1-score and matrix analysis were applied to the models.

   1. Head Pose Detection Results

      The SVM model classified the driver's head orientation into three categoriesright, left, and straightwith a general accuracy of 98.82% for head position detection. The data set's distribution was depicted as a pie chart, indicating the proportion of test samples associated with each class. A histogram was utilized to display the frequency of predictions for each category, indicating how well these models performed in different heads. The model's high reliability in detecting changes in head direction was validated by the confusion matrix, which showed minimal abortion. These outcomes demonstrate the system's efficacy in accurately detecting the movements of distracted heads, which is essential for driver damping monitoring.The model was constantly testing in testing samples, and when the driver ignores the road, its ability to issue notice in time helps to improve the general driver's safety.

      Figure 2: Head posture warnings for distractions on the left and right.

   2. Drowsiness Detection Results

      The model for detecting drowsiness achieved classification accuracy of 95.55%, successfully differentiated between the drawn and aroused the driver states. Exact, recall and F1 score calculations were calculated for both classes, all indicating strong models performance. A class distribution pie diagram and classificaton reports were generated to imagine these results, as well as accuracy, recall and F1 score with diagram once. Confusion matrix confirmed the effectiveness of the model to reduce false positivity and accurately identify signs of fatigue.

      Figure 3: A real-time alert for the detection of drowsiness is triggered.

   3. Visualizations and Performance

      During model evaluation, a number of visual aids were created to help comprehend the system's performance. A pie chart representing the square distribution of test data in three categoriesstraight, left, and right head slopewas made for the purpose of detecting head position. This ensured that the evaluation dataset was trustworthy and balanced. To further understand how the model consistently classifies various head orientations, a histogram was plotted to display the frequency of model conditions for each head position class.

      Figure 4: Head pose prediction count for the straight, left, and right classes.

      To detect drowsiness, a pie diagram was generated to imagine the distribution of test samples between the aroused and dried sections. The exact chart that displays the exact, recall and F1 score for both classes was also created to ensure a precise comparison of model performance in different evaluation matrix. Ultimately, the two features were combined in an confusion matrix to indicate the correct and incorrect categories. These visualizations show the system's ability to correct and detect unprotected driver behavior in real time.

5. DISCUSSION

   The results show that the proposed driver monitoring system detects both driver's district through head position analysis and drowsiness using the properties of the eye area. The SVM-based head position model received a high classification accuracy of 98.82%, indicating strong reliability under the supervision of head movements. Similarly, the model for detecting drowsiness was given 95.55% accuracy, which confirmed the system's ability to detect fatigue -related behavior. For real-time applications, the removal of facial sites through media networks proved to be a gentle and successful solution. Under controlled circumstances with consistent, suitable lighting and frontal face visibility, the system operated effectively. Also, various other visual aids like bar graphs, histograms (which analyze the shape of a model), pie charts and confusion matrisum provide clear indication that this

   allows them to quickly warn users against behaviour that may not be protected. Despite these encouraging outcomes, certain limitations were noted. Extreme changes in light, facial expressions, or poor camera angles can all affect the system's demonstration. Significant head rotation outside the camera's field of view can decrease the accuracy of a head position detection. When it comes to drowsiness detection, outside elements like sunglasses or low-resolution entrance can obstruct landmark detection and impact the outcomes. Overall, the system provides a non-grace, real-time solution to increase the driver's safety by continuously monitoring the driver's behavior. The current system already integrates header detection, drowsiness monitoring, mobile phones, facial identification for driver authentication and other risk – related behavior. Future reforms can be focused on expanding datasets to incorporate various real driving scenarios, including additional features such as smoking detection can enhance the system's performance in challenging circumstances, including low light or oxening conditions.

   Figure 5:Workflow of the system

6. CONCLUSION

   This research presents a real-time Driver Monitoring System (DMS) designed to improve driver safety by detecting distraction, drowsiness, and risky behaviors. The system integrates head pose detection using Support Vector Machine (SVM) with an achieved accuracy of 98.82%, and drowsiness detection using facial landmark-based features with 95.55% accuracy. Additionally, mobile phone usage detection and face recognition for driver authentication have been implemented to strengthen overall safety measures. The use of lightweight models, MediaPipe Face Mesh, and real-time processing ensures the system operates efficiently without the need for high-end hardware. The system provides immediate voice alerts when unsafe behaviors are detected, contributing

   to reduced accident risks and enhanced driver awareness. Future improvements will focus on expanding the dataset to include more diverse and real-world driving conditions, improving robustness under challenging environments such as poor lighting or partial occlusions, and integrating additional detection capabilities such as smoking and drinking behavior recognition to further advance the systems ability to ensure driver safety.

7. REFERENCE

1. Chengula, Tumlumbe Juliana, et al. Enhancing Advanced Driver Assistance Systems through Explainable Artificial Intelligence for Driver Anomaly Detection. Machine Learning with Applications, vol. 17, Sep. 2024, p. 100580. DOI.org (Crossref), https://doi.org/10.1016/j.mlwa.2024.100580.

2. D. Ghimire, S. Jeong, S. Yoon, S. Park, and J. Choi, Real- time sleepiness detection for driver state monitoring system, in Proc. of the IT Application Research Center, Korea Electronics Technology Institute, Jeonju-si, Republic of Korea, no. 561-844.

3. Guo, Kaiwen, et al. Contactless Vital Sign Monitoring System for In- Vehicle Driver Monitoring Using a Near- Infrared Time-of-Flight Camera. Applied Sciences, vol. 12, no. 9, Apr. 2022, p. 4416. DOI.org (Crossref), https://doi.org/10.3390/app12094416.

4. Díaz-Santos, Sonia, et al. Driver Identification and Detection of Drowsiness While Driving. Applied Sciences, vol. 14, no. 6, Mar. 2024, p. 2603. DOI.org (Crossref), https://doi.org/10.3390/app14062603.

5. Ibrahim, Moustafa, et al. Driver Monitoring System Using Deep Learning Techniques. Electronic Imaging, vol. 36, no. 3, Jan. 2024, pp. 314-1-31410. DOI.org (Crossref), https://doi.org/10.2352/EI.2024.36.3.MOBMU-314.

6. Jahan, Israt, et al. 4D: A Real-Time Driver Drowsiness Detector Using Deep Learning. Electronics, vol. 12, no. 1, Jan.2023,p.235. DOI.or(Crossref), https://doi.org/10.3390/electronics12010235.

7. Tang, Xin-Xing, and Pei-Yang Guo. Fatigue Driving Detection Methods Based on Drivers Wearing Sunglasses. IEEE Access, vol. 12, 2024, pp. 70946-

   62. DOI.org(Crossref),https://doi.org/10.1109/ACCESS.202 4.3394218.

8. Khalil, Hady A., et al. Low-Cost Driver Monitoring System Using

   Deep Learning. IEEE Access, vol. 13, 2025,

   61. DOI.org(Crossref),

   https://doi.org/10.1080/15389588.2019.1622005.

9. D. Patil, V. Lokhande, P. Patil, P. Patil, and S. Gaikwad, Real-time driver behaviour monitoring system in vehicles using image processing, International Journal of Advances in Engineering and Management (IJAEM), vol. 4, no. 5, pp. 18901894.[Online].Available:https://doi.org/10.35629/5252

   -040518901894

10. Schwarz, Chris, et al. The Detection of Drowsiness Using a Driver Monitoring System. Traffic Injury Prevention, vol. 20, no. sup1, Jun. 2019, pp. S157

61. DOI.org(Crossref),https://doi.org/10.1080/15389588.2019.1622005