Helmet and Number Plate Detection

Helmet and Number Plate Detection

Damini Dhawan1, Mohd Shariq2, Vishal Kumar3, Somesh Singh4, Mohd Talha5

1,2,3, 4,5Computer Science and Engineering Department,

MIT, Moradabad, India

daminidhawan414@gmail.com1 mohdsharikdpr2@gmail.com2

vishalchauhn110011@gmail.com3 someshsingh0312@gmail.com4 mohd2003talha@gmail.com5

Abstract

This research paper presents a novel real-time system for detecting helmet usage violations and extracting vehicle number plates using advanced computer vision techniques. Leveraging state- of-the-art deep learning models, specifically YOLOv11 for object detection and Optical Character Recognition (OCR) for license plate recognition, the proposed system aims to enhance road safety by automating the enforcement of helmet laws. The methodology incorporates robust data augmentation strategies and few-shot learning to ensure high accuracy under diverse environmental conditions. Experimental results demonstrate the system's effectiveness, achieving precision rates of 93% for helmet detection and 89% for number plate extraction. This automated solution not only streamlines traffic monitoring but also facilitates timely interventions to improve compliance with safety regulations. The findings indicate that integrating computer vision technologies can significantly reduce reliance on manual enforcement, contributing to safer roadways and enhanced public awareness regarding helmet usage. Future work will focus on optimizing the system for broader applications and improving recognition accuracy across varied contexts.

Keywords

YOLOv11, PaddleOCR, MySQL, Safety, Motorcyclists, Image Processing, Video Processing

1. Introduction

Road safety is a global concern, with motorcycle accidents being a significant contributor to traffic-related fatalities. Among the leading causes of these fatalities is the failure of motorcyclists to wear helmets, which increases the risk of severe head injuries and deaths. According to recent studies, wearing helmets reduces the risk of head injuries by 69% and fatalities by 37%. Despite these statistics, helmet law enforcement remains a challenge due to the reliance on manual monitoring methods, which are labour-intensive and prone to inefficiencies.

To address this issue, advancements in computer vision and deep learning have paved the way for automated systems capable of detecting helmet violations and identifying violators in real- time. These systems leverage advanced object detection algorithms such as YOLO (You Only Look Once) and Optical Character Recognition (OCR) technologies to monitor traffic, detect non-compliance with helmet laws, and extract vehicle number plates for identification.

The integration of such systems into traffic surveillance infrastructure offers several benefits:

• Real-Time Monitoring: Automated detection allows for immediate identification of violations without human intervention.

• Improved Accuracy: Deep learning models like YOLOv11 achieve high precision in detecting helmets and extracting license plate details under diverse conditions.

• Scalability: These systems can be deployed across multiple locations, reducing the burden on law enforcement agencies.

This paper proposes a real-time system for helmet detection and number plate extraction using state-of-the-art computer vision techniques. By combining YOLOv11 for object detection and OCR for license plate recognition, the system aims to enhance road safety by automating helmet law enforcement. The study focuses on:

1. Developing a robust detection framework capable of handling diverse environmental conditions such as varying lighting, occlusions, and camera angles.

2. Ensuring high accuracy in detecting helmet violations and extracting vehicle registration details for effective law enforcement.

The proposed solution contributes to reducing road fatalities by promoting helmet compliance and streamlining traffic monitoring processes. Through this research, we aim to demonstrate how cutting-edge computer vision technologies can be leveraged to create safer roads and improve public awareness regarding helmet usage.

2. Review of Literature

J. Chiverton, "Classifying helmet presence with motorcycle tracking and detection." IET (2012) The background subtraction method was used in this paper by J. Chiverton to separate the biker's background, separate the biker's head, and identify the helmet's features. In the previous years, numerous algorithms and models have been used for helmet detection. It employs the following methods:

1. The detection of helmets. 2. Tracking and detecting motorcycles.

   The motorcyclist is not alerted by this system; it only recognizes helmets and license plates.

   Z. Chen, "Detection and tracking of vehicles" in Proceedings of the IEEE Int. In this paper, Z. Chen suggested a system like chosen to employ a self-adaptive Gaussian mixture model for background subtraction and a multi-dimensional Gaussian kernel density transform. In contrast to the GMM provided in the paper, the improved GMM algorithm presented in this paper is less sensitive to unforeseen changes in the global illumination.er.

   It utilizes a self-adaptive GMM for the background model and a spatiotemporal Gaussian smoothing algorithm. A helmet recognizes geometrical features and can identify any other object as a helmet.

   C. Patel's "Automatic Number Plate Detection System," In this a paper, C. Patel presented a system that can produce encouraging results for number plate detection by evaluating trichromatic imaging using a color-discrete characteristic approach.1. One of the features is number plate detection.2. Being able to identify the number plate's characters

   One significant aspect of our visual experience is color. It can impact our hunger and mood, change the way we interpret the world, and even have symbolic meaning for certain individuals.

   1. Motivation

      Ensuring that wearing a helmet increases road safety and lowers the number of head injuries sustained in collisions, while number plate recognition enforces traffic laws and andurity.

      Law enforcement is less burdened by automation, which improves the efficiency and economy of policing.

      Automated system save money by eliminating the need of additional temporary staff and reducing malpractice opportunities.

   2. Related Work

      The detection of helmet usage and the extraction of vehicle number plates have been extensively studied in recent years, particularly with the advancements in computer vision and deep learning technologies. Below is a detailed review of related works focusing on helmet detection systems and their integration with object detection frameworks.

      The detection of helmet usage and the extraction of vehicle number plates have been extensively studied in recent years, particularly with the advancements in computer vision and deep learning technologies. Below is a detailed review of related works focusing on helmet detection systems and their integration with object detection frameworks.

      1. Helmet Detection Using Deep Learning

         1. YOLO-Based Approaches

            YOLO-LHD (Lightweight Helmet Detection):

            A lightweight framework based on YOLOv11 was proposed to detect helmets in complex industrial environments. The model incorporates a Coordinate Attention Mechanism and Focal Loss Function, improving its ability to detect small objects in intricate backgrounds. It achieved a 94.3% mAP50 with only 0.86M parameters, demonstrating high precision and efficiency for real-time deployment.

            YOLOv11 for Real-Time Detection:

            This approach utilizes a customized YOLOv11 algorithm to detect helmet usage in real-time. By integrating spatial attention mechanisms, the model achieves robust performance under varying lighting conditions and occlusions, making it suitable for dynamic environments like traffic monitoring.

            YOLOv11 and MobileNet:

            A study combined YOLOv11 with MobileNet to enhance accuracy and speed for detecting safety helmets. The model outperformed newer algorithms like YOLOv5 and YOLOX on specific datasets, highlighting its suitability for real-time applications where computational resources are limited.

         2. Lightweight Models GhostNet-Based Helmet Detection:

            GhostNet was used as the backbone for a lightweight helmet detection framework. By employing multiscale segmentation and feature fusion networks, the system demonstrated improved

            robustness against distance changes, viewpoint variations, and occlusions. It achieved high accuracy while maintaining low computational costs, making it ideal for real-world applications.

            SSD-MobileNet:

            A lightweight solution using SSD-MobileNet was developed for detecting safety helmets at construction sites. The model achieved satisfactory accuracy by leveraging convolutional neural networks (CNNs) for feature extraction from diverse datasets.

      2. Traditional Image Processing Techniques

         Earlier methods relied on image processing techniques such as color segmentation, edge detection, and morphological operations to identify helmets. For example:

         Che-Yen Wen et al. (2003) enhanced the Hough Transform for helmet detection, achieving moderate success under controlled conditions.

         Li et al. (2018) used HSV transformation and adaptive thresholding for helmet recognition but faced challenges in adapting to complex backgrounds or varying lighting conditions.

         While these methods were cost-effective, they lacked the robustness and adaptability of modern deep learning approaches.

      3. Number Plate Recognition Systems

         The integration of Optical Character Recognition (OCR) with object detection frameworks has enabled automated number plate recognition:

         Traditional methods like edge detection struggled with low-resolution images and complex fonts.

         Modern approaches leverage deep learning-based OCR tools combined with YOLO for license plate localization and text extraction, achieving higher accuracy in diverse conditions.

      4. Challenges Addressed by Recent Studies

         Recent works have focused on overcoming challenges such as:

         Small Object Detection: Incorporating attention mechanisms (e.g., Coordinate Attention) to enhance the detection of small targets like helmets in crowded scenes.

         Model Efficiency: Using lightweight architectures like GhostNetv2 to balance model size and accuracy for real-time applications.

         Occlusions and Complex Backgrounds: Employing multiscale feature fusion networks to improve robustness against occlusions and varying viewpoints.

      5. Comparative Analysis of Algorithms

   3. Proposed Methodology

      The proposed methodology for real-time helmet detection and number plate extraction leverages advanced computer vision techniques, specifically the YOLOv11 object detection framework and Optical Character Recognition (OCR) for license plate recognition. This section outlines the system architecture, data collection, model training, and implementation details.

      1. System Architecture

         The system consists of two main modules:

         Helmet Detection Module:

         • Utilizes YOLOv11 for detecting motorcyclists without helmets in real-time video feeds.

         • Incorporates enhancements like Coordinate Attention Mechanism and Focal Loss Function to improve detection accuracy for small objects in complex backgrounds.

           Number Plate Extraction Module:

         • Employs OCR to extract alphanumeric details from localized license plates detected by YOLOv11.

         • Integrates preprocessing techniques to handle low-resolution or occluded plates.

      2. Data Collection and Preprocessing

         1. Helmet Detection Dataset

            • A curated dataset containing images of motorcyclists with and without helmets was created using traffic surveillance footage and public sources.

            • The dataset includes diverse conditions such as varying lighting, weather, and camera angles to ensure robustness in real-world scenarios.

         2. License Plate Dataset

            • Images of license plates from different regions were collected to account for variations in fonts, languages, and resolutions.

            • Synthetic data generation techniques were employed to simulate challenging conditions like motion blur and partial occlusions.

         3. Data Augmentation

            • Techniques such as flipping, rotation, brightness adjustments, and cropping were applied to enhance model generalization under diverse environmental conditions.

      3. Model Training

      1. Helmet Detection Using YOLOv11

         The YOLOv11 model was fine-tuned on the helmet dataset with several optimizations:

         • Coordinate Attention Mechanism: Improves detection of small targets by focusing on relevant spatial features.

         • GhostNetv2 Integration: Reduces model complexity while maintaining high accuracy, making it suitable for lightweight deployment.

         • Multiscale Feature Processing: Enhances detection performance across varying distances and perspectives.

      2. License Plate Recognition Using OCR

         OCR models were trained using annotated license plate datasets to extract text accurately under challenging conditions such as low resolution or non-standard fonts.

      3. Loss Functions

         • Focal-CIOU Loss: Used in the helmet detection module to improve performance on hard-to-detect samples while reducing false positives.

         • Normalized Wasserstein Distance Loss (NWD): Applied during regression to minimize localization errors for small targets like helmets.

      1. Real-Time Implementation Workflow

         • Input Data Processing: Video frames from traffic surveillance cameras are processed sequentially using OpenCV libraries.

         • Helmet Detection Module: YOLOv11 identifies motorcyclists without helmets by generating bounding boxes around detected objects in each frame.

         • License Plate Localization: YOLOv11 further localizes license plates of violators within the detected bounding boxes.

         • OCR Processing: Extracted license plate regions are passed through PaddleOCR for alphanumeric recognition, storing violator details (image, license plate number, timestamp) in a database for law enforcement action.

      1. Key Innovations in the Proposed Methodology

         • Lightweight Deployment: By integrating GhostNetv2 into the backbone network, the system achieves a significant reduction in model size while maintaining high precision (94.3% mAP50).

         • Enhanced Small Object Detection: Incorporation of Coordinate Attention Mechanism improves detection accuracy for small targets like helmets in complex scenes.

         • Multiscale Feature Fusion: Enables robust performance across varying distances and perspectives by introducing high-resolution features into the feature fusion network.

   4. Model Training for Real-Time Helmet Detection Using YOLOv11

      The model training process for real-time helmet detection using the YOLOv11 framework involves several key steps, including dataset preparation, model configuration, training, and evaluation. This section outlines the methodology for effectively training a YOLOv11 model to accurately detect helmets in various environments.

      1. Dataset Collection and Preparation

         1. Data Sources

            A robust dataset is critical for training the YOLOv11 model effectively. The datasets can be sourced from various platforms, including:

            • Kaggle: Datasets containing annotated images specifically for helmet detection, categorized into "With Helmet" and "Without Helmet" classes.

            • Roboflow: Offers pre-annotated datasets that can be directly used for training YOLO models.

         2. Data Annotation

            Images must be annotated to define the bounding boxes around helmets. This can be done manually using annotation tools or by utilizing existing annotated datasets. The YOLO format requires an image file paired with a text file containing the coordinates of the bounding boxes.

         3. Data Augmentation

            To enhance the model's robustness and address potential class imbalances (where images of one class significantly outnumber another), data augmentation techniques are applied:

            • Flipping: Horizontally flipping images to create mirror images.

            • Rotation: Rotating images at various angles.

            • Brightness Adjustment: Modifying the brightness to simulate different lighting conditions.

            • Cropping and Resizing: Randomly cropping images to focus on different areas.

      Figure 1: Motorcyclist without Helmet

      Figure 2: System Architecture Design

   5. Dataset

      The dataset plays a critical role in training and evaluating deep learning models for helmet detection and number plate extraction. A well-prepared dataset ensures the robustness, accuracy, and real-world applicability of the system.

   6. Dataset Statistics

      The dataset statistics for helmet detection provide critical insights into the structure, size, and diversity of datasets used to train deep learning models for safety compliance monitoring. Below is a detailed analysis of key datasets, their characteristics, and class distributions.

   7. Results

      The results of recent studies and implementations of helmet detection systems using the YOLOv11 algorithm demonstrate significant advancements in accuracy and efficiency.

      1. Improved YOLOv11 Safety Helmet Detection Network

         Study Overview: This research focused on enhancing the YOLOv11 model to improve helmet detection in complex industrial environments.

         Key Findings:

         • The modified network achieved a mean Average Precision (mAP) of 92.0%, surpassing traditional detection networks in accuracy and recall.

         • Innovations included the introduction of a Dilation-wise Residual Attention Module and Atrous Spatial Pyramid Pooling, which improved multiscale feature processing and enhanced the model's ability to detect small targets against complex backgrounds.

         • The study emphasizes the importance of addressing misdetection and omission issues prevalent in traditional methods, particularly in environments with varying distances.

      2. Automated Helmet Detection System Using YOLOv11

         Study Overview: This project aimed to develop an automated helmet detection system for two- wheeler riders in India, focusing on improving road safety amidst rising accident rates.

         Key Findings:

         • The system demonstrated a 96.5% mAP with a detection speed of 27 frames per second (fps), showcasing superior performance compared to previous models.

         • The model was trained on a carefully curated dataset generated via the Robo Flow platform, incorporating Convolutional Neural Networks (CNN) for enhanced learning capabilities.

         • Ongoing refinements aim to further improve accuracy and bounding box precision, highlighting the system's potential impact on reducing road accidents.

      3. YOLOv11 for Real-Time Helmet Detection

         Implementation Overview: A project focused on implementing helmet detection using YOLOv11 for images and videos, providing scripts for easy deployment.

         Key Findings:

         • The project emphasizes ease of use with a straightforward input method for folder paths containing images or video files.

         • It demonstrates the versatility of YOLOv11 in handling real-time detection scenarios effectively.

   8. References

1. R. R. V. e. Silva, K. R. T. Aires and R. d. M. S. Veras, Helmet Detection on Motorcyclists Using Image Descriptors and Classifiers, 2014 27th SIBGRAPI Conference on Graphics, Patterns and Images, Rio de Janeiro, 2014, pp. 141- 148.

2. Li, J., Liu, H., Wang, T., Jiang, M., Wang, S., Li, K., Zhao, X. (2017, February). Safety helmet wearing detection based on image processing and machine learning. In Advanced Computational Intelligence (ICACI), 2017 Ninth International Conference on (pp. 201-205). IEEE.

3. K. Dahiya, D. Singh and C. K. Mohan Automatic detection of bike-riders without helmet using surveillance videos in real-time, 2016 International Joint Conference on Neural Networks (IJCNN),

   Vancouver, BC, 2016, pp. 3046- 3051

4. C. Vishnu, D. Singh, C. K. Mohan and S. Babu, Detection of motorcyclists without helmet in videos using convolutional neural network, 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, 2017, pp. 3036- 3041.

5. Adrian Rosebrock, Basic motion detection and tracking with Python and OpenCV.https://www.pyimagesearch.com/2015/05/25/basicmotiondetectionandtracking-with-python- and-open.

6. J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei, \"ImageNet: A Large-Scale Hierarchical Image Database,\" in IEEE Conference on Computer Vision and Pattern Recognition, 2009.

7. A. Krizhevsky, I. Sutskever, and G. Hinton, \"ImageNet Classification with Deep Convolutional Neural Networks,\" in Advances in Neural Information Processing Systems, 2012.

8. Y. LeCun, Y. Bengio, and G. Hinton, \"Deep Learning,\" in Nature, 2015.

9. G. Huang, Z. Liu, K. Q. Weinberger, and L. van der Maaten, \"Densely Connected Convolutional Networks,\" in IEEE Conference on Computer Vision and Pattern Recognition, 2017.

10. T. Mikolov, K. Chen, G. Corrado, and J. Dean, \"Efficient Estimation of Word Representations in Vector Space,\" in Proceedings of the International Conference on Learning Representations, 2013

11. A. Rajkomar, E. Oren, K. Chen, et al., \"Scalable and Accurate Deep Learning for Electronic Health Records,\" in npj Digital Medicine, 2018.