An Integrated Approach Real-Time Face Recognition System using Haar Cascade Classifiers and Eigen faces

An Integrated Approach Real-Time Face Recognition System using Haar Cascade Classifiers and Eigen faces

Md.Tanvir Chowdhury

East West University, Bangladesh mdtanvirchowdhury015@gmail.com

Abstract in institutional security, there is an aspect to being able to identify people accurately and promptly from video clips. Typical methods in this aspect count on human operators to spot people through recorded surveillance video footage. Unfortunately, this technique is not just labor intensive but also questionable to human inaccuracy. A tool will be reliable in this case to control safety by verifying people rapidly and precisely. In this project, we strived to evolve an ingenious software aiming to recognize persons within the bounds of institutions through recorded surveillance video footage and also to make the procedure more intelligible. We implemented recently developed algorithms to perfectly match faces from the supervised footage of the database where the images of the people of a particular institution are stored. After augmenting the initial dataset, we achieved our final images. We used Haar Cascade, LBPH, and Eigenface methods for performing the specific identifying task on our dataset. We added multiple layers to the models and developed them for eye-catching accuracy

Keywords Recognition, Acknowledgeable, biometric, security, recognize, face detection

1. INTRODUCTION

   Face is one of the most individualized and simply acknowledgeable biometric matrices. In modern world security is really very important for us. We are already depending on various technologies; those are there for making our life easier and secure. In this project we made an innovative software system which will detect a person and their identity within an institution. We applied machine learning to reach our goal and make it more efficient. Our system detects person by checking from its database and clarify that, if this person is from the institution or an outsider. Additionally, we have the facility to detect it in prior stage. We are using facial recognition because it is one of the most significant biometric features by which we can detect the correct person effectively. Now a days almost all of us use recorded surveillance video footage in our institutions. However, it is also important to detect the problems or inaccurate behaviors along with identification of any susceptive person. Our system will sort this process and, by using it people will be able to experience the real facilities of using recorded surveillance video footage for leading a secure life with immediate identification. We assure high accuracy as it is very important for increasing the reliability of our system. Face detection algorithms often require significant computational resources, which makes the process of face detection challenging. Many face recognition systems are designed to handle one face at a time. While recognizing a single face typically takes a short amount of time, identifying multiple faces sequentially using the same approach can be time-consuming. Therefore, it is crucial to develop a system capable of recognizing multiple faces simultaneously, to accelerate the recognition process. This paper explores a hybrid approach for multiple face recognition by combining the Haar Cascade and Eigenface

   methods. The objective is to enhance the performance and efficiency of the face recognition process by leveraging these techniques.

2. RELATED WORK

   The related research for identifying and evaluating convolutional neural network-based person detection models has been analyzed in this part. We have looked into a few cases to enlarge our comprehension of identifying persons, and they fall within the suitable classification. For the purpose of conducting a literature study, we explored some current studies in relevant fields

   The authors of this paper proposed an intellectual facial recognition system using Eigen Face values for criminal detection and recognition on CCTV data using criminal database where the identification of the suspect is done by face matched. They claimed their system would be able to detect face and recognize face automatically [1]. The researcher of this task proposes an algorithm for implementing an automated attendance management system for students of a class by making use of face recognition technique, by using Eigen face values, Principal Component Analysis (PCA) and Convolutional Neural Network (CNN) [2]. The experimenters of this research developed a face recognition based surveillance system using FaceNet and MTCNN on Jetson TX2. It is a portable system that tracks the subject or the suspect with the camera ID/location together with the timestamp and locates his/her presence in the database, using multiple camera installations [3]. The authors of this paper did study on this area and proposed a face recognition process using a hybrid process of Haar Cascades and Eigen face methods, it can detect multiple faces (55 faces) in a single detection process [4]. In this study, the authors presented a comprehensive overview of the recent innovations in face anti-spoofing state, analyzing previously existing methodologies, available benchmarking databases, reported performance results and, more importantly, the current issues and future research goals. The initial studies using deep convolutional neural networks (CNNs) have resulted in excellent intra-database test performance [5]. The intention of the researchers of this paper was to deep analysis of face detection using open CV. It is known from this research that face recognition involves three basic steps which include face detection, recognition, and extraction. Any system needs to encapsulate the image and manage as well as record the vital features to find out the location of the face. For the recognition of the captured image the records are kept of various features such as skin color, tone etc. [6].

   The researchers of this paper designed and evaluated a real-time face recognition system using CNN. They obtained maximum recognition of an eye-catching accuracy from the proposed system on using AT&T and real-time inputs respectively [7]. The authors developed a human face detection algorithm via Haar cascade combined with three additional classifiers – skin hue histogram matching, eyes detection and mouth detection. The proposed human face detection system is simple to implement due to availability of modules in Open CV. An amazing accuracy has achieved by the proposed detection system [8]. [9] The authors of this paper focused on a System for Criminal Detection and Recognition on CCTV Data Using Cloud and Machine Learning using – HAAR, Eigen Faces, Cam shift, CNNs, Viola-Jones Algorithm, Gaussian, Euclidean distance, AdaBoost and other classifiers. The most famous detection algorithm they came across through their research was HAAR [9]. The researchers reviewed various face detection techniques. They presented a comprehensive survey of various techniques explored for face detection in digital images using SVM, Neural Network, PCA, Eigen faces Method [10].

   the training data undergoes a series of movement, incorporating face detection, feature extraction, and the key face recognition process to generate results. These results are kept in a database, where the training data is subsequently employed for the face recognition process. Throughout the face detection process the webcam will detect the face by putting boxes to the detected faces.

3. METHODOLOGY

   The face recognition process for multiple faces, as proposed in this study, is illustrated in Figure 1. The process begins with the preprocessing of training data, where RGB images are converted to grayscale and their color depth s reduced to 8 bits. Following this, the Haar Cascade method is applied to the images, involving the Haar feature extraction phase, the creation of an integral image, and the use of a cascade classifier

   Figure 1 Face recognition method

   The subsequent step is the feature extraction process. These key features are vitally important for face recognition. The features are tend to be identified from the preprocessed images. Two methods, Eigenface method and Principal Component Analysis (PCA), after merging is used to extract features from the training data. Feature extraction is executed straightly using PCA for the test datasets.

   Later then, the recognition process begins. At this stage, the PCA-transformed training data (PCA_train) and test data (PCA_test) enage in resemblance measurement through using the Euclidean distance method. Based on the Euclidean distance calculations the recognition result (i.e., the identified face) is decided.

   Going on, the execution of the initial face recognition process, demonstrated in Figure 2, begins with capturing images through a webcam to get the training data. After that,

   Figure 2 Implementation method of the face recognition system.

   A. Haar Cascade Technique

   This technique begins with the Haar feature phase, where face detection is carried out by distinguishing more effectively between "face" and "non-face" regions.

   As depicted in Figure 3, certain areas of the image contain rectangular features that are either dark (black) or bright (white). Using these rectangles, Haar-like features are calculated. The calculation involves finding the difference between the sum of pixel values in the dark region and the sum of pixel values in the bright region, as represented by the following formula:

   F (Haar) = White Black

   Where,

   White = Sum of pixels of the bright area

   Black = Sum of pixels of the dark area

   When the Haar-like feature is higher than a certain threshold, it can be stated that a face or faces are within the area. To efficiently filter a high number of faces within the image, an integral image technique is used.

   Figure 3 An image with Haar-like feature

   The Haar feature is calculated using the Integral Image technique, which enables rapid computation of feature values by transforming each pixel into a new image representation. The integral image value is determined by accumulating the pixel values from the top-left corner up to a specific point. For instance, the pixel at position (a,b)(a, b) has an accumulated value derived from all pixels in the area up to position (x,y)(x, y). After applying the integral image transformation, the pixel at (x,y)(x, y) contains the sum of all pixel values within the corresponding shaded rectangle. The pixel values within rectangle D can be calculated using the formula: ((x4, y4) +(x1, y1)) – ((x3, y3) +(x2, y2)) ((x4,

   y4) + (x1, y1)) – ((x3, y3) + (x2, y2)).

   Utilizing the Integral Image value, the Haar feature value is discovered. It constitutes the difference between the white and black pixel regions, recurrently referred as the threshold value. This threshold is used as a parameter to recognize whether an object in the image is a face or not. By incorporating multiple weak classifiers into a stronger classifier, the Cascade Classifier process intensifies the detection accuracy. This is achieved by allocating weights to the output of the weak classifiers, which results in sharper images. The Cascade Classifier functions as a chain of filters, where each filter performs as an individual classifier instructed with a proportionally small set of weak classifiers through a boosting process.

   Figure 4 Workflow of the Cascade Classifier

   Through the filtering process, the threshold value for each filter is vigorously set. Each filter is instructed to distribute images that have passed through the previous phase. Also, if any filter fails to pass a region, that region is allocated as "non-face." And, if the image region does not meet the standard of a strong classifier, the process recurrent with an enlarged threshold value up to the weights are modified applicably.

   Any region is identified as a face if an image successfully passes through all filters in the chain. For ensuring accuracy, Haar-like features are processed utterly. If a region fails to meet the criteria at the first stage, it is rejected. Then, the algorithm moves to the next sub-window (phase-2) and recalculates the features. Once, the result meets the threshold, the region advances to the next filter stage. And, progressively narrows the sub-window and refines the result toward the actual image value.

   At each level, the filters are instructed using a database of faces to classify the regions passed down from prior filters. While executing, if a filter fails, the corresponding region is classified as "Not Face." If the filter succeeds, the region proceeds to the next phase. Regions that pass through all filters are ultimately classified as "Face."

4. RESULT & DISCUSSION

   Experiments utilizing the proposed face recognition process have been carried out, considering various factors, to evaluate the performance of the system developed using the suggested technique.

   The first experiment examined the impact of distance on the face detection process. The results of this experiment are presented in Table 1, which indicates that face detection is successfully achieved up to a distance of 250 cm.

   Table 1 Detection Face at Various Distance

   Distance	Targeted Distance	Detection Distance
   D1	90	90
   D2	120	120
   D3	170	170
   D4	250	250

   The Second experiment aimed to determine how many different faces can be detected at once during the face recognition process. The results, shown in Table 2, indicate that out of 136 faces to be detected, 127 were successfully recognized. This results in a success rate of 93.382% for the multi-face recognition process.

   Location	Targeted Number of Faces	Detected Number of Faces
   CSE430 LAB	11	11
   CSE479 LAB	26	25
   CSE412 LAB	19	19
   CSE347 LAB	21	18
   Faculty Lounge	29	26
   Cafeteria	30	28

   Table 2 Outcomes of the multi-face recognition process

   recognizing 127 out of 136 faces. It could also accurately detect faces from distances of 250 cm and more. Exacting Security this system replaces human operators, minimizing human error and risk in environments ranging from academic institutions to business offices and government buildings.The next steps toward enhancing the system include improving detection dynamics, analyzing and segmenting body features for better identification, and training models with more extensive datasets for increased accuracy. Further developments on the system will include a more user-friendly interface for ease of operation and the installation of live alerting systems for immediate notification. This system will also be enhanced to better reach decipher faces at a distance, and at varying angles. Instead of raw webcams, we are expecting that we will yield an exponentially better result in facial recognition by using high-end cameras

   After conducting the suggested facerecognition system, the evaluations demonstrate the powerful performance of the system over various conditions, with eye catching results attained in both distance and multi-face recognition examinations. The system successfully able to detect faces up to 250 cm away, promoting its capability to function in larger, driving environments. Additionally, the system achieved a high success rate of 93.38%, recognizing 127 out

   REFERENCES

   1. Abdullah, N.A., Saidi, M.J., Rahman, N.H.A., Wen, C.C. and Hamid, I.R.A., 2017, October. Face recognition for criminal identification: An implementation of principal component analysis for face recognition. In AIP conference proceedings (Vol. 1891, No. 1). AIP Publishing.

   2. Sawhney, S., Kacker, K., Jain, S., Singh, S.N. and Garg, R., 2019, January. Real-time smart attendance system using face recognition techniques. In 2019 9th international conference on cloud computing, data science & engineering (Confluence) (pp. 522-525). IEEE.

   3. Jose, E., Greeshma, M., Haridas, M.T. and Supriya, M.H., 2019, March. Face recognition based surveillance system using facenet and mtcnn on jetson tx2. In 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS) (pp. 608-613). IEEE.

   4. Mantoro, T. and Ayu, M.A., 2018, May. Multi-faces recognition process using Haar cascades and eigenface methods. In 2018 6th international conference on multimedia computing and systems (ICMCS) (pp. 1-5). IEEE.

      of 136 faces, in multi-face recognition, which shows its productiveness in crowded settings.

      Besides, the system established resilient to alternatives in face orientation that flawlessly recognize faces even when turned away from the camera. The systems potential for real-world applications such as surveillance and access control is emphasized by the outcomes. However, some challenges remain in balancing low-light conditions and occlusions. To, further enhance the accuracy and robustness, it is recommended to integrate advanced techniques, like deep learning. Altogether, the proposed system demonstrates strong word. Especially, for large-scale deployment, and, particularly in environments requiring real-time face detection and recognition.

      The high F1-scores [12] suggest that the model achieves a favorable equilibrium between precision and recall for every class. The model exhibits a high overall accuracy, indicating its ability to accurately categorize a significant proportion of the total occurrences. When considering these measures collectively, they indicate a strong model that consistently performs well in classifying data.

5. CONCLUSION

This work proposes a new surveillance system that overcomes the limitations of conventional human-monitored systems. By utilizing Haar Cascade, LBPH, and Eigenface methods for facial recognition, the system was trained on segmented video frames collected from different angles, achieving a success rate of 93.38% and correctly

1. Li, L., Correia, P.L. and Hadid, A., 2018. Face recognition under spoofing attacks: countermeasures and research directions. Iet Biometrics, 7(1), pp.3-14.

2. Goyal, K., Agarwal, K. and Kumar, R., 2017, April. Face detection and tracking: Using OpenCV. In 2017 International conference of Electronics, Communication and Aerospace Technology (ICECA) (Vol. 1, pp. 474-478). IEEE.

3. Pranav, K.B. and Manikandan, J., 2020. Design and evaluation of a real-time face recognition system using convolutional neural networks. Procedia Computer Science, 171, pp.1651-1659.

4. Cuimei, L., Zhiliang, Q., Nan, J. and Jianhua, W., 2017, October. Human face detection algorithm via Haar cascade classifier combined with three additional classifiers. In 2017 13th IEEE international conference on electronic measurement & instruments (ICEMI) (pp. 483-487). IEEE.

5. Shirsat, S., Naik, A., Tamse, D., Yadav, J., Shetgaonkar, P. and Aswale, S., 2019, March. Proposed system for criminal detection and recognition on CCTV data using cloud and machine learning. In 2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN) (pp. 1-6). IEEE.

6. Kumar, A., Kaur, A. and Kumar, M., 2019. Face detection techniques: a review. Artificial Intelligence Review, 52, pp.927-948.

7. Khan, M., Chakraborty, S., Astya, R. and Khepra, S., 2019, October. Face detection and recognition using OpenCV. In 2019 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS) (pp. 116-119). IEEE.

8. Velasco, J., Pascion, C., Alberio, J.W., Apuang, J., Cruz, J.S., Gomez, M.A., Molina Jr, B., Tuala, L., Thio-ac, A. and Jorda Jr, R., 2019. A smartphone-based skin disease classification using mobilenet cnn. arXiv preprint arXiv:1911.07929.

9. Ancheta, R.A., Reyes, F.C., Caliwag, J.A. and Castillo, R.E., 2018. FEDSecurity: implementation of computer vision thru face and eye detection. International Journal of Machine Learning and Computing, 8(6), pp.619-624.