VAPP: Student Facial Emotion Recognition in Virtual Learning Platform

VAPP: Student Facial Emotion Recognition in Virtual Learning Platform

Godwin Anna John Dept. of Comp Science &

Engineering

Mangalam College of Engineering Kottayam, India

Lekshmi T.R

Dept. of Comp Science & Engineering

Mangalam College of Engineering Kottayam, India

Meera Anna Joseph Dept. of Comp Science &

Engineering

Mangalam College of Engineering Kottayam, India

Sabna Nazar

Dept. of Comp Science & Engineering Mangalam College of Engineering Kottayam, India

Vishnu S Shekar

Dept. of Comp Science & Engineering Mangalam College of Engineering Kottayam, India

AbstractCoronavirus pandemic has upset schooling in more than 186 nations and impact 1.2 billion students. The gaining misfortunes emerging from the Coronavirus pandemic might convert into much more prominent long-term difficulties or adverse consequence on many youngsters' future, so the most ideal way to keep learning is far off learning. The significant test of remote learning is the absence of cooperation among educators and understudies. So, our point is to conquered the test and to look at the understudies feeling during the growing experience. For this we utilize a learning feeling acknowledgment model, which comprises of three phases: Element extraction, subset element and feeling classifier. The AI based approach called Haar method is utilized for object location and afterward through the Sobel edge discovery to acquire the trademark esteem. Utilizing a Brain Organization classifier preparing, six sorts of various enthusiastic classifications are obtained. This feeling order can assist educator with examining the understudies feelings and change the showing procedures as per understudies feeling.

KeywordsEmotion recognition, Facial recognition, Virtual learning

1. INTRODUCTION

   Emotion recognition is a technique that detect different facial expressions and also process and examine on a human's face. It recognizes emotions and it is done based on facial recognition technique.

   Virtual learning approach is now widely used in almost everywhere especially during the COVID-19 breakout. The main problem we face is whether the student is interested or not in the online teaching class. To overcome this, we introduce an emotion recognition system in which we use Haar cascade method and Sobel detection to detect facial features and Neural network method to identify different kinds of emotions. Thus, helping teachers to understand

   student emotion and maintain teaching strategies according to it. From the experiments it is clear that the proposed application is very accurate and efficient in recognising students emotions in real- time.

2. RELATED WORKS

   EEG (electroencephalogram) is a method to detect electrical activity in our brain. In this paper it uses EEG signal in DEAP dataset is used to classify the emotions. DEAP is a dataset that containing EEG signals for emotion analysis. To reduce the dimensions of pre-processed EEG data, principal component analysis is used, so that main feature can be obtained. These features are then classified using a convolution neural network (CNN). Convolution neural network are widely used for image based task.[1]

   Facial recognition is mapping of human face through technology. This paper proposes a system for a real time facial image recognition using Raspberry pi. Haar-cascade and HOG (histogram of oriented gradients) methods are used for facial recognition. Haar method is implemented on Raspberry pi using pi camera for the real time capturing of image. Both the methods are then compared using Eigen feature based recognition algorithm.[2]

   Facial expression is non-verbal communication. This paper deals with a facial expression recognition using decision tree. Facial expression can be performed in three steps: face detection, feature extraction and classification of expression. Using a geometrical approach calculate the six distances in order to measure parts of face to describe the facial expression. JAFF and COHEn database are applied in decision tree.[3]

   Machine learning and Deep learning are the emerging technologies that can classify data, pattern recognition etc. This paper proposes a facial emotion recognition using deep convolution network (DCNN). DCNN are used to identify pattern in images and video. Dataset used for proposed model is collected from a mobile phone camera. Convolution neural network is then trained using the dataset

   that contain five emotions, utilizing Adam optimization algorithm and categorized cross entropy as loss function, this model has comparable training accuracy and validation accuracy.[4]

3. PROPOSED SYSTEM

   The objective of the proposed system is to detect the emotions of student, while learning and classify the emotions and plot the feedback to the instructor. For the detection of input image we use Haar-Cascade method, by this we can extract the eyes and mouth feature. To obtain the characteristic value we use Sobel edge detection method. JAFF database is used for training the neural network and compared it with the extracted feature and predict that the learner are interested in topic or not and gave the feedback to the instructor in real time.

   1. Input Image

      • Continuous information pictures on a video outline are utilized in this proposed framework. Face of individual in input pictures should be completely or to some degree be seen on the video outline.

      • The framework requires a sensible number of pixels and a satisfactory measure of brilliance for handling.

   2. Preprocessing Stage

      In the pre-handling stage, we just crop the eye location and grab the characteristics esteem. This can cause bring down the exactness, to further develop accuracy we extract the mouth features using Haar-cascade method.

   3. Face Detection and Extraction Stage

      Haar cascade is an Item recognition calculation used to distinguish faces in a picture or a constant video. Using Haar cascade strategy we check that the picture is available. Then, at that point, we extract the eye and mouth features from the info picture. To obtain characteristics value we use Sobel edge detection.

   4. Classification Stage

      The JAFFE dataset comprises of 213 pictures of various looks from 10 changed Japanese female subjects. This JAFF database is used for training the neural network and compared it with the extracted features and classify the emotions.

   5. Feedback System

   Based on the emotion we can comprehend the student is keen on learning or not and consequently gave a feedback alert to teacher in real time. With this goal they can change the teaching strategy according to student emotions.

   1. Captur Image

      The webcam or camera catches pictures which are then utilized as a dataset to prepare the model.

   2. Face Detection and Feature Extraction

      Haar cascade algorithm is used for face detection and Sobel detection is used for characteristic value.

   3. Classifier Training and Test

      Using neural network, we train the feature extraction.

   4. Detecting Emotion

      Comparing the JAFF database and extracting feature. We detect the emotions.

   5. Feedback System

      Based on the emotion a feedback alert system is given.

   6. Instructor

   Instructor is the teacher who teaches the user.

   A. User

4. SYSTEM ARCHITECTURE

   Fig1. System Architecture

   User refers to the student who is attending theclass.

5. RESULT

   This proposed work utilizes a Haar cascade and Sobel edge detection for recognizing and extracting features of an input image. Then this extracted features compared with trained neural network for the classification of emotions of students. As the sudden breakout of covid 19 all the education institutions are closed. Teachers who are expert in blackboard, chalk, classrooms teaching are really new in this digital platform, so they are really confusing whether the students can understand the topic or not. So, our proposed system is really helpful to understand the students emotions thereby teachers can change their teaching strategies.

6. CONCLUSION

   This is a furnished solving trouble of an emotion reputation version based on facial recognition in virtual gaining knowledge of surroundings. The emotion reputation machine we use Haar cascade approach and neural network method. Then conversation among teachers and students are greater knowledge the use of emotion popularity. So coaching is more correctly for students. It is very accurate and efficient in recognition.

7. REFERENCES

[1] Guolu Cao,Yuliang Ma,Xiaofei Meng,Yunyuan Gao,Ming Meng. "Emotion Recognition Based On CNN".Proceedings of the 38th Chinese Control Conference.

[2] Seema Singh, Ramya R, Sushma V, Roshini SR, Pavithra R. "Facial Recognition using Machine Learning Algorithms on Raspberry Pi".2019 4th International Conference on Electrical, Electronics, Communication, Computer Technologies and Optimization Techniques (ICEECCOT).

[3] Salman, Madaini, & Kissi (2016) " Facial Expression Recognition using Decision Trees". IEEE International Conference on Computer Graphics, Imaging and Visualization (CGiV), 120-130.

[4] Pranav E,Suraj Kamal,Satheesh Chandran C,Supriya M.H."Facial Emotion Recognition Using Deep Convolutional Neural Network".2020 6th International Conference on Advanced Computing & Communication Systems (ICACCS).

[5] Yang, Kriegman, & Ahuja (2002) "Detecting Faces in Images: A Survey", IEEE Transactions on Pattern Analysis and Machine

Intelligence. 24( 1) 34 58

[6] Yow, & Cipolla (1997) "Feature-based Human Face Detection", Image and Vision Computing, 15(9), 713-735.

[7] Dai, & Nakano (1996) "Face-texture Model Based on SGLD and its Application in Face Detection in a Color Scene", Pattern Recognition. 29(6), 1007-1017.

[8] Yang, & Waibel (2015) "A Real-time Face Tracker", Proceedings of 3rd Workshop Applications of Computer Vision, 142-147.

[9] Durmusoglu, & Kahraman (2006) "Facial Expression Recognition Using Geometric Feautres", IEEE. DOI: 10.1109/IWSSIP.2016.7502700, 23-26.