- Open Access
- Authors : Ramya.R.S, Sujithraa.S, Vaishnavi.K.S, Vannamathi.S, Sakthi.R
- Paper ID : IJERTCONV11IS03049
- Volume & Issue : Volume 11, Issue 03
- Published (First Online): 22-06-2023
- ISSN (Online) : 2278-0181
- Publisher Name : IJERT
- License: This work is licensed under a Creative Commons Attribution 4.0 International License
Signtalk-detection And Conversion Of Sign Language Into Speech With Light Control
SignTalk-Detection and conversion of sign language into speech with light control
Ramya.R.S Department of Electronics and communication Engineering
AVS Engineering college Salem, TamilNadu, India firstname.lastname@example.org
Sujithraa.S Department of Electronics and communication Engineering
AVS Engineering college Salem, TamilNadu, India email@example.com
Vaishnavi.K.S Department of Electronics and communication Engineering
AVS Engineering college Salem, TamilNadu ,India firstname.lastname@example.org
Department of Electronics and communication Engineering AVS Engineering college
Salem, TamilNadu, India email@example.com
Department of Electronics and communication Engineering AVS Engineering college
Salem, TamilNadu, India firstname.lastname@example.org
Abstract Gesture recognition technology has become an integral part of human-computer interaction, especially for people with physical disabilities. The proposed system is a wireless glove that utilizes flex sensors for better recognition of hand gestures. The glove also includes a heartbeat sensor that monitors the user's heartbeat and sends emergency notifications using a GSM module. Using Zigbee wireless technology, information can be transmitted and received between the glove and the main kit. The system also includes audio playback with speaker and a display for better understanding of the gesture, and enables the user to control the light and fan through gesture. The battery provides the necessary power supply for the circuit. This paper describes the design and implementation of the wireless glove for gesture recognition and control with an emergency notification system.
Keywords Wireless glove, Flex sensors, Gesture recognition, Heartbeat sensor, GSM module, Zigbee wireless technology, Audio playback, Display, Light control, Fan control.
Gesture recognition technology is a rapidly growing field in the realm of human-computer interaction. The proposed system is a wireless glove that employs flex sensors for better recognition of hand gestures. The glove also includes a heartbeat sensor that monitors the user's heartbeat and sends emergency notifications using a GSM module. The system has a wide range of applications, including home automation and healthcare
Some papers used rule-based systems to convert signs into speech. These systems rely on a set of predefined rules and handcrafted features to recognize signs and generate speech. The rules are typically based on linguistic and phonological knowledge of sign language. However, these systems are limited by their inability to handle variations in signing styles and the need for manual feature engineer
The Statistical models such as Hidden Markov Models (HMMs) and Gaussian Mixture Models (GMMs) to recognize signs and generate speech. These models learn from data and do not require manual feature engineering. However, they are
limited by their inability to handle long-term dependencies in sign sequences and the need for large amounts of training data
Deep Learning-based Methods:
Deep learning-based methods such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to recognize signs and generate speech. These methods have shown promising results in Sign to Speech conversion due to their ability to learn complex representations from data and handle long-term dependencies. CNNs are particularly effective in recognizing signs from visual data, while RNNs are effective in modeling temporal dependencies in sign sequences.
The wireless glove includes flex sensors that measure the bending of the index finger, middle finger, and ring finger. These sensors are connected to a PIC microcontroller, which processes the data and recognizes the hand gesture. The system also includes a heartbeat sensor for monitoring the user's heartbeat, which is connected to the same PIC microcontroller. In case of an emergency, the system sends an alert message using a GSM module to their concerned person.
Using Zigbee wireless technology, information can be transmitted and received between the glove and the main kit. The system also includes audio playback and a display for better understanding of the gesture. The proposed system also enables the user to control the light and fan through gesture using a relay module connected to the PIC microcontroller. The system is powered by a battery, which provides the adequate power supply for the circuit.
Existing systems of sign to speech conversion can be classified into two categories: vision-based and data glove- based.
Vision-based systems use cameras to capture the sign language gestures and then process the image data to recognize the signs. These systems can be further classified into two types: marker-based and marker less. Marker-based systems use special markers or gloves with markers on them to track the hand movements. Marker less systems do not
require any markers and use computer vision techniques to track the hand movements.
Data glove-based systems use gloves with sensors to capture the hand movements and then process the data to recognize the signs. These systems are more accurate than vision-based systems but are also more expensive..
Its a wireless glove. Here the flex sensors are used for recognizing the hand gestures in better manner.The Glove contains flex sensor and heartbeat sensor which are connected with PIC microcontroller.The heartbeat sensor monitors the heartbeat of the person in case of any emergency, using GSM module the information will send to their concerned person.Using Zigbee wireless technology, information can be transmitted and received between the glove and the main kit. For better understanding of the gesture, the audio playback with speaker and a display are provided.This System also enable the user to control the light and fan through gesture.The Battery provides the adequate power supply for the circuit.
Fig. 1 shows the complete block diagram of the system, Here the flex sensor and the heart beat sensor are connected to the microcontroller, the sensor information will send to the microcontroller, this information further sent to the micro controller present in the kit. The sensor information are processed and the necessary command are provided using the speaker and LED display. The heart beat sensor measures the heartbeat, if the heart beat reaches the threshold value then, the emergency alert will send to the concerned person through GSM module.
Fig. 1. Transmitter block diagram
Fig. 2. Receiver block diagram
These are sensors that measure the degree of bending of the fingers. In this system, flex sensors are used to recognize hand gestures. The glove contains several flex sensors that are placed at different points on the fingers and thumb. When the user moves their fingers, the flex sensors detect the degree of bending and send signals to the microcontroller.
Fig. 3. Flex Senor
This technology is used to transmit and receive information between the glove and the main kit. The main kit contains a receiver module that receives the signals from the glove and sendsthem to the microcontroller for processing. The receiver module also sends signals to the glove to control the light and fan.
Fig. 4. Zigbee module
This is the central processing unit of the wireless glove system. The microcontroller receives signals from the flex sensors and heartbeat sensor, and it processes the data to recognize the user's hand gestures and heartbeat. The microcontroller also controls the other components of the system, such as the audio playback and light control.
Fig. 5. PIC microcontroller
This sensor measures the user's heartbeat and is used to send emergency information in case of any medical emergencies. The heartbeat sensor is connected to the user's index finger, and it constantly monitors their pulse. If the sensor detects any abnormalities in the user's heartbeat, it sends a signal to the microcontroller, which then triggers the
GSM module to send an emergency message to the user's designated contacts.
Fig. 6. Heart beat sensor
This module is used to send emergency messages to the user's designated contacts in case of any medical emergencies. When the heartbeat sensor detects an abnormal heartbeat, the microcontroller triggers the GSM module to send a message to the user's contacts, informing them of the emergency and the user's location.
Fig. 7. GSM module
The proposed system was tested and evaluated for gesture recognition and control. The system was able to recognize and differentiate between various hand gestures with high accuracy. The system also successfully controlled the light and fan through gesture. The system was also able to monitor the user's heartbeat and send emergency information using GSM Module. The Wireless glove proved to be an effective solution for gesture recognition and control.
The proposed wireless glove system is an efficient and effective solution for gesture recognition and control. The System uses flex sensors for recognizing hand gestures, and a heartbeat sensor for monitoring the users heartbeat. The system also includes audio playback and a display for better understanding of the gesture, and enables the user to control the light and fan through gesture. The system has potential application in home automation and healthcare. The proposed system proves to be an effective solution for gesture recognition and control, with high accuracy and reliability.
gesture segmentation and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1. M. D. Ding, and L. Lin, CNN and HOG Dual-Path Feature Fu- sion for Face Expression Recognition, Information and control, vol. 49, no. 1, pp. 4754,
2020. C. C. D. Santos, J. L. A. Vassallo and R. F. Vassallo, Dynamic Gesture Recognition by Using CNNs and Star RGB: a Temporal Information Condensation, Neurocomputing, vol. 400, pp. 238 254, 2020.  H. Zheng, R. L. Wang and W. T. Ji et al, Discriminative deep multi-task learning for facial expression recognition, Information Sciences, vol. 533, pp. 6071, 2020.  https://en.wikipedia.org/wiki/Sign_language accessed on 20 Mar 2023  https://www.elprocus.com/flex-sensor-working-and-
its-applications/ accessed on 25 Mar 2023 https://www.elprocus.com/introduction-to-pic- microcontrollers-and-its-architecture/ accessed on 28 Mar 2023