- Open Access
- Authors : Vinay Chhotelal Gupta , Yash Thakur , Ezra Rajendran , Sherilyn Kevin
- Paper ID : IJERTV12IS030116
- Volume & Issue : Volume 12, Issue 03 (March 2023)
- Published (First Online): 31-03-2023
- ISSN (Online) : 2278-0181
- Publisher Name : IJERT
- License: This work is licensed under a Creative Commons Attribution 4.0 International License
Hand Gesture Recognition (Unveiling the Power of Nonverbal Communication through Literature Review)
Ms. Sherilyn Kevin
Department of IT Thakur College of Science and Commerce Mumbai, India
Mr. Vinay Chhotelal Gupta Department of DS Thakur College of Science and
Commerce Mumbai, India
Mr. Yash Thakur
Department of DS Thakur College of Science and Commerce Mumbai, India
Mr. Ezra Rajendran
Department of DS Thakur College of Science and Commerce Mumbai, India
Abstract: In recent years, hand gesture recognition systems have gained significant attention due to their diverse applications and efficient interaction with machines through human-computer interaction. This paper presents a survey of recent systems for recognizing hand gestures, highlighting key issues and challenges faced in developing such systems. The review covers various methods for recognizing postures and gestures, as well as research results from different gesture recognition systems and databases. Furthermore, a comparison between the main phases of gesture recognition is provided. Finally, the advantages and drawbacks of the discussed systems are explained.
Keywords Hand gesture recognition, human-computer interaction, postures, gestures, neural networks, databases, comparison, advantages, drawbacks, key issues, challenges, surveys, recent systems, and research results.
In recent years, advancements in neural networks have revolutionized the field of hand gesture recognition systems, enabling more natural and efficient interaction between humans and machines. The aim of these systems is to create an intuitive communication channel that allows recognized gestures to control robots or convey important information . However, gesture interaction poses significant challenges such as segmentation, features extraction, and recognition. Human-computer interaction (HCI)  plays a vital role in ensuring system functionality and usability . Gestures, whether static or dynamic, have numerous applications in communication between humans and machines and sign language . With the recent advancements in neural networks, more sophisticated and accurate methods for acquiring necessary information have emerged to build gesture recognition systems. This paper presents a comprehensive review of recent hand gesture recognition systems, highlighting key issues, challenges, applications, and drawbacks. The paper is organized into different sections, starting with the key issues of hand gesture recognition systems, applications, challenges, literature review of recent systems, drawbacks, research results, and concluding remarks.
HAND GESTURE RECOGNITION:
Challenges in Feature Extraction and Extraction Methods
Gesture recognition systems typically involve three main steps, which researchers have categorized following the acquisition of an input image from a camera, video, or data glove instrumented device. These steps are extraction method, feature estimation and extraction, and classification or recognition, as depicted in Figure 1.
Fig 1. Gesture Recognition System Steps 2.1.Extraction Method and image pre-processing:
The recognition of hand gestures requires several steps, with
the first being the segmentation process, where the input image is divided into regions separated by boundaries . The segmentation process varies based on the type of gesture being recognized, whether it is dynamic or static. The hand gesture needs to be located and tracked  for dynamic gestures, while for static gestures, only the input image needs to be segmented.
The common helpful cue used for segmenting the hand is the skin color , which is easy and invariant to scale, translation, and rotation changes .However, this method is affected by illumination condition changes and different races . To overcome this problem, some researchers use data gloves and colored markers or infrared cameras . The segmentation  process is an open issue due to factors such as complex backgrounds, illumination changes, and low video quality . Preprocessing operations, such as subtraction, edge detection, and normalization, are applied to enhance the segmented hand image. Figure 2, provides examples of segmentation methods. Different tools and methods are used to model the hand, including parametric
and non-parametric techniques such as Gaussian Model (GM), Gaussian Mixture Model (GMM), and histogram- based techniques. The color space used in a specific application plays a vital role in the success of the segmentation process. Color spaces are sensitive to lighting changes, and researchers tend to use chrominance components only and neglect the luminance components, such as r-g and HS color spaces .
Fig. 2 Segmentation Method. [a) , b) , c) ]
The process of feature extraction is crucial in achieving accurate recognition of hand gestures, and relies heavily on the quality of the preceding segmentation process . A variety of methods can be used to extract features vectors from segmented images, depending on the specific application. Shape-based methods, such as hand contour and silhouette , are commonly used, while others may utilize information such as fingertips position or palm center.
For example, one study used a feature vector of 13 parameters , with the first parameter representing the aspect ratio of the hand's bounding box, and the remaining 12 parameters representing the mean brightness values of the image. Another approach utilized the  Self-Growing and Self-Organized Neural Gas (SGONG) neural algorithm to capture the shape of the hand, resulting in three features: palm region, palm center, and hand slope.
Another method involves calculating the  Center of Gravity (COG) of the segmented hand and the distance from the COG to the farthest point in the fingers, which is then used to estimate the number of fingers in the hand region. Additionally, some studies  divide the segmented image into blocks of varying sizes and extract the brightness factor for each block as the feature vector. However, experiments are needed to determine the optimal block size to achieve the best recognition rate.
Fig. 3 Features RepresentationSome studies also utilize geometric central moment as local and global features, extracting them using Gaussian pdf. Figure 3, provides examples of different feature extraction methods, such as partitioning the segmented image into terraces and extracting local and global geometric central moments, or dividing the segmented hand  into blocks and using the brightness factor of each block as the feature vector (blocks with black area are discarded) .
Recognizing hand gestures requires several steps, and the first one is to analyze the input hand image by selecting appropriate features and classification algorithms . Although edge detection and contour operators  aren't the ideal solutions since they might lead to misclassification , there are other alternatives, such as the Euclidean  distance metric, that can be utilized for gesture categorization.
For the purpose of recognizing hand gestures, a variety of statistical techniques  have been employed, such as Hidden Markov Models (HMMs), Finite State Machines (FSMs) , Learning Vector Quantization (LVQ), Principal Component Analysis (PCA), and neural networks. HMMs have sown particular success in recognizing dynamic gestures, while neural networks have been frequently used to extract hand form. Additionally, soft computing approaches like Fuzzy C-Means clustering (FCM)  and Genetic Algorithms (GAs)  have also shown success in identifying hand gestures.
The approach used is determined by the application and dataset, and recent advances in deep learning, notably Convolutional Neural Networks (CNNs), have demonstrated promising results for hand gesture detection. These developments indicate that hand gesture recognition systems will become increasingly accurate and popular in the future. Figure 4, explain the architecture of classification system.
Fig. 4 Architecture of gesture recognition system
APPLICATION AREAS OF HAND GESTURES SYSTEM
Hand gesture recognition systems have been deployed in a range of applications across several sectors . Sign language translation, virtual worlds, smart surveillance, robot control, and medical systems are some of these fields. Here's a rundown of some of the specific areas where hand motion detection is being used :
Sign Language Recognition:
Sign language has been a focus of attention for gesture recognition systems, as it is used to interpret and explain various topics during conversations . Several systems have been proposed for recognizing different types of sign languages . For instance, American Sign Language (ASL)  has been recognized using boundary histogram, MLP neural network, and dynamic programming matching. Japanese sign language (JSL) has been recognized using Recurrent Neural Network, 42 alphabet, and 10 words.  Arabic Sign Language (ArSL) has been recognized using two different types of neural networks, Partially and Fully Recurrent Neural Network.
Controlling a robot using hand gestures is an intriguing application in the field of hand gesture recognition. I recently came across a system that utilizes numbered hand poses to control a robot. The system works by assigning specific meanings and functions to each numbered pose, for instance, the one pose represents move forward and the five pose represents stop. By using these numbered poses, the user can give specific commands to the robot to perform different tasks.
Graphic Editor Control:
In a graphic editor control system, hand gestures are tracked and located as a preprocessing operation. To draw and edit graphics, 12 dynamic gestures are utilized. Drawing shapes include a triangle, rectangle, circle, arc, horizontal and vertical lines, while editing commands include copy, delete, move, swap, undo, and close.
Virtual Environments ( VEs):
Virtual environments (VEs) are a popular area of application for gesture recognition systems, particularly in communication media systems. A system proposed for  3D pointing gesture recognition in real-time using
binocular views, making it accurate and unaffected by changes in user characteristics or the environment.
Recently, hand gestures have been utilized in recognizing numbers as well. A new system has been  proposed that can isolate and identify a significant gesture from Arabic numerical hand motions in real-time using Hidden Markov Models (HMM).
People can use hand gestures and postures to control their television devices. For instance, a specific set of gestures can be utilized to turn the TV on and off, adjust the volume, mute the sound, and change the channel. These gestures typically involve opening and closing the hand in various ways.
Generating 3D models of the hand necessitates knowing the various forms and angles of the hand in order to correctly create and display the 3D shape. Many systems that employ hand silhouettes to build both 2D and 3D objects have been created. Moreover, 3D hand modelling is being investigated as a viable field of study for this reason. These models may be utilised to deliver more realistic and engaging experiences in a variety of applications such as virtual reality, gaming, and medical simulations. While there are still certain accuracy and efficiency problems to address, technological improvements are projected to increase the capabilities and possibilities of 3D hand modelling systems in the future.
Advantages that Hand Gesture Recognition Systems offer over other methods of Interaction and Communication:
Contactless Interaction: With the ongoing COVID- 19 pandemic, contactless interaction has become more important than ever. Hand gesture recognition systems offer a completely contactless way of interacting with devices and interfaces, which can help reduce the spread of viruses and bacteria.
Natural and Intuitive: Hand gestures are a natural and intuitive way of communicating, and they require little to no learning curve. This makes hand gesture recognition systems particularly useful for applications where users may not be familiar with traditional input methods, such as children or elderly users.
Accessibility: Hand gesture recognition systems can be particularly useful for people with disabilities, such as those who have difficulty using traditional input methods like keyboards or mice. Hand gestures can be a more accessible and natural way of interacting with devices and interfaces.
Enhanced User Experience: Hand gesture recognition systems can enhance the user experience by providing more natural and intuitive
ways of interacting with devices and interfaces. For example, hand gestures can be used to control devices in a more fluid and seamless manner, which can improve the overall user experience.
Versatility: Hand gesture recognition systems can be used in a wide range of applications, from gaming and entertainment to healthcare and industrial automation. This versatility makes hand gesture recognition systems a valuable tool for many different industries and use cases.
SUMMARY OF RESEARCH RESULTS:
Table 1 compares the recognition methods used by various hand gesture recognition systems. Table 2 provides a summary of the application areas and invariant vectors of some hand gesture recognition systems. Lastly, Table 3 displays a summary of the extraction method, feature representation, and recognition techniques used in the selected hand gesture recognition systems, including hand extraction technique, feature vector representation, and recognition.
This paper has explored various methods for hand gesture recognition, including Neural Network, Hidden Markov Model (HMM), and fuzzy c-means clustering. Orientation histogram has also been used for feature representation. For dynamic gestures, HMM has proven to be highly efficient, particularly in robot control  applications. Neural Networks have been used as classifiers  to capture hand shape , while various methods and algorithms have been employed for feature extraction  and shape capturing, such as Gaussian bivariate function in . The choice of recognition algorithm should depend on the specific application requirements.
Throughout this work, we have presented different application areas where gesture recognition systems can be used. We have discussed the challenges associated with gesture recognition and presented recent advancements in recognition systems. Finally, we have summarized some selected systems for hand gesture recognition.
Overall, hand gesture recognition systems offer a range of advantages over other methods of interaction and communication. They are intuitive and easy to use, and can be applied in various fields, including virtual reality, gaming, robotics, and healthcare. With advances in computer vision and machine learning, these systems are becoming mor accurate and reliable, especially with the use of deep learning algorithms. As we move forward, we expect to see continued growth and innovation in this field, and we encourage researchers, developers, and businesses to explore the possibilities of hand gesture recognition systems to improve user experiences and increase overall efficiency.