Gesture Recognition using Convex Hull and Hull Defects Method to Control Electronic Appliances in Home

DOI : 10.17577/IJERTCONV9IS05054

Download Full-Text PDF Cite this Publication

Text Only Version

Gesture Recognition using Convex Hull and Hull Defects Method to Control Electronic Appliances in Home

Raswitha Bandi1, K.Anurag2, K.Siva teja 3, M.Snehith 4, R.Varun5

1Assistant Professor, 2,3,4,5 UG Scholar,

Department of Information Technology, VNR Vignan Jyothi Institute of Engineering and Technology, Hyderabad, India

Abstract:An essential objective of gesture recognition is to make a framework that can recognize explicit human motions and use them for control. In this paper, we use Interface with PCs utilizing gestures of the human body, and hand movements. In Gesture recognition innovation, a camera peruses the developments of the human body and conveys the information to a PC that utilizes the motions as a contribution to control appliances. for easy communication daily in our lives, we use physical gestures. A bunch of actual motions may establish a whole language, as in gesture-based communications. The goal of this paper is to classify images of different hand gestures, such as a one, two, three, and others. So that if we show any of the gesture it can control electronic devices like turning them on and off. For example we can allot one to fan, two to light and so on. The method well be using is hull convex and hull defects . In this paper, we'll be planning to order various pictures of hand motions, which implies that the system should "learn" the highlights of each motion and group them accurately. For example, if it is given an image of a hand showing one finger, the output of the model needs to be one So if we can classify different gestures we can then connect it to hardware to control the home appliances.

Keywords- Machine Learning, gesture recognition, Internet of Things.


    There has been growing interest in development of new techniques and methods for Human Computer Interaction (HCI). Hand Gesture Recognition is one of the important areas of this technology. Gesture Recognition means interfacing with computer using motion of human body typically hand movements [1].

    As a Handicapped or old person cannot move easily and quickly to control the various electrical devices in the house or the person is unaware of controlling devices but the same task can be done easily with hand gesture recognition.


    dimensional frame in order to display an image. These pixels exhibits different colors with three primary colors known as RGB where R stands for Red, G stands for Green, B stands for Blue. Each of the three colors has 256 different intensities ranging from 0 to 255. For example, [ 0, 0, 255] indicates Green with high intensity. We can also get different colors by mixing with varying intensities of the three colors RGB in different combinations. For example, [255,255,0] indicates yellow color with high intensity. As two- dimensional array of RGB represents an image we can perform Relational and logical operations on each pixel [3].

    There are various reasons for a person to be not using various devices properly. These include the following main reasons in our daily routine.As a person is handicapped, he cannot move fast enough to control it. Lack of Knowledge of using it.As people get older, they may forget about how to use it.


    Damage to devices Prone to accidents Loss of life Causalities Purpose:

    There are many instances where handicapped person has to depend on others even for their household works. They are unable to live the life by themselves. According to a report taken in 2018, over all 2.21% of Indian population has one or other kind of disability. This means 26.8 million people in India are disabled. These many people are always had to depend on others [2].

    This paper deals with how to control various devices using hand gestures. This problem was implemented in different ways such as neural networks, Open CV. In this we deal with Open CV in recognition of hand gesture. So, we recognize the hand gestures and control various devices according to it. The main task is to make the life of handicapped person easier.


In this chapter we discuss about various research papers which were referred while implementing this. Let us understand more about image processing techniques in detail.First, we need to know how an image is stored in digital format. In this digital world every bit of information is stored in 0s and 1s. It is same for image as well where every image is displayed with the help of pixels. Each pixel can represent 16,777,216 colors. These pixels form a two- There are different methods by which we can detect hand gesture.

  • Glove-based analysis

  • Vision-based analysis

    1. CNN

    2. Open CV

      Glove-based analysis:

      The glove-based approach employs sensors (mechanical or optical) attached to a glove that acts as transducer of finger flexion into electrical signals to determine hand posture Vision-based analysis:

      Vision based system has three main steps, namely, image capture, threshold application, and number recognition. By using Open CV we do vision- based analysis

      Open CV:

      Open CV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision. In simple language it is library used for Image Processing. It is mainly used to do all the operation related to Images.

      Convolution neural network (CNN):

      Convolution Neural Network (CNN) is a deep learning network used for classifying images. The basic premise behind CNN is using predefined convolving filters to identify patterns in image edges, parts of objects and the build on to this knowledge to detect complete objects like animals, human being, automobiles etc.


      In this proposed system we control various electrical devices by hand gestures using image processing techniques. The approach holds various stages of image processing with the help of Open CV python. The results will control electrical equipment with help of hand gestures [4].

      There are various steps used in image processing which include:

      Finding contours

      Morphological transformation

      HSV Conversion

      Convex Hull

      Obtain Region of Interest (ROI)

      We have different parameters that have to calculate for recognizing hand gestures properly. Some of parameters that calculated are:

      Calculating the area of figure formed by joining corners of given gesture

      Calculating the area of figure formed by image noises

      Calculating the height and angles in figure formed by joining corners of given gesture [5].

      The step in Outcome:

      Controls electrical devices Advantages:

      No physical contact is required between individual and gesture-recognition enabled device

      Various devices can be controlled using simple hand gestures

      Controlling devices using hand gestures makes life of handicapped easier

  • In pandemic situation like covid-19 people are feared to touch devices which are used by someone. This system helps to control devices without even touching it.


The main objective of this project is to control various devices by recognition of hand gestures with reasonable accuracy and makes the life of handicapped person easier.

The results include various gestures given by hand and they are tested out. It implies different hand gestures are properly used for different controls in a device.



  1. Image Pre-processing

    • Image Capture

    • Calculate average and maximum HSV

  2. Image Processing

    • Convexity defects

  • Extract contours

    Fig 1. Flow diagram for Action Recognition for Controlling Electronic Appliances in Home.

    In this system we will first capture the frame of video using region of interest specified [7]. When we press v the whole project starts after we run file. When we press v, we first have to place our hand in a blue box and we have to press v again. This is used to calculate the average and maximum HSV values. It then shows a dialogue box with captured image in HSV format. If we are satisfied with the captured image, we can go to step or else we have to execute it again. In the next step we set region of interest where we have to place our hand. This frame inside region of interest is captured and is converted into HSV image. Using in Range method we convert this frame in to black and white. And then dilate and erode are applied on this using kernel of ones. And now using find contours method we calculate the contours and maximum contour area out of all contours. We then analyze contours using convex hull and convexity defect hull methods. After analyzing we then show the result on the screen.


    Implementation Steps :

    • Take input video from the camera.

    • Using the frame captured calculates lower bound, upper bound and average of HSV values of Hand.

    • Now, take the input Video until program exits.

    • For every frame captured, first convert it into HSV image.

    • Using lower bound and upper bound values separate the hand from background (background black color and Hand

white color) [8].

  • Apply Contours on hand for detection of hand.

  • Using contours find the Convex Hull or Approximate Polygon.

  • Now, calculate the Convex Defects based on Convex Hull.

  • Using some Mathematical Calculations Count the number of considerable defects.

  • Using contours calculate area occupied by the contours.

  • Using Convex Hull calculate the area occupied by the Hull.

  • Calculate the Area ratio from Convex Hull and Contours [9].

  • Using Area Ratio and Number of defects conclude the result [10].

4.1 IMPLEMENTATION OF THE MODEL: To implement this System we use Python IDLE. Python Packages:

Open CV:

Open CV is a computer vision library which is an open source, optimized and cross platformandwritteninc++programminglanguage.Itisoriginally developedbyinteland later on it was supported by Willow Garage. It is also used in python, Java interfaces. It provides interface for any image processing technique. There are many different applications of open CV such as Human- computer Interaction, robotic, segmentation and recognition, motion tracing, Gesture and Facial recognition.

Num py:

Num Py or Numerical Python is library used to perform some operations on arrays. It containsan- dimensionalarrayobjectthatisusefulinapplyingalgebraandvari ousother mathematical formulas to the array as a whole. NumPy object can also be referred as Numpy array. It stores data such that it effectively performs mathematical calculations.NumpyArray stores the data in the tabular format i.e. .rows and columns. In order to use Numpy first check whether numpy is installed or not? If not install the numpy package using below steps.

To install numpy, go to command prompt. Type the command: pip install numpy

The numpy array has many advantages when compared with list because:

  1. Less Memory

  2. Fast

  3. Convenient

    We can perform different operations such as ndim, itemsize, dtype, reshape, slicing, linspace, max/min, square Root and Standard Deviation etc.

    5. RESULTS

    1. Welcome Screen

    2. For HSV image calculation

    3. HSV image

    4. Outputs


      A method for hand gesture recognition is introduced in our system. Our system Hand Gesture Recognition captures videos from camera that is installed within the system and able to control electrical devices based on gestures given by user. The hand region is detected by using the range of skin color.

      The performance of our system is evaluated on different sets of inputs. The experimental results show that our approach performs well and is fit for the real-time applications.

      Our system can neglect the image formed by noises and small moving objects with the color that is similar of skin. Various parameters are calculated like area formed by figures of both joining the corners of skin and of noises which will make the output to be more accurate.

      The image and the details such as gesture given are displayed on the screen using the Open CV operations which gives the information about various controls should be performed on

      devices. This will help the users to use the electrical devices properly and avoid accidents.

      Hence the hand gesture given will be able to control electrical devices which will make the life of handicapped person easier and happier.


      1. P. Gonzalo and A. Holgado-Terriza Juan, "Control of home devices based on hand gestures", 2015 IEEE 5th International Conference on Consumer Electronics – Berlin (ICCE-Berlin), pp. 510-514, Sep. 2015.

      2. V. I. Pavlovic, R. Sharma and T. S. Huang, "Visual interpretation of hand gestures for human-computer interaction: a review", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 677-695, July 1997.

      3. C. Pham, "Mobirar: Real-time human activity recognition using mobile devices", 2015 Seventh International Conference on Knowledge and Systems Engineering KSE 2015 Ho Chi Minh City Vietnam October 8-10 2015, pp. 144-149, 2015.

      4. F. Dominio et al., "Hand gesture recognition with depth data", ACM/IEEE Int Work. on Anal. & Retrieval of Tracked Events & Motion in Imagery Stream, 2013.

      5. D. Lee, H. Yoon and J. Kim, "Continuous gesture recognition by using gesture spotting", 16th ICCAS, 2016.

      6. J. Ducloux et al., "Accelerometer-based hand gesture recognition system for interaction in digital TV", IEEE Int Instrum. & Measur. Tech. Conf., 2014.

      7. Y. Ma et al., "Hand gesture recognition with convolution neural networks for the multimodal UAV control", Work. on Research Education and Development of Unmanned Aerial Systems, 2017.

      8. F. Alemuda and F. J. Lin, "Gesture-based control in a smart home environment", IEEE Int iThings & GreenCom & CPSCom & SmartData, 2017.

      9. L. Xu, W. Yang, Y. Cao and Q. Li, "Human activity recognition based on random forests", Int Conf. on Natural Comp. Fuzzy Systems & Knowledge Discov., 2017.

      10. H. Chu, S. Huang and J. Liaw, "An acceleration feature-based gesture recognition system", IEEE Int Conf. on Systems Man & Cybernetics, 2013.

Leave a Reply