Hand Gesture Gaming using Ultrasonic Sensors & Arduino

- In the industrial 4.0 revolution, hand gestures are playing significant roles in the domains of IoT and robotics. Hand gestures are widely used in smart house, wearable devices, automobiles, virtual reality, etc in the domain of IoT. In robotics, these are used to control them and guide them. Now a day’s research scholars are working on the development of hand gesture computer that can be worked without the help of any hardware devices but with the help of hand gestures and sensors. There are few researchers who actually proved by showing that we can operate the video player, web browser and a text document with the help of hand gestures by using arduino and ultrasonic sensors. In this paper, we are going to show how we can control small games using hand gestures efficiently.


I INTRODUCTION
In today's world, vision and gesture are playing a crucial role in establishing communication between humans and computers just like physical devices (keyboard and mouse) do. There are various ways available for establishing communication between humans and computers. Out of those techniques, the hand gesture is very easy and fast to perform operations done by keyboard and mouse. Hand gestures can be implemented in various ways by using IR sensors, digital cameras, color bands and ultrasonic sensors. Each technique has its advantages as well as disadvantages. But out of all given techniques, using ultrasonic sensors is considered to be more comfortable as well as faster.
Initially, we take two ultrasonic sensors and fix them on the top of the laptop separated by a distance. Then we make circuit connections between the ultrasonic sensors and the arduino uno board which is attached on the top panel of the laptop. Now we use the ultrasonic sensor to measure the distance between the sensor and our hand placed in front of it. Based on the distance particular operation is performed. The primary objective of our paper is to play small games using hand gestures without the help of keyboard, mouse or any other physical device but using only arduino and ultrasonic sensors.
II LITERATURE REVIEW In the year 2006, university of Moncton used infrared sensors to perform mouse operations using hand gestures. Here a hand glove is made by fitting IR sensors which are used to recognize hand gestures of disabled people and convert these into meaningful messages. These messages are used in real-time to perform mouse operations [3].
In the year 2011, the Institute of Space Technology of Islamabad published a computer-based on hand gestures with the help of web camera and projector. Here the projector is used to project the display on the wall or any other plain surface. Users can interact with this screen using his fingertips which are tracked using 'camshaft' tracker in the air. A robust method has implemented to detect and recognize these fingertips, which are used to perform actions [4].
In the year 2015, University of Malaya developed computer and presented in an IEEE conference using color bands. The first color and depth sensor called Kinect was developed by Microsoft for the Xbox console and released in November 2010. This sensor projects an infrared pattern of 307,200 dots in a 640 × 480 mesh and receives the reflected pattern through a CMOS monochrome sensor. This structured light application allows the device to measure the depth of every point using triangulation. Moreover, an RGB camera provides synchronized color information for each point. Microsoft Kinect was formerly developed for full-body tracking to interact with video games using body movements and gestures [5].

III HARDWARE COMPONENTS SETUP A. Basic Hardware Setup
To implement the proposed method we require an arduino uno board, a pair of ultrasonic sensors, a USB cable and finally jumper wires. The sample layout is shown below.

B. Working of Ultrasonic Sensors
The Ultrasonic sensor operates on a similar principle as that of the radar system. Here the electrical energy is used to generate waves and these waves are transmitted by a transmitter module into the air to detect an object. The echo waves that are reflected by hitting an object are received at receiver module and the distance is measured based on the following formula. Distance = (Speed of the waves × Time taken to travel)/2 Here we divide the total distance into half because the waves are travelled to and fro.

D. Circuit Connection
For left sensor we connect VCC to 5V power pin in Arduino-Uno board, then we connect ground pin to first GND pin. Then we connect trigger pin to pin 2 and finally echo pin to pin 3 in arduino board. For the right sensor, we connect VCC pin to 3.3V power pin, then we connect ground pin to second GND pin. Finally we connect trigger pin to pin 4and echo pin to pin 5 in arduino board. The main purpose of installing this software is to use python packages that will simplify the code and also helps in performing various operations effectively.

C. Installation of PIP Package Installer
Pip is one of the best and widely used package manager used in Python for installing, deleting and managing packages. For some of the packages, pip comes preinstalled. So in order to check whether Pip is installed in your system or not we type "pip -version" in python console. If we get any error then we have to install it from below documentation. https://pip.pypa.io/en/stable/

D. Installation of PyAutoGUI Library
Pyautogui is a library in python which allows us to control keyboard as well as mouse to perform operations over a computer without the help of them. So to install it we use following command "pip install pyautogui".

E. Installation of PySerialI Library
PySerial library is used to provide backend support for python running on our computer. The major role of this library is it enables our program to communicate with the serial port. So to install it we use following command "pip install pyserial". V OUTPUT & RESULTS For many small games and even for big games the main keys are arrow keys or w, a, s, d keys. Here for proving this, we are taking a small car race game. Where the major operations are going forward, sideways and break. So in this, we have implemented those operations as listed below:

A. Implementation of UP Arrow
To implement this operation initially we place our right hand in front of the right sensor at a distance less than 40 cm. Then the car starts moving forward with increasing speed. If we remove the hand then the speed automatically falls slowly to zero. The following picture depicts the increase in speed when we place our hand as specified.

B. Implementation of Down Arrow
To perform this operation initially we place our left hand in front of the left sensor at a distance less than 40 cm. Then the car starts slowing down to zero. The following picture depicts the decrease in speed when we place our hand as specified above.

C. Implementation Right Arrow
To perform this task we place our right hand in front of the right sensor at a distance greater than 50 cm and less than 80 cm. Then the car starts turning right until we remove hand it keeps on turning towards right. The following picture depicts how car turn towards right when we place our hand as mentioned.

D. Implementation Left Arrow
To perform this task we place our left hand in front of the left sensor at a distance greater than 50 cm and less than 80 cm. Then the car starts turning left until we remove hand it keeps on turning towards left. The following picture depicts how car turn left when we place our hand as mentioned. CONCLUSION In this paper, we have presented how we can play smaller games with hand gestures with the help of arduino and ultrasonic sensors by taking small call race as an example.

International
Here we use sensors for measuring the distance between our hand and sensors. Based on that distance we can perform various operations by programming the arduino board with the help of arduino IDE. We have used python package pyautogui for controlling keyboard using hand gestures. In future, we implement the same approach for playing large games by using sensors of high quality and performance.