Cursor Control using Human Eye

Cursor Control using Human Eye

Meghana M N

Assistant Professor, Dept. of ECE Maharaja Institute of Technology Thandavapura Mysuru, Karnataka, India

Nanditha P

Student, Dept. of ECE Maharaja Institute of Technology Thandavapura Mysuru, Karnataka, India

Thanu N

Student, Dept. of ECE Maharaja Institute of Technology Thandavapura Mysuru, Karnataka, India

Sachin S N

Student, Dept. of ECE, Maharaja Institute of Technology Thandavapura Mysuru, Karnataka, India

Keerti Patil

Student, Dept. of ECE, Maharaja Institute of Technology Thandavapura Mysuru, Karnataka, India

Abstract – Computers are really important in our lives and people are always finding new ways to use them. The project called Cursor Control Using Human Eye is trying to make it easier for people to use computers. This project lets the cursor, on the computer screen move around based on where you’re looking. The people who made this project used something called Visual Studio Code to build it. They also used a tool called OpenCV to help the computer understand pictures. The Cursor Control Using Human Eye system works by finding the users face and then keeping track of where their eyesre moving. When you move your eyes the pointer on the screen goes in the direction as your eyes. If you blink that is like clicking the mouse. This means you do not need a mouse or keyboard. This project is really helpful for people who have trouble using the devices because their body is not able to.The program worked well when we tested it. It was accurate. Moved smoothly even when the light, in the room changed or things were moving behind the person. The computer vision system is a way to make it easier for people to interact with computers. This project shows how computer vision can make it better for people to use computers. Computer vision is a part of this system and it really helps with human computer interaction. This thing gives us a starting point for making things like controlling stuff with just our voices, tools that help people who need it and computers that we can use without even touching them. It is really about making applications, like virtual control and assistive tools and even gesture-free computing.

I.INTRODUCTION

Computers are all around us nowadays. We use computers for many things such as studying and working. We even use computers to watch movies. They are used in hospitals too. Computers are a part of our daily lives. Usually, we use a mouse or a keyboard to control computers. There are people who have difficulty using their hands. This makes it very difficult for these people to use a computer like everyone else. This project is about computers. It is about using eye movement to move the mouse pointer, on computers. This thing is created using Visual Studio Code. It uses OpenCV to watch the video

from a regular webcam. The system watches the eyes and face of the persons. It moves the cursor to where OpenCV and the system think the person is looking at. OpenCV and the system can even tell when the person blinks and it can use that as a click. So the person does not have to touch the mouse or the keyboard at all when they are using OpenCV and the system. This thing is very useful for people who have difficulty moving their hands. It is also useful in hospitals where you do not want to touch things. For people who play games or use reality, this kind of control system makes everything feel very real and fun to use. The concept of this thing is amazing. There are still some flaws in this thing. The camera needs light to work properly. If the person moves around quickly or if the room, behind them changes, the system gets confused. This system also needs a very powerful computer to work quickly and smoothly. If we have Artificial Intelligence tools and computers that work faster, then we can make these flaws smaller. We can really make the problems with Artificial Intelligence, with better Artificial Intelligence tools and faster computers. In simple words, this project shows how technology can make life easier for everyone. It provides a means for physically challenged individuals to operate computers by themselves. In the future, such systems could become widespread and revolutionize the way we interact with computers entirely.

II.METHODOLOGY

The system is designed to allow you to control the computer cursor using your eyes and face without having to use your hands. The system uses a webcam to record videos of you all the time, which it uses to determine what you want to do. The video is analyzed frame by frame and each frame is cleaned up using OpenCV. This means that the video is converted from its colors to simpler colors, which makes it easier to see and understand. This makes it easier for the system to see things clearly and reduces errors and makes it even better at finding the key parts of your face and eyes. Once the preprocessing is complete, the system examines the face and eyes. It uses computer algorithms, such as the Haar Cascade classifier or the Mediapipes Face Mesh model, to

locate and track the eyes. The system observes the location of the eye landmarks. It then computes the Eye Aspect Ratio or EAR for short to determine when the eyes blink. This is based on the distance between the eyelids. The Eye Aspect Ratio is crucial because it helps the system determine when a blink occurs. The blink number, over a period of time, is used to perform different operations on the mouse. One blink performs a left click. Two blinks perform a click. Three blinks perform a drag or hold action. Four or more blinks perform a click. The direction you are looking is also used to move the cursor around the screen. The PyAutoGUI library is used to control the mouse to perform actions. It does this very accurately and instantly. The program runs endlessly in a loop until it is stopped by the user, providing a smooth and user-friendly interface that improves accessibility for people with motor disabilities as well as offering an innovative way of controlling a computer for all users.

Figure 1: Block diagram

Figure 2: Flow chart

III.IMPLEMENTATION

The eye-controlled cursor system was developed using Python in the Visual Studio Code environment. The eye- controlled cursor system required assistance to function very well. Therefore, we employed various libraries to improve the system. The OpenCV library was responsible for video acquisition and image processing. The Mediapipe library was responsible for the tracking of the eye-controlled cursor system’s landmarks. The PyAutoGUI library assisted the eye- controlled cursor system in moving the cursor and clicking the mouse. We employed the above libraries to ensure that the eye-controlled cursor system functioned very well. When the system is run, the webcam begins to capture a video of the user’s face. The video is processed into frames. Each frame undergoes various transformations, in OpenCV. It is resized. Additionally, it is converted from BGR to RGB and then to grayscale. The transformations make the image clearer. They also remove some of the noise. This helps the system to accurately detect the face and eyes. The Mediapipe Face Mesh model examines the face. It identifies the key features, particularly the eyes. It relies on these points to determine the Eye Aspect Ratio. The Eye Aspect Ratio is used to monitor the movement of the eyelids.

The system uses the Eye Aspect Ratio to track the users eyelid movements. When the EAR reading suddenly goes down that means the user has blinked. Every time the user blinks it gets recorded so that the system can know what the user wants to do with the EAR system. The system observes the number of times you blink in a short time and does different things with the mouse. For example, if you blink once it is like you clicked the button on the mouse. If you blink twice it is like you double clicked the mouse. If you blink three times it starts to drag something. If you blink four or more times it is like you clicked the button on the mouse. The system also observes where you are looking on the screen to move the cursor. PyAutoGUI does all these things away so it feels smooth and easy to use the mouse with your eyes and blinks. The program runs continuously in a loop, updating frames to maintain accurate tracking and consistent performance.

Figure 3 : Implementation

IV.RESULTS AND DISCUSSION

Figure 4 : Opening a document

Figure 5 : Right clicking

Figure 6 : Drag / Select

Our eye-controlled cursor allows the user to perform all the actions of the mouse. You can click, right click, move, and double click by merely looking and blinking your eyes. To click, you either focus your eyes for a brief moment or give a brief blink. To perform other functions, the procedure is to focus your eyes on the target and then blink to select the target. If you double blink or stay for a brief moment on the target, the computer recognizes what you are trying to perform, which is the action of clicking the target. To move objects, the procedure is simple: focus your eyes on the target, then move your eyes to the spot where you want the target, and then blink.

As a whole, these abilities prove that the system can simulate typical mouse behavior by means of eye movement alone. This is a very pertinent alternative for individuals whose hand movement is limited. The ability to adapt as it

does will allow users to interact more fluidly with the computer.

TABLE I: Summary of Implemented Results

V.CONCLUSION

This project is intended to develop an application where one can operate the computer cursor without the usage of the mouse, as it is able to track the eye movement and facial expressions of the user. The main objective is to develop an application where one can operate the computer without difficulty, as it only requires the user to look at the screen and make facial gestures.

There is no requirement for specialized equipment other than a common webcam. The system for controlling the cursor is simple and natural to use so that a person with limited ability in their hands is able to operate the computer by viewing and indicating focus with facial expressions. It is based on computer vision algorithms created using the Python programming language with the assistance of OpenCV & Dlib for tracking the focus on a point being observed by the user.

One major advantage is that there is no need for physical contact, which is very important for people having issues using a mouse or keyboard. This system is very helpful since it improves accessibility while providing a clean and contact- free way to interact with a computer.

In general, the paper aims towards, and encourages, more accessible computer interaction that allows humans to control computer systems by themselves. This paper also gives an indication of various applications in fields such as gaming, virtual reality, and more.

DECLARATION STATEMENT

REFERENCES

1. Muhammad Usman Ghani, Sarah Chaudhry, Maryam Sohail, M.Nafees Geelani, Gaze Pointer: A Real-Time Mouse Pointer Control Implementation Based on Eye Gaze Tracking, Journal of Multimedia Processing and Technologies, vol.5, no. 2, June, 2021.

2. Yiu-mingCheung, Senior Member, IEEE, and Qinmu Peng, Member, IEEE, Eye Gaze Tracking With a Web Camera in a Desktop Environment, IEEE transactions on human-machine systems, vol. 45, no. 4, August 2020.

3. J. Sreedevi; M. Shreya Reddy; B. Satyanarayanan Eyeball movement-based cursor control using Raspberry Pi and OpenCV International Journal of Innovative Technology & Exploring Engineering May 2020.

4. Sivasangari A; Deepa D; Anandhi T; Anitha ponraj & Roobini M.S Eyeball based cursor movement control International Conference on Communication on Signal Processing June 2020.

5. Satyanarayana B., et al. “Eyeball Movement based Cursor Control using Raspberry Pi and OpenCV.” 2020 International Journal of Innovative Technology and Exploring Engineering (IJITEE).

6. Sovit Ranjan Rath, et al. Machine Learning and Deep Learning, Face detection with Dlib using HOG and Linear SVM.

7. C. A. Chin “Enhanced Hybrid Electromyogram / Eye gaze tracking cursor control system for hands-free computer interaction, Proceedings of the 28th IEEE EMBS Annual International Conference, New York City, USA, Aug 30-Sept 3, 2006.

8. S. Mathew, A. Sreeshma, T. A. Jaison, V. Pradeep, and S. S. Jabarani, “Eye Movement Based Cursor Control and Home Automation for Disabled People,” 2019 International Conference on Communication and Electronics Systems (ICCES), Coimbatore, India, 2019.

9. C. Fagiani, M. Betke, and J. Gips, Evaluation of tracking methods for human-computer interaction, Proceedings of the IEEE Workshop on Applications in Computer Vision, Orlando, Florida, December 2002.

10. L. Chen, G. Kukharev, K. Peng and S. Ruan, A robust algorithm for eye detection on gray intensity face without spectacles, JC and T, vol. 5, 2005.

11. Zhao, Matti Pietikainen, Abdenour Hadid & Takeo Kanade(2022), Facial Expression Recognition Using Deep Learning: A Survey, IEEE Transactions on Affective Computing.

12. Xucong Zhang, Yusuke S., Mario F.,& Andreas Bulling(2021), A Survey on Eye Gaze Tracking Applications in Computer Vision and HCI, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).

13. Virtual Mouse And Keyboard For Computer Interaction Mishaha MK, Manjusha MS ,IJETMS 2022.

14. Gesture Recognition Based Virtual Mouse and Keyboard Sugnik Roy Chowdhury, Sumit Pathak, M.D. Anto Praveena, 978-1-7281- 5518-0/20/$31.00 ©2020 IEEE.

15. Robert J. K. Jacob, The use of eye movements in human-computer interaction techniques: what you look at is what you get. ACM Trans.

    Inf. Syst. 9, 2, pp. 152169, 1991. doi:10.1145/123078.128728

16. Fahim, Shahriar Rahman, et al., A visual analytic in deep learning approach to eye movement for human-machine interaction based on inertia measurement. IEEE Access 8: 45924-45937, 2020.

AUTHORS PROFILE

Meghana M N M. Tech in Signal Processing, pursuing PhD in the field of Video Processing from Visvesvaraya Technological University, Belagavi, under the guidance of Dr. Rajalakshmi M C, Professor, Department of ECE, VVIET, Mysore. Presently working as an Assistant Professor in the Department of Electronics and Communication Engineering, Maharaja Institute of Technology Thandavapura, with 8 years of teaching experience. Field of interest includes Digital Electronics, Verilog HDL and Multimedia Communication.

Nanditha P, is an undergraduate student pursuing a Bachelor of Engineering in Electronics and Communication Engineering at Maharaja Institute of Technology, Thandavapura, Mysuru, India. She is passionate about applying electronics and intelligent systems to solve real-world problems. Technical interests include Embedded Systems, Internet of Things (IoT), Artificial Intelligence & Machine Learning, VLSI, and Automation. She has successfully completed projects such as a RADAR System using Arduino, biometric authentication, and mobile-based access control,and a Water Level Detector for monitoring and alert applications .She possesses hands-on experience with Python, C programming, MATLAB, Arduino, and CAD tools, along with strong problem-solving and teamwork skills. She has also completed professional certifications in VLSI, Python Fundamentals, AI & ML, and Arduino, reflecting her commitment to continuous learning and technical growth .She aspires to build a career in the field of Electronics and Intelligent Systems, contributing to innovative, reliable, and technology- driven solutions.

Thanu N , Electronics and Communication Engineering graduate with a strong foundation in embedded systems and digital design concepts. Possess hands-on knowledge of the 8051 microcontroller and Verilog hardware description language for basic digital system modelling. Familiar with embedded development and simulation tools such as Keil (v4 & v5), Arduino IDE, and MATLAB. Skilled in C and C++ programming for hardware- oriented applications, with working knowledge of Python and Java for support and analysis. Experienced in using Windows operating systems and MS Office for technical documentation. A self-motivated, detail-oriented learner aiming to apply core ECE skills in a professional engineering environment.

Sachin S N , A motivated Electronics and Communication Engineering undergraduate with a strong academic background and hands-on experience in embedded systems, IoT, automation, and basic AI/ML concepts. Possesses practical knowledge of programming languages such as C and Python, along with experience in microcontrollers, sensors, and hardware prototyping platforms like Arduino and NodeMCU. Familiar with industry-standard tools including MATLAB, Multisim, Cadence, and Keil µVision.Has successfully completed multiple academic projects involving real-time sensing, automation, healthcare monitoring, assistive technologies, and computer vision, including a vision-based eye movement through internships and certifications. Demonstrates strong problem-solving ability, teamwork, and adaptability, with a keen interest in developing innovative and efficient technological solutions in electronics, communication, and automation domains.

Keerti Patil , final-year Electronics and Communication Engineering (ECE) student with a strong interest in computer vision and Internet of Things (IoT) technologies. I have practical experience in developing projects that integrate OpenCV for real-time image processing and IoT platforms for data acquisition, monitoring, and automation. have developed projects smart waste segregator using IOT, automatic water irrigation plant and smart monitoring plant system

.My academic background includes embedded systems, sensors, microcontrollers, and Python-based computer vision applications. Through this project, I focused on designing an efficient and reliable system that combines visual intelligence with connected devices to solve real-world problems I am highly motivated to explore emerging technologies and continuously enhance my technical and research skills, aiming to build a career in core electronics and intelligent systems.