An Affordable Surveillance Robot using Raspberry PI and YOLO

An Affordable Surveillance Robot using Raspberry PI and YOLO

Pranay Thumati

Department of ECE, Institute of Aeronautical Engineering,

Hyderabad, India

Siva RamaKrishna J

Department of AIML, Institute of Aeronautical Engineering,

Hyderabad, India

Pranav V

Department of ECE, Institute of Aeronautical Engineering,

Hyderabad, India

Rahul Karipe

Department of ECE, Institute of Aeronautical Engineering,

Hyderabad, India

Abstract

surveillance plays a critical role in safeguarding and offering owners the ability to maintain real-time awareness and effectively respond to potential intrusions or unauthorized activities. However, traditional surveillance methods often rely on basic motion detection sensors and limited face recognition capabilities, resulting in fragmented monitoring solutions that fall short in providing immediate threat identification. Many systems also offer only delayed notifications, which can hinder timely responses to security breaches.To overcome these limitations, we developed an affordable, advanced surveillance robot that combines real- time face and object recognition, continuous video streaming, and remote control via a user-friendly web interface. Constructed with a Raspberry Pi 3B+, PiCamera, and motor driver, this system uses LBPH (Local Binary Patterns Histograms) for face recognition alongside the YOLOv3 algorithm for object detection, forming a responsive and adaptive surveillance tool capable of distinguishing individuals and objects within its field of view. Unlike traditional setups, this robot ensures uninterrupted video streaming with minimal lag, allowing users to view, monitor, and control the system remotely from any location by providing immediate audio feedback. This solution not only surpasses the constraints of existing systems but also offers a practical, efficient, and cost- effective approach to real-time security, making it ideal for the owners seeking a comprehensive and versatile surveillance option.

Keywords- Surveillance, Face recognition, Local Binary Patterns Histograms (LBPH), Object detection, YOLO (You Only Look Once).

1. INTRODUCTION

   The security of residential spaces has gained significant importance. Innovations in robotics, artificial intelligence (AI), and the Internet of Things (IoT) have paved the way for advanced surveillance systems capable of offering real-time

   monitoring, object detection, and remote control. Unlike traditional security solutions such as CCTV cameras, which often lack mobility and sophisticated recognition features, modern autonomous surveillance robots can proactively patrol homes, detect unusual activities, and send immediate alerts to owners [1].

   The Raspberry Pi platform plays a pivotal role in the development of these intelligent surveillance systems. Known for its cost-effectiveness and versatility, the Raspberry Pi 3B+ model is particularly well-suited for robotics projects, supporting various peripheral devices and enabling real-time processing. Its compact design and compatibility with open- source software make it an excellent choice for creating a mobile robot dedicated to security [2].

   This research focuses on a mobile surveillance robot that utilizes the Raspberry Pi to facilitate real-time video monitoring and object recognition within the environment. Equipped with a PiCamera for video capture and leveraging OpenCV for image processing, the robot can effectively identify faces and objects. By incorporating the YOLO (You Only Look Once) v3 model for object detection, the system is capable of recognizing potential threats or anomalies, such as intruders or unattended items, while also distinguishing familiar pets [3, 4]. The robots mobility is managed through GPIO pins linked to motors, allowing it to navigate autonomously or respond to remote commands via a web interface built with Flask.

   A standout feature of this system is its dual capability to recognize both objects and familiar faces [5]. This functionality is essential for differentiating between known family members and potential intruders, allowing for more personalized and precise feedback. Users can monitor live video feeds, issue

   movement commands, and receive real-time notifications of detected activities, even when they are away from their homes. This integration of facial and object recognition enhances the robot's adaptability to various surveillance contexts within a household [6].

   The combination of mobility and intelligent surveillance results in a highly interactive security solution. The robot's ability to patrol and monitor different areas of the adds an extra layer of protection, and it can promptly alert owners in case of suspicious activity. The energy-efficient design of the Raspberry Pi, alongside the lightweight structure of the robot, ensures continuous operation without significant power consumption.

   The structure of this paper is organized as follows: Section II reviews relevant literature on robotics and surveillance technologies. Section III details the methodology along with the hardware and software components used. IV discusses the results of the experimental setup and evaluates the robots performance in various surveillance scenarios. Finally, Section V concludes the paper by discussing potential improvements and future applications of the system.

2. LITERATURE SURVEY

   As the demand for security solutions grows, advancements in robotics, artificial intelligence (AI), and Internet of Things (IoT) technologies are shaping the future of residential surveillance. The development of autonomous mobile robots equipped with real-time video monitoring, face recognition, and object detection is a promising direction for enhancing security. In this section, recent work in key areas such as robotic surveillance systems, computer vision, facial and object recognition, and Raspberry Pi-based IoT applications will be discussed.

   A surveillance robot designed for international border security leverages the ESP-32 Wi-Fi CAM, Arduino, and PIR sensors to detect trespassers and notify security control units, enhancing safety in sensitive areas through real-time web-based monitoring and obstacle avoidance [7]. InterBot 1.0 [8] showcases the concept of the Internet of Robotics (IoR), integrating environmental sensors to enable comprehensive industrial monitoring. This setup provides remote access to real-time data visualization, suggesting the applicability of IoT- enhanced robotics beyond traditional security into industrial automation.

   Real-time surveillance for hostile or hazardous environments as detailed by [9], underscores the importance of adaptable robotics. This system, which uses ZigBee networks and includes audio and video capture, offers tactical advantages in military operations, where mobility and remote control are essential. A movable camera system that improves upon traditional surveillance methods such as CCTV, drones, and camera traps by providing enhanced coverage, portability, and control. This application have shown potential for use in high- risk areas, using secure protocols for remote image and video

   monitoring [10]. Another noteworthy project utilizes a Raspberry Pi and various sensors to monitor environmental hazards, including gas leaks, and remotely transmits data to a cloud server for remote access, making it suitable for sensitive locations [11]. A spy robot with multiple hazard-detection sensors, such as a smoke detector and PIR motion sensor, has been developed to capture live images an track GPS locations for short-range applications, ensuring personnel safety through remote hazard detection [12].

   A security system that utilizes a robot equipped with cameras and acoustic sensors for home surveillance. Controlled via a Raspberry Pi, the robot follows a predetermined path using an IR-based system to monitor its surroundings. The system aims to enhance home security by autonomously patrolling large areas and providing a real-time environmental feed, alerting users of suspicious activities [13]. A wireless mobile surveillance robot equipped with a camera and various sensors to monitor hazardous conditions and provide real-time video feedback via a web interface. Users can control the robot through an RC transmitter using the NRF24L01 module or a DTMF module interfaced with Arduino and L293D motor drivers, allowing for flexible remote operation. It offers an efficient and affordable solution for enhancing security and surveillance capabilities [14].

   Face recognition systems using the Local Binary Patterns Histogram (LBPH) algorithm have also shown effectiveness in home security applications. These systems, powered by Raspberry Pi, use Haar Cascade Classifier for real-time face detection and recognition, triggering access controls or security alarms based on facial data [15]. Similarly, a camera-based surveillance solution using Raspberry Pi allows users to remotely control the camera's rotation through mobile or desktop applications. Operating on a Linux-based platform, the system offers an accessible, scalable, and financially viable surveillance solution [16]. Advanced applications for military purposes, as seen in [17] and [18], focus on integrating AI for face detection and threat evaluation. By implementing machine learning models for target identification and various hazard detections, these robots are designed to enhance military and border surveillance capabilities. The combination of real-time monitoring and defensive mechanisms in these systems represents a forward-looking approach in automated defense technology.

3. METHODOLOGY

   The implementation of the surveillance robot project involves integrating various hardware and software components into a unified system that can perform real-time monitoring, face and object detection, and remote control. The following section explains the key steps taken during the implementation process to ensure the robot functions effectively.

   1. Setting Up the Hardware Components

      The hardware setup for this project starts with assembling the robot's body and connecting the required components as shown in the Fig. 1. The Raspberry Pi 3B+ serves as the brain of the

      system, controlling all other peripherals. It is mounted onto a chassis along with a PiCamera, L298N motor driver, motors, and power supply. The motors are connected to the L298N motor driver, which allows for forward, backward, and rotational movement of the robot. The motor driver is linked to the GPIO pins on the Raspberry Pi, enabling software control over the robots mobility. The PiCamera is positioned on the robot to capture live video, which is used for image processing tasks such as face and object detection. A portable power bank powers both the Raspberry Pi and the motor driver to ensure mobility and continuous operation.

      Fig. 1. Prototype of Surveillance Robot

   2. Installing and Configuring the Software

      Once the hardware setup is complete, the software side of the project is configured. The Raspberry Pi runs the Raspbian OS, and several libraries are installed, including OpenCV for image processing, Flask for the web interface, and Pyttsx3 for text-to-

      speech functionalities. The OpenCV library is critical for handling both face detection and object recognition. Haar Cascade classifiers are used to detect faces, while YOLOv3, a deep learning-based object detection algorithm, identifies various objects. Both models are configured to process real- time video from the PiCamera, which streams directly into the Raspberry Pi. To enable remote control and monitoring, a web interface is built using Flask, a lightweight web framework for Python. Flask hosts a simple web page that streams the video feed from the robot and provides buttons for moving the robot in different directions. The user can access this web interface through any device connected to the same network as the robot, making remote surveillance possible.

   3. Controlling the Robots Mobility

      To give the robot the ability to move and patrol different areas, motor control is handled by the Raspberry Pi's GPIO pins, connected to the L298N motor driver. The motors are responsible for the robots movement, including forward, backward, and turning motions. The web interface allows the user to control the robot remotely. From the web page, the user can send commands to move the robot in a specific direction. This enables manual control; the robot can be patrolled reacting to the presence of faces or objects by moving.

   4. Real-Time Alerts and Notifications

      One of the essential features of the project is the real-time alert system. Whenever the robot detects an unfamiliar face or an object that could signify a potential threat, the system triggers an alert. Through the web interface, the owner receives notifications about any unusual activity, enabling them to take immediate action. The robot also provides audio feedback through a speaker. If the system recognizes a face or object, it announces the detection aloud. For instance, it will say, "[Persons Name] is recognized" or "Unknown person detected." This audio functionality makes the system more interactive and helps users who might not be constantly monitoring the video feed.

   5. Integrating Face and Object Detection

      A key part of the implementation process is the integration of face and object detection. The system captures frames from the PiCamera and processes them using OpenCV. The Haar Cascade classifiers are used to detect faces in real-time, and the Local Binary Patterns Histograms (LBPH) algorithm performs facial recognition to identify known individuals [19]. The Fig. 2 represents the flowchart for the face recognition process. If a detected face matches one in the systems database, it labels the persons name on the screen. Otherwise, it marks the face as "unknown" [20, 21].

      Simultaneously, YOLOv3-tiny is employed to detect and classify objects within the video frames. The YOLO model processes each frame and identifies objects such as pets, household items, or potential security threats. The system labels each recognized object on the video feed, and audio feedback is provided using Pyttsx3, allowing the robot to announce the detected objects or individuals [22].

      Fig. 2. Flowchart of face recognition

   6. Data Collection and Preprocessing

   The effectiveness of the system heavily relies on the quality and quantity of the data used for training the models.

   1. Data Collection

      The first step involves capturing images of faces for the face recognition module. This is achieved by using the Raspberry PiCamera. The user is prompted to capture images of various individuals, which are stored in a structured directory. Each individual's images are organized in a separate folder named after the person, ensuring a clear labeling system that facilitates efficient training of the recognition model. To create a robust dataset, multiple images are captured for each person under different lighting conditions, angles, and facial expressions. This variability enhances the model's ability to generalize and accurately recognize faces in real-world scenarios. In total, approximately 300 images per individual are collected to ensure adequate representation. Additionally, for the objectdetection component, a pre-trained YOLO model is employed, which relies on the COCO dataset containing thousands of labeled images across multiple classes.

   2. Data Preprocessing

   Once the images are collected, preprocessing is essential to prepare the data for model training. For face images, each captured image is converted to grayscale. This conversion simplifies the data, reducing computational load while retaining the essential features necessary for face detection and recognition. The grayscale images are then normalized, which involves adjusting the pixel values to a standard range, aiding the model in learning effectively. Before training the LBPH face recognizer, the images are resized to a consistent

   dimension, ensuring uniformity across the dataset. This resizing process not only speeds up training but also helps the model learn better, as it can focus on the essential features without being distracted by variations in image size. In the case of object detection, the images fed into the YOLO model undergo a specific preprocessing pipeline. This includes resizing to the dimensions required by the model (e.g., 320×320 pixels) and normalizing pixel values to the range (0, 1). These steps ensure that the model can effectively process the images, enhancing detection accuracy.

4. RESULTS AND DISCUSSIONS

   1. Implementation Results

      The surveillance robot successfully achieves real-time face and object recognition, continuous video transmission, and remote control via a web interface. It accurately identifies individuals using face recognition, labeling them as either known or unknown, and detects objects using the YOLOv3 algorithm. The robot streams live video through the Pi Camera, allowing users to monitor their surroundings from a web browser with minimal latency. Additionally, the system provides audio feedback, announcing the names of recognized individuals or objects, enhancing user interaction. This combination of real- time recognition, video streaming, and web-based control makes the robot an effective solution for security, enabling remote monitoring and ensuring the safety of the required environment.

   2. Face and Object Recognition Results

      The surveillance robot's face and object recognition system has shown efficiency in detecting and identifying individuals and objects within its field of view. The system uses a combination of Haar Cascades for face detection and YOLOv3 for object detection. These algorithms work together to provide a real- time analysis of the environment, allowing the robot to identify both known individuals and objects from predefined datasets. In practical testing, the face recognition system was able to successfully detect and identify multiple faces even under varying lighting conditions and camera angles as shown in Fig.

      3. The system accurately labeled the detected faces with either the name of the person or classified them as "unknown" if the face was not recognized. This functionality is critical in surveillance as it allows users to monitor and track who is entering or leaving the environment.

      Fig. 3. Face recognition result

   3. Web-Based Control and Monitoring

      Another important aspect of the robots functionality is its integration with a web-based control system. Through a user- friendly web interface, users can remotely control the robots movements and monitor the video feed. The interface allows users to manage and configure various aspects of the surveillance system, such as controlling the robot, pausing or resuming video transmission and initiating face and object recognition is shown in the Fig. 4. This remote control capability greatly enhances the flexibility and usability of the system, as users can operate the robot from anywhere, even when they are not at home. This feature is especially beneficial for users who travel frequently or need to monitor from a distant location.

      Fig. 4. Web-based Control and Video Transmission

   4. Continuous Video Transmission

      One of the core functionalities of the surveillance robot is its continuous video transmission feature. The video feed is streamed in real-time via the Pi Camera to a web interface, allowing users to remotely monitor the environment from any location through a web browser. The quality of the video stream was stable, with minimal delays or interruptions, making it highly reliable for surveillance purposes. This feature proved to be particularly useful for real-time monitoring, as users can observe live activities, view recognized individuals and objects in real-time, and even receive alerts when the

      system identifies a new face or object. The seamless video streaming is key for providing uninterrupted surveillance, giving users the ability to stay aware of their environment at all times.

   5. Audio Output for Recognized Faces and Objects

      A key enhancement in the system is the integration of real-time audio feedback through a text-to-speech (TTS) system. When the surveillance robot identifies a face or an object, it immediately provides verbal feedback through the speakers, announcing the name of the recognized individual or object. If a face is unrecognized, the system announces, unknown person is detected. This audio output adds an extra layer of interaction and convenience for users, as they do not need to constantly monitor the web interface to know if something has been detected. The verbal announcements serve as instant alerts, notifying users of recognized individuals or objects without needing to visually check the video feed.

   6. Comparison

   Our surveillance robot achieves similar objectives as the robot system described in [23], but with certain improvements and specialized features tailored for residential security. Both systems utilize Raspberry Pi for control, motor drivers (L298N) for mobility, and PiCamera for real-time video transmission. Our surveillance robot offers several distinct advantages over the reference surveillance system [23], both in terms of technology and cost-efficiency. The reference system primarily relies on motion detection using a PIR sensor and employs face recognition through a Haar Cascade Classifier, with notifications sent via email or smartphone alerts. In contrast, our system eliminates the need for a PIR sensor by leveraging more advanced technologies such as YOLOv3 for object detection and LBPH for face recognition. This not only reduces hardware costs but also increases the systems accuracy and versatility, allowing for detailed object identification alongside facial recognition.

   In terms of video streaming, a comprehensive web-based interface is built which enables control of the robot and monitors through the same platform. The added advantage is continuous object and face recognition along with audio output when a person or object is identified. This additional functionality enhances the interactivity and usability of the system for real-time security applications. One of the standout features of our robot is its real-time audio alert system. Instead of waiting for delayed notifications, our system provides immediate voice feedback when a face or object is recognized. This real-time interaction enhances user experience, especially in security scenarios where immediate responses are crucial. The integration of continuous video streaming through a web- based interface also sets our system apart. Users can simultaneously control the robot and monitor the live feed, which simplifies operation and improves overall usability.

   In terms of cost, our surveillance robot is notably more economical. By utilizing a Raspberry Pi 3B+ and replacing the PIR sensor with software-based detection algorithms, we achieve the same level of functionality at a significantly lower

   price. Additinally, our power management setup, which uses a portable power bank for the Raspberry Pi, streamlines the design and further reduces costs. Overall, our surveillance robot combines advanced detection capabilities, real-time interaction, and cost-effective design to provide a superior security solution. It offers more functionality at a lower price, making it a better fit for real-time surveillance needs.

5. CONCLUSION

The surveillance robot developed in this project successfully integrates facial recognition, object detection, and real-time video transmission into a single, user-friendly system. By utilizing Raspberry Pi 3B+, PiCamera, and advanced algorithms such as LBPH for face recognition and YOLOv3 for object detection, the robot provides efficient and accurate surveillance with real-time feedback. The inclusion of an audio alert system enhances immediate response capabilities, making the system more interactive and suitable for security. Additionally, the robot can be remotely controlled and monitored through a web interface, streamlining its functionality and providing users with a seamless experience. This cost-effective solution demonstrates that high-end security features can be achieved using affordable components, making it a viable option for everyday surveillance. The systems ability to differentiate between known individuals and potential intruders, alongside the continuous video feed, adds significant value for ensuring safety in real time. Overall, the project achieves its goal of creating a robust, efficient, and budget- friendly surveillance system suitable for required environments.

REFERENCES

1. Neha Gupta, M. Hammad Zaid, Mohd Bilal, Harshit Saxena, Rahul Dabral, Smart Surveillance System through Computer Vision, International Research Journal of Engineering and Technology (IRJET), Vol. 10, 2023.

2. Rohan Namdeo, Sahil Sharma, Varun Anand, Chanchal Lohi, Smart Automated Surveillance System using Raspberry Pi, International Journal of Recent Technology and Engineering (IJRTE), 2020.

3. Sujeet Potdar, Sanket Ghule, Rohit Nandiwale, Baburao Chandkote, Samarth Patil, Yogini Patil, Raspberry Pi Based Surveillance

   Robot for Home Security, International Research Journal of Engineering and Technology (IRJET), 2022.

4. Chinmaya Kaundanya, Omkar Pathak, Akash Nalawade, Sanket Parode, Smart Surveillance System using Raspberry Pi and Face Recognition, International Journal of Advanced Research in Computer and Communication Engineering, 2017.

5. Shrutika V. Deshmukh, U. A. Kshirsagar, Face Detection and Face Recognition Using Raspberry Pi, International Journal of Advanced Research in Computer and Communication Engineering, 2017.

6. P. Surendra Kumar, V. Venkatesh, S. Geetha Priyanka, Sk. Tausif Ahamed, Video Surveillance Robot with Multi Mode Operation, International Journal of Engineering Research and Technology (IJERT). 2016.

7. Venu Nookala, IOT Surveillance Robot Using ESP-32 Wi-Fi CAM and Arduino, International Journal of Food and Nutritional Sciences. 2022.

8. Nayyar, Anand Puri, Vikram Nhu, Nguyen , Dac-Nhuong, Smart Surveillance Robot for Real-Time Monitoring and Control System in Environment and Industrial Applications. 2018.

9. Shaikh Zeeshan, Priyanka Gaikwad, Nikhil Kare, Swapnali Kapade,

   M.V Korade, A Implementation on Surveillance Robot Using Raspberry-Pi Technology, International Research Journal of Engineering and Technology (IRJET), vol. 4, 2017.

10. Bandi Narasimha, Sudheer Reddy, Sadhanala Mohan, Tibirisettti, Veerababu Muggulla, Sairam, Movable Surveillance Camera using IoT and Raspberry Pi, International Conference on Computing, Communication and Networking Technologies, 2020.

11. Rutuja Dhumal, Kalyani Deshmukh, Kajal Patil, Varsha Surwase, Surveillance Robot Using Raspberry Pi, International Journal of Research in Engineering, Science and Management, Vol. 3, 2020.

12. A. Arthi, G. Kalpana, M. Kavitha, Jaya Surya, Smart Spy

    Surveillance Robotic System, International Journal of Engineering Research and Technology (IJERT), 2018.

13. Sujeet Potdar, Sanket Ghule, Rohit Nandiwale, Baburao Chandkote, Samarth Patil, Yogini Patil, Raspberry Pi Based Surveillance Robot for Home Security, International Research Journal of Engineering and Technology (IRJET), vol. 9, 2022.

14. Abhishek Shigavan, Suraj Temkar, Satish Somase, Vitthal Ingle, Smart Surveillance System Surveillance Robot , IJSART.

15. Faishal Rahaman, Abdullah Al Noman, Muhammad Liakat Ali, Mahfuz Rahman, Design and implementation of a face recognition based door access security system using raspberry pi, International Research Journal of Engineering and Technology (IRJET), Vol. 08, 2021.

16. Mukul Singh, Md Nawaj Khan, Mohammad Makki, Md Irshad, Manohar Hussain, Md Saqib, Camera Based Smart Surveillance System, International Journal on Recent and Innovation Trends in Computing and Communication, vol. 9, 2021.

17. Suraj V Ghorpade, Vrindha Sham Bhatt, Aruna Rao, AI Based

    Armed Surveillance Robot, International Journal of Computer Applications Technology and Research, Vol. 10, 2021.

18. Bhavyalakshmi R, B. P. Harish, Surveillance Robot with Face Recognition using Raspberry Pi, International Journal of Engineering Research and Technology (IJERT), Vol. 8, 2019.

19. F.M. Javed Mehedi Shamrat, Anup Majumder, Probal Roy Antu, Saykot Kumar, Barmon, Itisha Nowrin, Rumesh Ranjan, Human Face Recognition Applying Haar Cascade Classifier, International Conference on Pervasive Computing and Social Networking, 2021.

20. Adlan Hakim Ahmad, Sharifah Saon, Abd Kadir Mahamad, Cahyo Darujati, Sri Wiwoho Mudjanarko, Supeno Mardi Susiki Nugroho, Mochamad Hariadi, Real time face recognition of video surveillance system using haar cascade classifier, Indonesian Journal of Electrical Engineering and Computer Science. Vol. 21, 2021.

21. Arnav Madan, Face Recognition using Haar Cascade Classifier, International Journal for Modern Trends in Science and Technology, Vol.07, 2021.

22. Pranav Adarsh, Pratibha Rathi, Manoj Kumar, YOLO v3-Tiny: Object Detection and Recognition using one stage improved model, International Conference on Advanced Computing and Communication Systems, 2020.

23. H. Meddeb, Z. Abdellaoui, F. Houaidi, Development of surveillance robot based on face recognition using Raspberry-PI and IoT, Microprocessors and Micro-systems, volume 96, 2023.