IoT- Based Smart Bag with Image Recognition for Automated Luggage Item Tracking

IoT- Based Smart Bag with Image Recognition for Automated Luggage Item Tracking

Bhavadharani M Computer Science and Engineering Thiagarajar College of Engineering

Madurai, India

Evelyn David Computer Science and Engineering Thiagarajar College of Engineering

Madurai, India

Harishma D Computer Science and Engineering Thiagarajar College of Engineering

Madurai, India

Dr.N Anita Assistant Professor Computer Science and Engineering Thiagarajar College of Enginneering

Madurai, India

Abstract Managing and tracking personal belongings during travel has always been a critical challenge, with traditional technologies like RFID (RadioFrequency IDentification), QR codes (Quick Response), and barcodes attempting to address the issue. However, these systems are limited by their inability to provide real-time updates, reliance on manual scanning, and lack of robust integration with mobile platforms. Existing solutions often fail to deliver seamless functionality, timely notifications, and user-centric experiences. Leveraging IoT technology, an embedded ESP32 microcontroller (Espressif Systems) integrated with a high-resolution camera and YOLOv3 (You Only Look Once, Version 3) enables automated item tracking and identification, overcoming the limitations of earlier systems. YOLOv3 leverages its real-time object detection capabilities, accurate multi-scale predictions, and the efficient Darknet-53 backbone to achieve robust detection and classification of objects, ensuring high precision even for smaller and more challenging targets. The system processes data in real-time and delivers updates via a mobile application over Wi-Fi (Wireless Fidelity) with features such as real-time notifications, automated item management, and integration with other smart systems for enhanced convenience and security. Additionally, the system achieves key parameters such as time efficiency, accurate item tracking, and real-time notifications, simplifying packing, reducing manual effort, and ensuring seamless organization. By combining automation and user- friendly interfaces, this innovation redefines the standards of smart travel technology, offering a comprehensive solution for modern luggage challenges while enhancing the travel experience with greater security and ease.

Keywords ESP32-CAM, Wi-Fi, YOLO v3, Image Processing

1. INTRODUCTION

   Travelers often face significant challenges in managing and keeping track of their belongings, particularly during trips where the risk of losing or misplacing items is heightened. Traditional methods, such as manual packing lists, labeling systems, or tagging technologies like RFID, QR codes, and barcodes, have notable shortcomings. These methods typically lack real-time tracking capabilities, rely heavily on user input, and do not provide sufficient automation to ensure seamless management. Consequently, travelers experience inefficiencies, increased chances of oversight, and heightened stress during packing, repacking, or transit scenarios [1]. The absence of intelligent solutions in luggage

   management contributes to various issues, including misplaced items, delays caused by disorganization, and the anxiety of staying organized throughout the journey. For instance, travelers often struggle to recall whether they packed essential items or left something behind, leading to last-minute disruptions or the need for replacements [3]. An advanced, intelligent luggage tracking system that integrates real-time notifications and automates organizational tasks can effectively address these challenges. Such a solution would minimize manual effort, provide instant alerts about misplaced or forgotten items, and ensure a smoother travel experience [5]. By offering features like automated inventory tracking, location-based notifications, and seamless integration with travel itineraries, this system has the potential to revolutionize luggage management. Incorporating modern technologies, such as IoT-enabled devices, can enhance efficiency and reduce stress, offering travelers peace of mind [6]. Frequent travelers, in particular, stand to benefit significantly from such solutions, which can address typical challenges related to packing and luggage tracking.Ultimately, this approach would contribute to a more enjoyable and stress-free travel experience, fostering greater satisfaction throughout the journey [6].

2. LITERATURE REVIEW

   The Smart Luggage Carrier System was to design an autonomous luggage carrier that would trace a predetermined path; track its location, being assisted by the GPS; and detect obstacles through ultrasonic sensors, reporting theft or loss. SLCS (Smart Luggage Carrier System) implemented key features such as autonomous movement, real-time location tracking, obstacle detection, and theft/loss alerts using Arduino, sensors, and GPS modules [1].

   The IoT-based Smart Bag Using Arduino was to design an innovative luggage solution that would automate movement, improve user control, and ensure the safety of belongings during travel. Incorporating motion detection, obstacle avoidance, and GPS-based location tracking was done with the ability to provide user-friendly, smooth travel experiences [2]. Users could control the movement of the bag and track its location using a dedicated mobile application.

   The project provides better security and Fingerprint recognition, and linked to GPS technology [3]. The findings of the fingerprint-based security and GPS tracking were successfully implemented and integrated to increase safety and monitoring of luggage during travel.

   This project targeted at integrating STM32 microcontrollers (STMicroelectronics), GPS, Bluetooth and a mobile app with the goal of making luggage more safe and convenient. It includes secure unlocking via Bluetooth, real-time GPS tracking and proximity alerts to ensure the safety of the suitcase [4]. The outcome of the smart suitcase project includes features like Bluetooth unlocking and so on.

   This smart IoT bag combines features like RFID, biometric authentication and solar-powered energy , providing safe access ,smart inventory management, and eco-friendly operation [5]. It elevates travel convenience with smart home automation and sustainable design.

   The backpack uses renewable energy through a solar panel that is used to power its system which includes microcontrollers and sensors. When the zipper is opened, the microcontroller sends a notification to users via an android app. In addition to that the app supplies updates on the battery charge level, which ensures efficient energy management [6].

   A smart baggage tracing system using RFID tags and IOT addresses aviation challenges like lost or damaged baggage. Integrated with a cloud server, it operates at check-in and check-out, securely storing passenger and airline details with a security-enhanced algorithm [7]. RFID readers monitor baggage , track its real-time location, and record data in the cloud, enabling passengers to access identification details for seamless retrieval.

   This paper explains a method for tracking airport baggage handling using the Blynk IoT platform. It addresses challenges in baggage management and security, analyzing existing issues and proposing solutions [8]. A Blynk-based mobile app enables RFID baggage tracking, and a four-zone segregation conveyor belt enhances efficiency and flow in high-volume airports.

   This paper introduces a battery-powered GPS tracker with IoT integration, offering continuous tracking and an SOS feature. It uses a microcontroller and GPS module to sendlocation data via GSM to registered users and a central MongoDB-Node.js database, accessible through an AWS- hosted map interface [9]. Users receive SMS alerts with Google Maps links, while the SOS mode notifies nearby devices, enhancing security and connectivity.

   Table 2.1 Decision Matrix

   Table 2.1 highlights the distinct features of each product, with the proposed solution standing out due to its comprehensive capabilities such as image-based security, low power consumption, and efficient item tracking.

3. PROPOSED WORK

   The aim of this project is to create a organized luggage item tracker that reduces the chances of misplaced items during travel. The system will efficiently identify and authenticate items in luggage, hence reducing the effort and t ime required for repacking.

   The objectives of this project are as follows,

   • To automate tracking, where cameras capture and track items in the smart bag.

   • To remind users in real-time if there are any items left behind during repacking via the mobile app.

   • To have a current and correct list of items to enhance organization and reduce the possibility of leaving important belongings behind.

   • To ease the process of packing and unpacking for a more organized and stress-free experience.

     The solution suggested is a smart bag system that incorporates IoT devices such as cameras and microcontrollers to track and manage items in the luggage automatically. Through image recognition technology, the system captures and authenticate items placed in the bag in real-time. There is an application of a mobile app to inform users of lost or forgotten items, thus checking unnecessary manual checks and improving travel efficiency. The goal of the system is to limit the amount of time used on repacking and packing so that users receive quick notices of missing items and eventually reduce stress from mismanagement of personal items on trips.

     1. System Design:

        The Figure 3.1 illustrates the system architecture of this project as a whole. It starts with the luggage that has an ESP32-CAM installed in it, which takes pictures of the contents of the luggage whenever something is placed or removed. The embedded camera functions autonomously, activated whenever a packaging event occurs, ensuring that visual data is always up-to-date without the need for user intervention. The camera communicates directly with the rest of the system through Wi-Fi enabling real-time data transmission for further processing.

        The smart bag system consists of six core components that automate item tracking and management.

   • Luggage with Integrated Camera: The bag has a built- in camera that captures images of its contents, either automatically or manually triggered by the user, to monitor items.

   • Image Processing Module: The captured images are analyzed using object detection algorithms, such as YOLOv3, to identify and classify items, ensuring precise recognition.

   • Item Database: Identified items are stored in a database with details such as images, names, categories, and descriptions for easy retrieval.

   • User Interaction: Users can manage items (add, edit, delete) through an intuitive interface, allowing full control over the contents of their bag.

   • Mobile App: The app serves as the user interface, providing real-time updates, notifications, and controls for efficient item management.

   • Backend Server: The server synchronizes data between the mobile app and the database, ensuring smooth communication and secure data handling.

     Fig 3.1 System design of the smart bag for automated item tracking.

     YOLO V3:

     Fig 3.2 Structure of YOLOv3.

     YOLOv3 is a cutting-edge object detection algorithm renowned for its real-time performance and exceptional accuracy [13]. The architecture efficiently predicts object

     locations and classifications using a single neural network pass. It improves upon its predecessors with a deeper architecture, better feature extraction, and multi-scale predictions, making it suitable for tasks like surveillance, traffic monitoring, and autonomous driving.

     Figure 3.2 illustrates the hierarchical structure and multi- scale detection mechanism. The input image is divided into a grid, with each cell responsible for predicting bounding boxes for objects. Convolutional layers extract hierarchical features using alternating 3×3 and 1×1 filters, grouped into convolutional sets. Residual blocks enhance gradient flow and allow deeper networks to learn complex patterns. The network progressively downsamples feature maps for large- scale object predictions while utilizing upsampling to support medium- and small-scale predictions, integrating features from earlier layers to improve accuracy. Predictions are generated at three scales, encompassing bounding box coordinates, object confidence scores, and class probabilities [13].

     YOLOv3 balances speed and accuracy through simultaneous classification and bounding box regression. Its multi-scale detection and residual blocks enhance precision, especially for small objects. Innovations like the Darknet-53 backbone network and advanced anchor box mechanisms distinguish it from YOLOv2, achieving higher accuracy and better handling of complex scenes. The diagram highlights these advancements, reinforcing YOLOv3's position as a robust solution for real-time object detection [13].

     B. Working Principle:

     The system functions through the integration of image processing and sensor technology. The key steps in its operational process include:

   • Item Detection:

     When an object is placed inside the smart bag, the embedded ESP32-CAM captures the image of the item and processes it using image recognition algorithms.

     The proximity sensors also detect changes, confirming the presence or removal of items.

   • Real-Time Item Tracking:

     The captured data is transmitted to a mobile application via Wi-Fi or Bluetooth for real-time inventory management. If any of the item is not recognized during unpacking then the app alerts the user.

   • User Interaction:

     The mobile app provides an interactive platform for the users by displaying the inventory list, that allows manual management of items where users can add, edit or delete items from the list ensuring flexibility in managing their luggage content.

   • Security Mechanism:

     When an unauthorized person tries to access the bag or remove an item from the smart bag, the system ensures security by sending real-time alerts to the mobile app.

4. SCOPE AND REQUIREMENTS

   (

   ) =

   1 1+

   Vol. 14 Issue 04, April-2025

   (1)

   SCOPE:

   • Automated Tracking: Develop an IoT-based system for automated detection and item tracking inside the bag using ESP32-CAM.

   • Mobile Application: Build a user-friendly mobile app for managing the inventory list, receiving real-time notifications, and manually modifying the list, including adding, deleting, and editing items, with timely notifications for item changes.

   • Local Database: Implement local storage (MySQL) to store data on packed items and track packing history for fast access.

   • User Control: Enable users to manually add, delete, and edit items in the inventory list and provide timely notifications for item changes.

   • Future Enhancements: Integrate GPS for global tracking and improve object detection accuracy for enhanced image recogniton.

     REQUIREMENTS:

     To ensure seamless functionality, the project must fulfill the following requirements:

   • Adaptability to lighting conditions.

   • Stable Bluetooth/Wi-Fi connection.

   • Optimized image processing.

   • High power usage reduces battery.

5. RESULTS AND DISCUSSION

   This project combines hardware and software to build a smart luggage management system. It uses an ESP32-CAM to take pictures and connects via Bluetooth or Wi-Fi. With real-time tracking and object detection, the system helps keep track of items in the luggage easily and efficiently.

   The mobile app, built with Flutter, makes it easy to add, edit, and delete items while keeping a local record of your packing list and history. It also includes helpful features like checklists, item tracking with timestamps, and a way to manage deleted items. These features make packing and unpacking more organized, reducing stress and making luggage management simpler and more convenient.

   YOLOv3 is a powerful object detection system that quickly and accurately identifies objects in real time. It uses a smart network (Darknet-53) to learn important features, groups objects better with k-means clustering, and improves accuracy with special calculations. This makes it great for detecting different objects of various sizes with high speed and precision.

   Formulaic Comparison: YOLOv3 uses Darknet-53 as its backbone, which has, accuracy is enhanced through the combined efficiency of FLOPs (Floating Point Operations blocks. This ensures a balance between computational efficiency and representational power. Per Second) in various layers, further optimized by the integration of residual learning. Darknet-53's architecture combines 53 convolutional layers with optimized residual blocks. This ensures a balance between computational efficiency and representational power.

   Mathematical Basis: YOLOv3 replaces the softmax function with independent logistic regression for each class p(c), calculated using equation 1:

   where z represents the output from the final fully connected layer [13].

   Anchor Box Calculation: YOLOv3 uses predefined anchor boxes optimized via k-means clustering on the dataset, minimizing the IoU loss between ground truth and predicted boxes [13] using equation 2:

   = 1 ( ) (2)

   Speed Formula:

   Inference time is calculated using the equation 3:

   ()

   = (3)

   ( )

   YOLOv3's optimized Darknet-53 ensures lower inference time by reducing unnecessary computations, evidenced by its 51 ms inference time compared to RetinaNet's 198 ms [13].

   Precision Formula:

   Precision measures how many of the detected objects are actually correct. Its calculated by comparing the correctly identified objects (True Positives) to the total detected objects (True Positives + False Positives). YOLOv3 improves precision by using advanced techniques to detect objects of different sizes more accurately.

   YOLOv3 stands out from other methods like RetinaNet because it is both fast and accurate. It uses smart techniques like IoU loss for better object detection and optimized anchor boxes for precise placement. It can detect multiple objects at once while maintaining both speed and accuracy, making it ideal for real- time applications.

   Table 5.1 Hardware and Software Components.

   Component	Details
   Hardware Dependencies	ESP32-CAM, Bluetooth Module, Wi-Fi Module.
   Software Dependencies	Mobile App, Object Detection Algorithms, SQLite/MySQL (Local Database)
   Connectivity Dependencies	Wi-Fi, Bluetooth

   Table 5.1 lists the main parts of the system, such as the ESP32- CAM, Bluetooth and Wi-Fi modules, a mobile app, object detection algorithms, a local database, and wireless connectivity The system uses an ESP32-CAM to take pictures and connects through Bluetooth and Wi-Fi for real-time tracking. A mobile app helps track and manage items using object detection, while a local database stores the item list and packing history. Wi-Fi and Bluetooth allow smooth communication between the smart

   bag and the app, making everything work efficiently in real time. The app, built with Flutter, makes it easy for travelers to manage their items. They can add, edit, or remove items as needed. If a user moves an item from their main luggage to a handbag, they can delete it from the list. The app will then ask if theyre keeping it somewhere else, ensuring accurate tracking. This ensures that even if an item is stored separately, its location is recorded in the deleted section of that specific trip. This feature enhances visibility, giving users complete awareness of the status and location of their belongings throughout their travels. The app empowers travelers to remain organized and informed, transforming the often-chaotic process of packing and unpacking into a manageable and stress-free experience.

   Figure 5.1 illustrates the "Effortless Trip Creation Interface," where users can conveniently add a new trip by entering the trip name and date. This streamlined design ensures a user- friendly experience, enabling quick setup and easy management of trips directly within the mobile app.

   Fig 5.1. Effortless Trip Creation Interface

   Figure 5.2 shows the "Creation of Checklist Items" interface, enabling users to add items to a trip by entering their names. This feature facilitates efficient checklist creation and tracking of essential trip items within the app.

   Fig 5.2 Creation of checklist items.

   Figure 5.3 demonstrates the "Track Item Additions and Checks" interface, displaying each checklist item with its addition and check-in timestamps. Users can view, edit, delete items, and track checked items, aiding in the organized packing and unpacking processes.

   Fig 5.3 Track item additions and checks.

   Fig 5.4 Confirm intentional item removal.

   Figure 5.4 illustrates the confirmation dialog that appears when a user attempts to delete an item, prompting them to verify if the item is being intentionally removed. The user has the option to specify a location where the item is being placed before finalizing the action by selecting "Delete" or "Done".

   Fig 5.5 Viewing deleted items.

   Figure 5.5 shows the deleted items screen, detailing removed items with their location and deletion timestamp. This feature ensures time-efficient, accurate tracking and real-time notifications, simplifying packing, reducing manual effort, and improving luggage management.

6. CONCLUSION AND FUTURE WORK

The smart bag system effectively addresses the common challenges of managing and securing items during travel. By utilizing IoT technologies such as cameras and microcontrollers, the system automates item tracking, ensuring real-time updates and reducing the risk of misplaced belongings. The mobile application enhances user experience by providing timely notifications and manual control over the item list, making the packing process more efficient and stress-free. This innovative solution bridges the gap between traditional luggage management and modern IoT-driven automation, offering a reliable, user-friendly, and secure system. In the future, the focus will be on improving image recognition accuracy, reducing power consumption, and incorporating GPS for global tracking, further enhancing the systems efficiency and versatility.

REFERENCES

1. Terence, E., Siva Reddy, K. Madhu Reddy, and T. Veera Reddy. "Smart Luggage Carrier System." In Proceedings of the Intrnational Conference on Innovative Computing & Communication (ICICC). 2022.

2. Talib, Shazia, Tanvi Save, Arwa Ujjainwala, and Arjun Jaiswal. "IoT- based smart bag using Arduino." international journal for research in applied science and engineering technology 10, no. 11 (2022):1486- 14993.

3. Valarmathy, S., K. Radhika, K. Bashkaran, S. Selvarasu, and C. Srinivasan. "Intelligent Baggage Management in Airports: A Cognitive IoT Approach for Real-Time Tracking, Optimization, and Passenger Engagement." In 2023 7th International Conference on Electronics, Communication and Aerospace Technology (ICECA),

   pp. 1876-1880. IEEE, 2023.

4. Al-Shareeda, Mahmood A., and Selvakumar Manickam. "IoT Technology and STM32 Microcontroller Based Design of Smart Suitcase." Journal of Computer Science & Computational Mathematics, Volume 13, Issue 1,

   March 2023, DOI: 10.20967/jcscm.2023.01.002

5. Khader Basha Sk, B Pravallika, D Swathi, P Sri Lakshami Durga and Ch SivaTeja, Intelligent Travel Companion: The IoT-Enabled Smart Bag, International Journal for Modern Trends in Science and Technology, 2024, 10(02), pages. 70-76. [Online]. Available: https://doi.org/10.46501/IJMTST1002010 .

   1. Pradeep, S., Srivani, B., Supriya, B.N., Bhavana, D. and Srinidhi, K., 2024. AUTONOMOUS LUGGAGE

      HANDLING SYSTEM. International Journal of Information Technology and Computer Engineering, 12(3), pp.902-911.

   2. Kabir, A.T., Saha, P.K., Kibria, G., Ta-sin, A.J. and Zishan, M.S.R., 2020, November. A comprehensive smart IoT tracker for the children, elder, and luggage with the assistance of mobile app. In 2020 International Conference on ICT for Smart Society (ICISS) (pp. 1-5). IEEE.

   3. Farhadi, Ali, and Joseph Redmon. "Yolov3: An incremental improvement." Computer vision and pattern recognition. Vol. 1804. Berlin/Heidelberg, Germany: Springer, 2018.

6. Sharma, Prince, Himanshu Kumar Jaiswal, Prashant Tyagi, Pulkit Mittal, and Shubham Shukla. "Designing a Smart Bag Using Cyber Physical Devices." International Journal for Research in Applied Science & Engineering Technology (IJRASET) ISSN: 2321- 9653; IC Value:

   45.98; SJ Impact Factor: 7.538 Volume 10 Issue IV Apr 2022. [Online]. Available: www.ijraset.com .

7. A.Singh, S.Meshram, T.Gujar and P.R.Wankhede, Buggage Tracing and handling system using RFID and IoT for airpots, 2016 International Conference on Computing, Analytics and Security Trends (CAST), Pune, India, 2016, pp. 466-470,doi: 10.1109/CAST.2016.7915014.

8. Alghamdi, A., Alansari, S. and Alshamsi, R., 2024, June,A Proposed Solution for Tracking Air Baggage Handling System Using IoT Platform. In 2024 Advances in Science and Engineering Technology Internation Conference (ASET) (pp.01-05). IEEE.

9. Jacob, R.O., Alyaan, S.M., Nikitha, K. and Murthy, H.N., 2022, March. IoT Based GPS tracking system with SOS Capabilities. In 2022 International Mobile and Embedded Technology Conference (MECON) (pp. 72- 75). IEEE.

10. Valarmathy, S., Radhika, K., Bashkaran, K., Selvarasu,

S. and Srinivasan, C., 2023, November. Intelligent Baggage Management in Airports: A Cognitive IoT Approach for Real-Time Tracking, Optimization, and Passenger Engagement. In 2023 7th International Conference on Electronics, Communication and Aerospace Technology (ICECA) (pp. 1876-1880). IEEE.