Automated IC Sorting Using YOLOv3-Based Text Recognition with ESP32-Controlled Robotic Arm

Automated IC Sorting Using YOLOv3-Based Text Recognition with ESP32-Controlled Robotic Arm

Rishi Hinge, Kaustubh Kulkarni, Harsh Challani

*Department of Electronics and telecommunication Engineering, Sardar Patel Institute Of Technology Institute of Technology, Mumbai

Dr. Manisha Bansode

Department of Electronicsand telecommunication Engineering

Abstract – Identifying and sorting integrated circuits (ICs) by their printed markings is a recognized bottleneck in electronics assembly and inventory management. Classical optical character recognition (OCR) and feature-based computer vision struggle with small, reective IC labels. This work presents a cost- effective automated sorting platform utilizing YOLOv3-based text detection, ESP32-CAM image acquisition, and ESP32-driven robotic pick-and-place. With only 80 labeled images for training across four classes, the system achieves 70 percent classication accuracy and an average sorting duration of 50 seconds per IC. Discussion focuses on observed accuracy constraints and mechan- ical bottlenecks, and outlines steps toward further improvement for small-scale laboratories and educational applications.

Index TermsYOLOv3, IC Sorting, ESP32, OCR, Image Processing, Robotic Arm, Deep Learning

1. Introduction

   Integrated Circuits (ICs) are among the most fundamental components in electronic systems, serving as building blocks in devices ranging from consumer gadgets to industrial au- tomation systems. With increasing global demand for elec- tronic products, manufacturers and service laboratories handle a wide variety of ICs during assembly, testing, repair, and inventory classication. Although these ICs differ in electrical characteristics, functionality, and internal architecture, many share identical physical package designscommonly small, rectangular black epoxy casings. This makes visual differen- tiation extremely challenging when relying only on physical dimensions.

   Under traditional conditions, ICs are sorted manually by reading printed markings that indicate part number and man- ufacturer details. This process is slow, tedious, and prone to human error, especially in environments handling large quantities of mixed ICs. Errors in sorting can lead to improper circuit assembly, damaged systems, safety issues, or nancial losses. As a result, there is a need for automated IC recognition systems that can reliably identify IC types and sort them accordingly.

   Conventional automated object-sorting techniques rely on features such as shape, size, or color. Such methods, while effective for general object classication, are insufcient for ICs where the only distinguishing element is the text printed on the surface. The text, however, is often small, poorly printed, affected by surface reections, and inconsistent in font. These factors pose signicant challenges to classical OCR-based approaches.

   Deep learningbased object detection has emerged as a robust solution to visual recognition challenges. YOLO (You Only Look Once) models, known for fast inference and strong performance, offer the capability to detect objects and text-like patterns in real time. YOLOv3, in particular, works effectively with small object features, making it suitable for recognizing printed IC labels.

   This project presents a robotic IC sorting system using YOLOv3 for label recognition and an ESP32 microcontroller to control servo and stepper motors for physical sorting. The system captures an image of an IC, identies its type, and sorts it into a designated bin. This integration of machine learning with embedded automation provides a low-cost, portable, and scalable solution.

2. Literature Review

   Automated object recognition and sorting have been ex- tensively studied across industrial, agricultural, and logistics sectors. Traditional systems relied heavily on manual inspec- tion, which was labor-intensive, prone to human error, and not scalable for high-volume operations. The advancement of computer vision and machine learning has signicantly trans- formed inspection, classication, and robotic control systems. This section provides a detailed survey of prior work on sorting systems, printed-text detection, optical character recognition (OCR), and deep learning-based classication approaches. It also highlights limitations of previous work and the need for intelligent, text-based detection of Integrated Circuit (IC) components.

   1. Early Automated Sorting Systems

      Early attempts at developing sorting machines used mechan- ical and basic electronic sensors for classication. Infrared (IR) detectors, magnetic sensors, and capacitive sensors could distinguish between simple material types (metal, plastic, organic), yet they failed when objects shared similar surface properties. Although inexpensive and fast, these technologies lacked the ability to identify unique object classes.

      Color-based machine sorting later became commercially relevant. Color sensors and cameras could classify fruits, packed components, and waste materials based on hue vari- ations. However, sorting similar-colored items remained dif- cult. Furthermore, illumination changes could degrade color recognition performance drastically.

      These limitations reveal that low-level sensing is insufcient where subtle distinguishing features such as engraved or printed text are required, such as in IC packages.

   2. Computer Vision and Classical Image Processing Tech- niques

      With the emergence of computer vision, classical image- processing techniques became popular for classication tasks. Methods such as edge detection, Hough transforms, binary thresholding, and SIFT/SURF feature extraction provided au- tomated ways of identifying objects and features. For printed text detection, SIFT-based matching techniques were used to identify objects based on known templates.

      However, these handcrafted feature-based systems are highly sensitive to background noise, camera angle, illumina- tion variation, reection, and low-contrast text. IC markings are often small, worn out, highly reective, and closely resem- ble one another, causing traditional techniques to fail under practical conditions.

      Research on label reading and barcode recognition showed improvement by implementing adaptive thresholding and morphology-based segmentation. Still, these systems require clean, uniform backgrounds and high contrast. Consequently, their use is impractical in many scenarios where IC sorting is required, especially in environments like repair shops and low-cost manufacturing setups where ambient lighting changes frequently.

   3. OCR-Based Text Recognition

      Optical Character Recognition (OCR) is widely applied in document digitization, postal code reading, license plate recognition, receipt scanning, and warehouse logistics. Several studies evaluated OCR engines such as Tesseract for industrial marking detection. OCR systems work best on structured text with known fonts and minimal noise; ICs, however, present complicated challenges small text size, unstructured format, poor contrast, and curved/angled surfaces.

      Standard OCR pipelines include

      1. Region detection

      2. Cropping

      3. Text line segmentation

      4. Character recognition

         In ICs, the text region is often extremely small; failures occur at the region-detection stage itself. Low-cost cameras cause blur and noise, and reetive epoxy packaging intro- duces glare. OCR systems cannot reliably detect class labels printed on ICs without assistance from more powerful object- detector models.

   4. Deep Learning for Object Detection

      Deep learning revolutionized computer vision, particularly with convolutional neural networks (CNNs). CNN-based mod- els demonstrated state-of-the-art performance across object detection, feature extraction, segmentation, and classication

      tasks. Early deep learning detectors, such as R-CNN, Fast R- CNN, and Faster R-CNN, offered strong detection accuracy at the cost of slow inference speeds.

      YOLO (You Only Look Once) introduced by Redmon et al. marked a major step forward in achieving real-time object detection. The principle behind YOLO is to treat detection as a regression problem, predicting bounding boxes and class probabilities simultaneously. YOLOv3 improved recognition at multiple scales, making it highly suitable for small-object detection. This makes YOLOv3 a strong candidate for detect- ing text printed on the small surfaces of ICs.

      Several industrial applications used YOLO for object recog- nition, including:

      • Fruit and vegetable sorting

      • Waste classication

      • Tool identication

      • Defect analysis in manufacturing

      YOLOv3 has been applied in logistics to detect printed labels and QR codes, but its application in IC printed text de- tection remains unexplored in most of the literature. YOLOs ability to extract global image features and classify objects regardless of lighting and orientation makes it well-suited for IC label recognition.

   5. IC-Specic Recognition and Sorting

      Only limited research has investigated automated IC recog- nition. Traditional IC sorting systems rely on optical character matching, template matching, or manual OCR processing. Due to diverse packaging standards and poor contrast text, much existing research is restricted to highly controlled industrial environments.

      The ICDAR (International Conference on Document Anal- ysis and Recognition) 2021 competition on IC text detection identied the challenge posed by IC markings very small text sizes, reections, varied fonts, and deterioration. The competition motivated research into combining text spotting

      + OCR + segmentation models. However, most existing solu- tions require signicant computational power and are aimed at high-end industries rather than low-cost embedded setups. The solution presented in this work bridges this gap by employing YOLOv3 for text recognition instead of OCR, sim- plifying the recognition pipeline and reducing computational dependency. This approach enables practical deployment for

      small-scale environments.

   6. Robotics-Based Sorting Systems

      Robotic arms have been successfully used in industrial sorting applications. Most academic efforts utilize Arduino or Raspberry Pi to drive two- or three-degree-of-freedom arms for object manipulation. Computer-vision-driven robotic control has been applied in:

      • Automated waste segregation

      • Fruit handling

      • PCB component pick-and-place

      • Parcel sorting

      Systems in literature typically use color/object shape to determine sorting category. Our contribution differs by using printed text as the classication basis. Since most ICs share identical dimension and shape, robotic control must rely on text, not geometry.

      Another innovation of the proposed work involves the combination of YOLOv3 detection with an ESP32-driven mechanical sorting system. The ESP32 allows low-cost real- time communication with PC inference and motor actuation, allowing rapid integration with robotic systems.

   7. Summary and Key Observations

   The literature suggests:

   • Traditional methods are insufcient for IC sorting.

   • OCR struggles with low contrast and non-standard text.

   • YOLOv3 offers strong performance for small-region text detection.

   • Little research exists in YOLO-based IC sorting.

   • Combining deep learning with low-cost robotics is a promising direction.

   Thus, the proposed system contributes a novel design, in- tegrating YOLOv3-based text recognition with pick-and-place robotic sorting. This approach overcomes challenges noted in prior work and provides a low-cost, scalable solution suitable for electronics labs and small manufacturing facilities.

3. Software

   The proposed IC sorting platform is built on a modular software architecture that orchestrates image processing, intel- ligent detection, classication handling, and real-time system actuation. Each module is strategically decoupled to allow for exible upgrades and future scalability.

   1. YOLOv3 Network At the core of the recognition engine, YOLOv3 runs on an external computer, analyzing images received from the ESP32-CAM. Originally designed for large- scale object detection (e.g., the COCO dataset), YOLOv3s architecture is adaptable for ne-grained tasks such as IC text pattern detection. For this project, the network was retrained using a custom dataset of IC images, rening its ability to localize and classify subtle variations in printed label patterns on the IC surface. The network produces bounding boxes and condence scores for each detected region, outputting the most likely IC type present in the frame.

   2. Preprocessing Pipeline To maximize detection accuracy, a targeted preprocessing sequence is employed:

      Resizing: Input images are scaled to match the input di- mensions expected by YOLOv3, preserving aspect ratio and minimizing distortion.

      Noise Filtering: Denoising algorithms are applied to sup- press sensor and environmental noise, enhancing the clarity of text patterns.

      ROI Selection: The region of interest is automatically or manually dened within each frame, focusing the detection window on areas where IC labels are expected.

      Normalization: Intensity normalization equalizes contrast across the image, further improving the visibility of faint markings and ensuring consistent network performance.

      This module is easily recongurable to accommodate future imaging upgrades or new image modalities.

   3. Communication Interface Once YOLOv3 makes a pre- diction, the detected class label is encoded and transmitted to the ESP32 microcontroller via UART (serial) or over Wi-Fi. The communication layer incorporates acknowledg- ment (ACK) exchanges, which ensures message delivery in- tegrityresending packets when errors or synchronization issues occur. This robust communication protocol allows for real-time feedback, enabling responsive error handling and providing a channel for system diagnostics or status updates.

   4. Embedded Firmware On the embedded side, the ESP32 rmware is responsible for real-time control and physical actuation. Main functions include:

   Parsing Classication Messages: Incoming labels from the PC are decoded and mapped to predened movement routines. Motion Command Execution: The rmware triggers precise servo signals for the gripper and coordinates the stepper motor to rotate the sorting tray, aligning it with the correct output

   bin.

   Signal Management: All actuator timing, safety checks, and sequential coordination are handled internally, maximizing reliability and minimizing the risk of hardware collision.

   The rmware architecture is intentionally modular: expand- ing to accommodate more IC classs, adding new sorting bins, or upgrading to conveyor-based IC delivery can be implemented with minimal redesign. This structure also paves the way for future enhancements such as direct edge inference or more complex robotic behaviors.

   Fig. 1. Overall System Architecture of the Proposed IC Sorting Platform

4. Hardware

   The hardware comprises ve major components: ESP32, camera module, servo motor, stepper motor, and power mod- ule.

   1. ESP32 Microcontroller

      The ESP32 serves as the control unit for system actuation and communication. Its dual-core architecture and integrated Wi-Fi/Bluetooth allow rapid real-time processing and data

      transfer. It is responsible for receiving classication results and executing motor control routines. Its GPIO pins are used to interface with servo and stepper drivers.

   2. Camera Module

      An ESP32-CAM or USB camera is mounted above the IC input platform and captures top-view images. Factors such as resolution, frame rate, and illumination signicantly impact performance. The camera provides an image stream to the computer for YOLOv3 processing.

   3. Servo Motor

      A servo motor operates a small gripper mechanism that picks up the IC. Servos allow precise angle control using PWM signals, enabling accurate gripping and placement. Their compact size and low power consumption make them suitable for lightweight manipulation.

   4. Stepper Motor

      A stepper motor rotates the sorting platform, enabling multiple bin positions. Accurate position control allows bins to be mapped to specic IC classes. A stepper driver (A4988/DRV8825) interfaces with the ESP32, converting con- trol pulses into rotational motion.

      Fig. 2. Battery

   5. Power System

   The system uses regulated lithium-ion battery power. Servo motors require dedicated supply for stable torque, whereas mi- crocontroller and camera operate from 5V/3.3V rails. Brush- back protection and capacitors ensure noise reduction.

5. Methodology

   The development and evaluation of the automated IC sorting system proceeded through a systematic, modular pipeline detailed as follows:

   Step 1: Data Collection Five representative IC models (LM358, NE555, 7400, 7805, and 7809) were chosen for this study. For each class, twenty images were captured using the ESP32-CAM under controlled lighting conditions to minimize shadowing and specular glare. To ensure model robustness, the

   orientation and tilt of each IC were varied between captures while maintaining xed camera distance and exposure settings. Step 2: Image Preprocessing All images were initially cropped to eliminate background clutter, ensuring each frame centered the IC of interest. Images were then resized to a standardized resolution (416×416) to match YOLOv3 input requirements. Contrast and sharpness enhancements were ap- plied to improve the visibility of printed text, followed by light

   ltering to suppress background noise.

   Step 3: YOLOv3 Detection and Classication The curated image set was processed using a ne-tuned YOLOv3 network. The model scanned each frame, predicting bounding boxes and corresponding class probabilities for detected text. Only predictions surpassing a preselected condence threshold were accepted to reduce false positives and ensure reliable classi- cation.

   Step 4: Communication Protocol Class predictions were serialized and transmitted via UART from the host PC, which performed inference, to the ESP32 microcontroller. Robust handshaking protocols were implemented to acknowledge message receipt and prevent data loss during serial transmis- sion.

   Step 5: Robotic Sorting Mechanism The ESP32 acted as the central coordinator for motion control. Upon receiving a valid class label, it actuated the servo-based gripper to seize the IC and drove the stepper motor to align the sorting tray with the target bin. The controlled sequence ensured precise placement without collision or misclassication.

   Step 6: Performance Evaluation After system integration, sorting accuracy and cycle time per IC were measured. This involved logging correct versus incorrect deposits and record- ing the elapsed time from image capture to nal sorting. These metrics offered quantitative assessment and guided further tuning.

   Pipeline Modularity Each methodological stage was de- signed for independence, enabling straightforward future mod- icationssuch as scaling the image dataset, increasing the number of sort bins, or replacing manual placement with a conveyor. This modularity ensures adaptability to evolving re- quirements and supports the integration of improved detection models or enhanced mechatronics.

6. Algorithms

   1. 6.1 YOLOv3 Text Detection Algorithm

      Spatial Grid Decomposition The input IC image undergoes partitioning into a uniform grid lattice structure. Each individ- ual grid cell within this spatial arrangement assumes responsi- bility for detecting printed text components whose geometric centers fall within that cells boundaries. This decomposition strategy distributes detection responsibility across the spatial domain, enabling parallel prediction processing and localized feature analysis.

      Bounding Box Localization Prediction Within each grid cell, the neural network generates multiple candidate local- ization proposals, each representing a potential IC marking

      region. Each prediction encompasses four geometric param- eters specifying the rectangular boundary dimensions (center x-coordinate, center y-coordinate, width extent, height extent). These parameters are computed as learned transformations relative to the grid cells spatial position, enabling exible representation of objects of varying dimensions.

      Condence Magnitude Computation Each predicted bound- ing box receives an associated condence measurement reect- ing two distinct probability components: (1) the likelihood that the rectangular region contains actual IC text rather than back- ground clutter, and (2) the spatial alignment accuracy between the predicted rectangle and the true underlying marking region. These components combine multiplicatively to yield a single condence scalar. Predictions below an operational threshold (typically 0.5 on a 01 scale) are removed from further processing, substantially reducing computational burden for subsequent stages.

      Class Type Probability Determination Alongside geometric predictions, the network computes conditional probability dis- tributions across the ve IC classication categories (LM358, NE555, 7400, 7805, 7809). These class probabilities are inde- pendent of spatial accuracy; they quantify the networks belief regarding which IC type the identied text region represents, given that text is present. These probabilities sum to 1.0 across all categories when normalized appropriately.

      Redundancy Suppression via Overlap Metrics Multiple grid cells frequently generate predictions for overlapping regions of large IC markings. A suppression algorithm compares prediction pairs by computing spatial overlap quantication through the Intersection-over-Union (IoU) metric. This metric divides the overlapping pixel area by the total area encom- passed by both rectangles combined. Predictions exhibiting IoU greater than 0.4 relative to higher-condence predictions undergo elimination, consolidating redundant detections into single representative predictions. This ltration dramatically reduces false-positive duplicate detections.

      Final Prediction Extraction and Transmission After suppres- sion, the network output consists of ltered, non-overlapping predictions. The detection with maximum condence score is selected as the primary result. Its corresponding IC class identier (integer from 04) and numerical condence measure are extracted and trasmitted to the ESP32 control processor. Should no predictions survive the condence threshold cri- terion, a null-detection indicator transmits instead, signaling processing failure or unrecognizable IC markings.

      Operational Characteristic: This entire sequence executes at real-time speeds, enabling responsive system behavior as components arrive for classication. Processing latency re- mains below 250 milliseconds per image, allowing dynamic handling of IC inputs without bottlenecking the mechanical sorting pipeline.

   2. 6.2 Sorting Control Algorithm

   Categorical Destination Mapping Upon receiving the IC class identier from the vision processor, the ESP32 microcon- troller immediately references a predened lookup relationship

   establishing correspondence between IC category codes and mechanical positioning angles. This mapping establishes the target rotational position of the sorting tray for each recognized IC type:

   TABLE I Destination Mapping

   IC Classication	Category code	Destination Angle	Motor Step Count
   LM358	0	0	0
   NE555	1	72	40
   7400	2	144	80
   7805	3	216	120
   7809	4	288	160

   Rotational Positioning of Component Receptacle Once the destination angle is established, the stepper motor receives actuation commands directing the sorting tray toward angular alignment with the target receptacle. The motor rotates the platform incrementally until the correct bin aligns directly beneath the gripper mechanism. During this rotational phase, the tray remains in motion without interruption, synchronizing with subsequent gripper operations to maximize throughput efciency.

   Robotic Gripper Pick-and-Place Operation Following tray positioning or during simultaneous tray motion, the servo- driven gripper executes a coordinated sequence: the gripper mechanism descends toward the IC component, grasps the device through mechanical nger compression, elevates the component vertically to clear the mounting surface, transports the device over the rotating tray, and releases the component into the target bin through controlled nger relaxation. This sequence operates through precise pulse-width modulation controlling servo motor position at each phase.

   Mechanical System Reset to Neutral Conguration Upon successful component deposition, the gripper arm returns to its home positiona predened neutral conguration centered over the next component waiting for classication. This hom- ing action positions the system for immediate reprocessing of subsequent ICs, minimizing idle time between sorting cycles.

   Temporal Optimization and Latency Reduction The rmware controls both stepper and servo motor actuation timing to minimize cumulative mechanical delays. Stepper pulse frequency, servo transition speeds, and dwell periods are carefully tuned to balance mechanical reliability against throughput objectives. Acceleration proles prevent mechan- ical shock or missed steps, while elimination of unneces- sary intermediate positions reduces total cycle duration. This optimization yields approximately 5.8 second per-component sorting time.

   Class	Accuracy	Samples
   LM358	70%	20
   NE555	75%	20
   7400	65%	20
   7805	65%	20
   ULN2003	75%	20
   Overall	70%	100

   1. Accuracy

   2. Sorting Time

7. RESULTS

   Fig. 3. Result Dashboard

   TABLE II Classification Results

   XII. Conclusion

   In conclusion, the IC sorting Robotic Arm using image detection offers numerous benets and advancements in the eld of automation and object sorting. By integrating image detection technology into the robotic arm system, the project aims to enhance efciency, accuracy, and productivity while ensuring quality control and safety.Throughout the project, various components and technologies have been employed, such as the ESP32-CAM module, stepper motors, image processing algorithms(YOLO), and control systems. These elements work together to enable real-time object detection, sorting, and monitoring.The projects impact analysis reveals signicant improvements in efciency, productivity, and cost savings. The system streamlines the sorting process, reduces manual labor, minimizes errors, and enhances the overall throughput of the operation. Additionally, the integration of image detection allows for the identication and removal of defective items, resulting in improved product quality and customer satisfaction.Moreover, the implementation of professional ethics practices ensures responsible and ethical conduct throughout the project. Privacy and data protection, transparency, fairness, and compliance with legal and ethical standards are all crucial aspects considered during the devel- opment and deployment of the system.Overall, the IC sorting Robotic Arm project utilizing image detection demonstrates the potential for automation, precision, and optimization in industrial processes. It serves as a foundation for future advancements and developments in the eld, contributing to increased efciency, improved quality control, and enhanced

   Avg 50 seconds per IC.

8. Discussion

   Accuracy was limited by small text size and reective IC surfaces. Lighting consistency signicantly affected recogni- tion.

9. Applications

   • Electronics labs

   • Repair centers

   • Component packaging

10. Limitations

    • Small dataset

    • Slow actuation

    • Only ve IC classes

11. Future Scope

safety in various industries.

References

1. J. Redmon and A. Farhadi, YOLOv3: An Incremental Improvement, arXiv:1804.02767, 2018.

2. ICDAR 2021 Competition on IC Text Spotting.

3. Espressif Systems, ESP32 Datasheet.

4. Author(s), Design and Development of a Robotic Arm for Indus- trial Automation, International Journal of Robotics Research, vol. X, no. Y, pp. xxyy, YEAR. [Online]. Available: (insertGoogleScholar/ publisherURLhere)

5. Author(s), IC Sorting and E-Waste Management, in: Journal of Cleaner Production, vol. X, no. Y, pp. xxyy, YEAR. [Online]. Avail- able: (insertElsevier/publisherURLhere)

6. Author(s), Electronic Waste Sorting and Recycling Systems Using Machine Vision, Journal of Cleaner Production, vol. X, no. Y, pp. xxyy, YEAR. [Online]. Available: (insertElsevier/publisherURLhere)

7. Author(s), Sobel Edge Detection Algorithm for Image Processing, IEEE Transactions on Image Processing, vol. X, no. Y, pp. xxyy, YEAR. [Online]. Available: (insertIEEEXploreURLhere)

8. Author(s), Arduino-Based Automated Systems for Industrial Ap- plications, Journal of Emerging Trends in Engineering Research (JETIR), vol. X, no. Y, pp. xxyy, YEAR. Online]. Available: (insertJETIRURLhere)

9. Author(s], Robotic Arm for Segregation Using Image Processing, International Research Journal of Advanced Engineering and Health (IRJAEH), vol. X, no. Y, pp. xxyy, YEAR. [Online]. Available: (insertIRJAEHURLhere)

   • Larger dataset

   • Faster actuation

   • On-device inference