Object Sorting Robot Using Image Processing

Object Sorting Robot Using Image Processing

A Sampreeth

Dept. of Electronics and Communication Engineering, Vidyavardhaka College of Engineering,

Mysore, Karnataka, India.

Aishwarya Rao

Dept. of Electronics and Communication Engineering, Vidyavardhaka College of Engineering,

Mysore, Karnataka, India.

Rohith K

Assistant Professor,

Dept. of Electronics and Communication Engineering, Vidyavardhaka College of Engineering,

Mysore, Karnataka, India

Harsha A K

Dept. of Electronics and Communication Engineering, Vidyavardhaka College of Engineering,

Mysore, Karnataka, India

Karthik V Desai

Dept. of Electronics and Communication Engineering, Vidyavardhaka College of Engineering,

Mysore, Karnataka, India

Praveena K S

Assistant Professor,

Dept. of Electronics and Communication Engineering, Vidyavardhaka College of Engineering,

Mysore, Karnataka, India.

AbstractThe process of sorting objects can be made completely autonomous by using machines which can recognize objects. In this paper we present a system in which a robotic arm sorts objects according to their colour and shape. Objects are categorized into three colors which are red blue and green. The objects are also differentiated based on their shape into two categories, one with edges and other without edges. The image of the object to be sorted is captured using a webcam and image processing is done using MATLAB. The robotic arm is controlled by an ARM 7 based system. Geared DC motors are used for operating the robotic arm.

Keywords — Robotic arm, microcontroller, Webcam, Geared DC motors, image processing.

1. INTRODUCTION

   Automation has become an extremely important aspect in our everyday life. Robotic automation is implemented in various industries. It is very important for industrial environments because it reduces the error capacity of the system and also saves time as compared to human beings. Intelligent machines are very important in today's world. The is a great need for robotic systems which can operate without human intervention.

   Object sorting systems are widely used in various industries like packaging and logistics. There is a great need for all these systems to be completely automated. In this paper we present a system which is completely automated and can accurately sort objects based on their shape and color. The sorting system consists of a robotic arm which is controlled using an ARM 7 microcontroller. A single web cam is used for image input. Image processing is done using MATLAB to detect the color and shape of the object. The objects are categorized into three colors and two basic shapes. The objects on the conveyor belt are picked up by the robotic arm and are placed in their respective predetermined places by the arm. Geared DC motors are used to control the movements of the robotic arm.

2. LITERATURE REVIEW

   1. Development of an autonomous ball picking robot

      This paper is about the design and development of an autonomous ball picking robot. It aimed to improve the capabilities of a preexisting 6-Axis robot. Image processing was done using two cameras and an external computer. One camera finds the location and also determines the color, while, the other camera is used for feedback control of the robotic arm. This paper shows how outside sensors and processing can control a robotic arm to sort objects by color. this papers major drawback was that it proposed a system which could only differentiate based on color.

   2. Real time color-based sorting robotic arm system.

      In this paper, a robotic sorting arm system was presented which could sort objects based on its color. In this robotic system, a camera captures image frames of the object and image processing is done using OpenCV to recognize the color of the object. The position of the object is calculated by its mass center in image. Using Inverse Kinematics algorithms, the control input for the robotic arm is calculated and then sent to an Arduino microcontroller. The microcontroller drives the motors on the robotic arm to sort and position the objects according to their color. The major drawback of this project was that it could only sort objects based on color.

   3. Object sorting robotic arm based on color sensing

   In this paper too, a system was proposed to sort objects based on their color. A robotic arm is used to pick and sort the objects. This is controlled by an ARM 7 based system. A light to frequency converter is used to sense and differentiate objects based on their color. three different colors could be separated, red, green and blue. The major drawback of this paper was that the proposed system didnt differentiate based on the shape of the object.

3. PROPOSED SYSTEM

   A. Object Sorting System

   In this project, we propose a fully automated robotic sorting system which can sort objects based on their shape and color. a robotic arm is used to pick and place the object in their predefined place based on their shape and color parameters. This robotic arm is controlled by an ARM 7 based system. A single camera is used to capture images to be processed using matlab. Matlab is used for processing the captured image frame and to determine the color and shape of the object on the conveyor belt. This system can differentiate between three different color which are red, green and blue. This system can differentiate between objects with edges and without edges which are squares and circular objects. Geared DC motors are used for the controlled movements of the robotic arm. A DC motor is also used to move the conveyor belt so as to bring the object into place to be picked up by the robotic arm. An LCD screen displays where the object has been placed by showing the degree of rotation of the arm.

   Fig. 2. Object on the conveyor belt.

   C. Geared DC Motors

   Three geared DC motors are used in this system. The one at the base of the robotic arm is used to control the angle of rotation of the robotic arm. It rotates at 10rpm. The prefixed angles which it can rotate are 30, 60 and 90 degrees both in clockwise and anticlockwise directions. The second DC motor is used to move the robotic arm itself so as to make it grip the recognized object. This DC motor rotates at 100rpm. The third DC motor is used to control the working of the conveyor belt. The motor drive used to control these DC motors are L293D motor drives.

   Fig. 1. Block diagram of the proposed system.

   B. Matlab and Image Processing

   1. Camera

      Fig. 3. Robotic arm setup

      MATLAB is a multi-paradigm numerical computing environment and fourth-generation programming language. A proprietary programming language developed by MathWorks, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, Java, Fortran and Python.

      The camera used in this case will be overhead camera. It will take the picture of the object for shape sensing purpose. The captured image is sent to the computing system to be processed. MATLABs image acquisition toolbox is used for this process. Once the image is captured the color is first recognized and then the shape. After the identification of the object, commands are sent to the ARM 7 based system on how to sort the object.

      The camera used here is an intex webcam with 640×480 and 30 Megapixel resolution. It has frame rates up to 30 fps. The i/o interface is through USB 2.0. the powe consumption is 160mW (typical). The image sensor is an 1/7 CMOS sensor. This camera is mounted such that it focuses on the conveyor belt. Once an object arrives on the conveyor belt, an image is captured and sent to the computer system to be processed by using MATLAB.

   2. Conveyor Belt

      The conveyor belt consists of two pulleys. One of the pulleys is controlled using a DC motor. The white belt of the conveyor system proves as an ideal background aiding in accurate processing of the image captured by the camera.

      Fig. 4. Conveyor belt system

   3. Microcontroller

      In this project ARM7 based microcontroller LPC2148 is used which is a 32-bit microcontroller with RISC architecture. It has 48 I/O lines.

      Fig. 5. LPC 2148 Microcontroller

      Once the colour of the ball is detected, the ball is picked up from the belt and dropped in the right basket using the robotic arm. The robotic arm movement is controlled by using geared dc motors. Rotation of the servo motors converted to arm movement. That angle of rotation is based on color and shape of the recognized object.

   4. LCD display

      The LCD display used is a 16×2 alphanumeric LCD. This is implemented in the system so as to show where the recognized object will be placed. The degree and direction of the robotic arms movement is displayed on the LCD.

4. ALGORITHM FOR COLOR DETECTION

   • Start

   • Capture the video frames using the video input.

   • Acquire image frame of the object.

   • To identify the color of the object, subtract the color component from gray scale image.

   • Tracking RED objects in real time: Subtract the RED component from the gray scale image to extract the red components in the image.

   • Similarly track GREEN and BLUE objects.

   • Convert the gray scale image into a binary image.

   • Remove all those pixels less than 300 connected pixels.

   • Select all the connected components in the image.

   • Apply bounding box for object recognized and display the color detected.

   • Stop.

5. PROCESS OF SHAPE DETECTION

   After the object is detected, a bounding box is applied around it. The centroid of the object is then determined. Also, the area and perimeter of the object is determined. The variable Ba is used to store the value of the surface area of the object. Bp is used to store the perimeter of the surface captured by the webcam. The metric value is then calculated using the formula, 4*(pi*Ba/Bp)2. If the resulting metric value is less than 0.7, the object is identified as a square. Otherwise it is a circular object.

6. RESULTS

   The object sorting system is successfully implemented. Figure shows a blue cubical object being sorted. Circular objects are placed in the clockwise direction and squares are placed on the anticlockwise direction. Blue, green and red objects are placed at 30, 60 and 90 degrees respectively.

   Fig. 6. Object being sorted.

7. CONCLUSION

   This project describes an Object Sorting Robotic Arm Based on Colour and Shape Sensing. It gives an easy and less expensive way to implement an autonomous object sorting system. By using this system, the human intervention and manual work in production and distribution areas can be reduced. Object sorting robots make for more efficient industrial environments. The are less prone to errors and faster compared to human labour. In this paper we presented an efficient microcontroller based system that pick up objects and put it down at right place based on its shape and color to optimize the object sorting process. The system reduces the need for human intervention.

8. FUTURE SCOPE

The Proposed project can be improved in certain areas. The sorting system can be made to operate faster by using a more powerful microcontroller. More complex image processing can be done to identify objects of multiple colors and multiple shapes.

REFERENCES

1. Karl Nicolaus, Jeremy Hooper, Richard wood and Chan Ham, Development of an autonomous ball picking robot.IEEE, 2016.

2. Yonghui Jai, Guojun Yang and Jafar saniie. Real time color based sorting robotic arm system.IEEE, 2017.

3. Ashraf Elfasakhany, Eduardo Yanez, Karen Baylon, Ricardo Salgado, Design and Development of a Competitive Low-Cost Robot Arm, International Journal of Emerging Technology and Advanced Engineering, Volume 3, Issue 5, May 2013, pp 10-15.

4. Aji joy, Object sorting robotic arm based on color sensing, IJAREEIE 2014.

5. Balkeshwar Singh, Kumaradhas , N. Sellappan, Evolution of Industrial Robots and their Applications, IJEIT, Volume 2, Issue 2, November 2011, pp24-26.

6. Prof. D. B. Rane1, Gunjal Sagar S, Nikam Devendra V, Shaikh Jameer U, Automation of Object Sorting Using an Industrial Roboarm and MATLAB Based Image Processing, International Journal of Emerging Technology and Advanced Engineering, Volume 5, 2004.

7. S. Lee, J. Kim, M. Lee, et al. 3D visual perception system for bin picking in automotive sub-assembly automation, Automation Science and Engineering (CASE), 2012 IEEE InternationalConferenceon, 2012, pp.706-713.

8. T N. Tsagarakis, M. Laffranchi, B. Vanderborght, and D. Caldwell, Acompact soft actuator unit for small scale human friendly robots,IEEE International Conference on Robotics and Automation Conference(ICRA), 2009, pp. 43564362.

9. A. Edsinger-Gonzales and J. Weber,Domo: A force sensing humanoid robot for manipulation research, 4th IEEE/RASInternational Conference on Humanoid Robots, 2004, pp. 273291.

10. Ashraf Elfasakhany, Eduardo Yanez, Karen Baylon, Ricardo Salgado,Design and Development of a Competitive Low-Cost Robot Arm withFour Degrees of Freedom, Modern Mechanical Engineering, 1, 2011, pp: 47-55.

11. GoldyKatal, Saahil Gupta, ShitijKakkar,Design And Operation Of Synchronized Robotic Arm, International Journal of Research inEngineering and Technology, Volume: 02 Issue: 08, Aug-2013.

12. ShwetaPatil, Sanjay Lakshminarayan,Position Control of Pick and Place Robotic Arm,EIEs 2nd Intl Conf.Comp., Energy, Net., Robotics and Telecom. EIE Con2012.