Real-time Robust Lane Detection and Warning System using Hough Transform Method

Real-time Robust Lane Detection and Warning System using Hough Transform Method

Prajakta R. Yelwande1 1MIT World Peace University, Pune, India

Prof. Aditi Jahagirdar2 2MIT College of Engineering, Pune, India

Abstract- Many people die each year in roadway departure crashes caused by driver inattention. Lane detection systems are useful in avoiding these accidents as safety is the main purpose of these systems. Such systems have the target to detect the lanes and to warn the driver in case the vehicle has a tendency to depart from the lane. A lane detection system is an important aspect of many intelligent transport systems. Lane detection is a demanding task because of the varying road conditions that one can come across while driving. In the past few years, plentiful approaches for lane detection were proposed and successfully demonstrated. In this paper, after a brief overview of existing methods, we present a robust lane detection based on Canny edge detection and Hough

transform method.

Keywords Canny Algorithm, Edge Detection, Feature Extraction, Hough Transform, Lane Detection, Region of Interest(ROI).

1. INTRODUCTION

   Now a day the road accidents have increased to a great extent. Most of the accidents occur due to drivers negligence and carelessness while driving. Advance driver assistance system (ADAS) plays an important role in providing safety to drivers. It helps to automate the car system and increases the driving experiences. The Advance driver assistance system (ADAS) provides a safe system to reduce the road accidents. The system takes an vigorous step like warning the driver or takes a corrective action to avoid an accident during the risky situation.

   The Lane Departure Warning (LDW) is an important unit in Advance driver assistance system. In vision based lane departure system, a camera is placed behind the wind shield of the vehicles and images of the road is captured. The white stripes on the road are interpreted and lanes are identified. Whenever the vehicle goes out of the lane then the warning is given to

   the driver. In lane departure warning system, the lane detection is the primary step to be taken.

   There are two types of approaches used in lane detection: the feature based approach and the model based approach. The features based approach detects the lane in the road images by detecting the low level features such as lane edges or highlighted lanes etc. This approach requires well highlighted lines or strong lane edges, otherwise it will fail. This approach may suffer from occlusion or noise. The geometric parameters such as assuming the shape of lane can be presented by straight line or curves are used by the model based approached.

2. LITERATURE OVERVIEW

   Most of the time accidents are caused by lack of concentration and not maintaining a safe car distance to the car in front, or changing lanes without paying attention for vehicles which is next to the car. This project is about detecting the boundaries of the lane and to tell the driver if he/she is going to change the lane without signifying for his/her intention. The system should also try to measure the distance to the vehicle in front of that vehicle and signalize if the distance in not safe enough.

   Lane detection in driving scenes is an significant component for autonomous vehicles and advanced driver assistance systems. In recent years, many complicated lane detection methods have been proposed. However, most methods focus on detecting the lane from one single image, and often lead to unacceptable performance in handling some extremely-bad situations such as heavy shadow, severe mark degradation, serious vehicle occlusion, and so on. In fact, lanes are incessant line structures on the road.

   Vehicle safety plays an important role for safety of all road users and also useful to measure the crash avoidance or reduction of injury. The purposes of Advanced driver assistance systems are to reduce the risk and assist post impact care are also investigated for future application.

   Table 1: Literature Survey

   Sr. No	Paper Reference No.	Year	Methods Used	Advantages	Accuracy
   1.	[1]	2018, IET Jour.	CNN, pre-processing, feature detection, fitting tracking, kalman filter, particle filter.	It gives high accuracy. Better results for detecting curved lanes.	98%
   2.	[2]	2018, IEEE	Feature extraction, model fitting, Random Sample Consensus (RANSAC) technique	Better computation efficiency, High accuracy	–
   3.	[3]	2018, IEEE	Principle Component Analysis Technique	Real-time performance within a low computation hardware platform	–

   	4.	[4]	2018, IEEE	Median strip detection approach, Lane change detection approach	Smart use of spatio-temporal information provided by the embedded sensors technology	–
   	5.	[5]	2018, IJPAM Journal	Review paper	–	–
   	6.	[6]	2018, EURASIP Journal	Hough transform and Kirsch operator, feature extraction	the robustness and adaptability of the detection results are enhanced, the computational complexity of the algorithm is reduced by the matrix operation.	–
   	7.	[7]	2018, Hindawi Journal	Kalman filter, Hough transform, Feature extraction, colour extraction	Better accuracy and faster processing speed	95%
   	8.	[8]	2017	LDWS Algorithm, Canny's Algorithm, Hough Transform Technique	High accuracy and robustness against noise and model imperfection	–
   	9.	[9]	2017, IEEE	Canny edge detection algorithm, Hough transform Method	Faster processing speed	–
   	10.	[10]	2017, IEEE	Gabor filter, Hough transform method, Sobel operator, least squares algorithm	System is real-time, efficient and enhance the adaptability for the changing environment of road scene.	93%
   	11.	[11]	2017, IEEE	Spatio-Temporal incremental clustering algorithm, PCA technique	Accurately detects straight as well as curved lanes, Algorithm does not require database for storing images	95%
   	12.	[12]	2017, IEEE	FPGA system	System is useful to monitor the vehicle to track online the vision detection lane mark and execute obstacle avoidance.	–
   	13.	[13]	2017, IEEE	Hough transform, morphological operations	Detecting straight as well as curved roads of hilly areas using vision based techniques.	81.67%
   	14.	[14]	2017, IEEE	Histograms of oriented gradients, SVM Classifier, kalman filter	Accurately detects straight as well as curved lanes	96.3%
   	15.	[15]	2017, IEEE	Mono- vision based lane detection technique, Sobel filter	Addressed the problem of the generation of an optimal constrained lane reference to be tracked by the automated guided vehicle.	–
   	16.	[16]	2017, IEEE	Hough- transform, RANSAC Bezier splines fitting, Gaussian filter	Able to find vehicles in front of our vehicle like cars, buses but unable to find two wheelers.	85%
   	17.	[17]	2017, IEEE	Kalman filter, SVM Classifier	High Accuracy	98.1%
   	18.	[18]	2017, IEEE	Canny algorithm, Sobel operator, Hough transform	Can detects linear lanes based on Hough transform	–
   	19.	[19]	2017, IEEE	Feature extraction	Detects lanes in different environment conditions	–
   	20.	[20]	2017, ICROS	Kalman filter	Accurately detects straight lanes	–
   	21.	[21]	2017	Randomized Hough Transform	Good accuracy for straight roads	–
   	22.	[22]	2017	Sobel filter, Hough Transform for Lane Detection	Hough transform was still able to track the loss of lane marks by assuming the lane was still there by counting the number of the lost frame. If the lost track is more than the defined number of frames, then it stopped the Tracking operation.	–
   	23.	[23]	2017	fuzzy c-means for Segmentation, modifying the Hough transform i.e. hybridization of additive Hough transform with artificial bee colony edge detection to detect curve lanes	In this modify Hough transform i.e. additive Hough transform with artificial bee colony based edge detector is used to get better straight lane as well as curved lane road images	–
   	24.	[24]	2017	Ellipsoidal Neural Networks with Dendrite Processing (ENNDPs)	We have shown how the proposed methodology can be successfully applied to Automatically detect lanes in urban highways.	–
   	25.	[25]	2017	Hough Transform	This system will work in both day and night situation	–
   	26.	[26]	2017	Lane coloration Algorithm (modifying the Hough transform i.e. fuzzy logic)	Fuzzy Logic is used to improve straight lane as well as curved lane road images	–
   	27.	[27]	2016, IEEE	Phase angle varying range (PAVR) to achieve a better position judging	Analyzes the edge position detection method of segmental wireless power supply for electrical vehicles without position sensors	–
   	28.	[28]	2016, IEEE	CNNs, Hough transform, Canny operator	System can achieve higher recall and accuracy in real scenes videos	90.7%
   	29.	[29]	2016, IEEE	Speed-adaptive ratio based algorithm	Can predict the speed-adaptive lateral ratio between left and right lanes	–
   	30	[30]	2016, IEEE	SVM model	Can detect the normal and abnormal lane changes instances	90%
   	31.	[31]	2016	RFID, V-I Positioning algorithm	–	–

   	32.	[32]	2016, IJRITCC Journal	Hough transform, Vanishing point based boundary detection	Good results, both straight and slightly curved road are detected.	–
   	33.	[33]	2015	Gaussian mixture model, RANSAC method	Detects lanes even in sunny and shadow road	95.7%
   	34.	[34]	2015, IRJET Journal	Hough transform, Bilateral filter, Canny edge detector	Optimal edge detection	–
   	35.	[35]	2014, IJCSMC Journal	Review Paper	–	–
   	36.	[36]	2013, IEEE	RANSAC Model, Kalman filter	Faster processing speed, good performance of the system.	93.2%
   	37.	[37]	2006, IEEE	Review Paper	–	–
   	38.	[38]	–	Hough transform, Gaussian filter	Faster processing speed (123 m/sec)	–
   	39.	[39]	2018, IEEE(AJCT Journal)	Review Paper	–	–
   	40.	[40]	2019	Hough transform, sobel operator	Better accuracy	96%

3. SYSTEM ARCHITECTURE

   Fig. 2. Proposed Architecture of lane detection system

   System is based on following steps:

   1. Video: Live video is captured using camera fixed in vehicle.

   2. Frame Conversion: Frames can be obtained from a video and converted into images.

   3. Edge detection: sudden changes of discontinuities in an image are called as edges. Significant transitions in an image are called as edges. Most of the shape information of an image is enclosed in edges. So first we detect these edges in an image and by using these filters and then by enhancing those areas of image which contains edges, sharpness of the image will increase and image will become clearer.

   4. Hough Transform: The Hough Transform (HT) is a robust method for finding lines in images that was developed by Paul Hough.

   5. Lane Detection: Hough Transform is a popular technique to detect any shape, if you can represent that shape in mathematical form. It can detect the shape even if it is broken or distorted a little bit.

   6. Car Detection: Object Recognition is a computer technology that deals with image processing and computer vision, it detects and identifies objects of various types such as humans, animals, fruits & vegetables, vehicles, buildings etc..Every object in existence has its own unique characteristics which make them unique and different from other objects. RNN (Recurrent Neural Network) is used to detect object (here car).

   7. Analysis: Hough transform detects lane, change in lane whereas RNN detects vehicle and system analyze and alert if lane changes.

   8. Alarm: An alarm will alert system when it changes the lane.

4. IMPLEMENTATION

   1. Dataset Collection:

      The dataset is of Video Format which is converted into frames for processing which contains videos frames of different videos with different conditions

      i.e straight road, curved road, night scene etc.

      Fig.3. Snapshot of Dataset

   2. EXPERIMENTAL RESULTS:

   After applying different algorithms, we have obtained outputs for given system. Canny edge detection and Hough-Transform algorithms have been applied over the dataset.

   Step 1: Load image or video

   Step 2 : Frame Conversion

   The dataset is of Video Format which is converted into frames for processing.

   Fig.4. Snapshot of Frames

   Step 3: Edge Detection of Image

   Algorithm: Canny Edge Detection

   Canny edge detection is a method to take out useful structural information from different vision objects and significantly decrease the amount of data to be processed.

   Output:

   Fig.5: Snapshot of output of edge detection

   Step 4: Region of Interest Segmentation

   After edge detection by canny edge detection algorithm, we can see that the obtained edge not only includes the required lane line edges, but also includes other unnecessary lanes and the edges of the surrounding fences. This method can increase the speed and accuracy of the system.

   Step 5: Lane Detection

   Algorithm: Hough-Transform

   Fig.6: Snapshot of output of lane detection using Hough-transform method

   Step 6: Car Detection (Object Detection) Algorithm: Artificial Neural Network

   Fig.7. Snapshot of Car Detection

   Step 7: Alarm or Warning

   Fig.8. Snapshot of condition when system gives warning after detection of object(car)

5. PERFORMANCE METRICS, RESULTS & ANALYSIS

   1. Performance Metrics

      As ground truth is not available so we can evaluate the performance metrics of lane detection algorithms by comparing input frames and output frames by calculating true positive(TP), or true negative(TN) or false positive(FP) or false negative(FN).

      • TP is when lane region exists in input frame and it is detected successfully by the model proposed.

      • FP is when method detects the lane roads even when there is no lane in input frame.

      • FN is when there exists a lane region in input frame but method fails to detect.

      • TN is when there is no lane region in input frame and algorithm fails to find it.

      The metrics used to evaluate performance are the standard methods such as precision, recall, accuracy, F score etc.

      Fig 9: Equations to evaluate Performance metrics

   2. Results:

      Following snapshots shows the results of videos named as VIDEO:1(contains only straight road), VIDEO:2(contains curved road), VIDEO:3(contains mixture of straight plus curved road), VIDEO:4(contains curved road) etc.

      Fig.10. Snapshot of result of VIDEO:1

      Fig.11. Snapshot of result of VIDEO:2

      Fig.12. Snapshot of result of VIDEO:3

      Fig.13. Snapshot of result of VIDEO:4

      Following table shows that the results of Hough transform operator which is performed on video dataset which contains video frames of straight and curved roads.

      Table 2: Performance metrics of Video dataset trained using Hough transform

   3. Analysis:

   According to following results straight road gives the better accuracy than curved roads. The accuracy for

   straight road is 96.23% and accuracy for curved road is 90%.

   1. Accuracy:

   2. Recall:

      In information retrieval, recall is the fraction of the relevant documents that are successfully retrieved. It is also called as true positive rate(TPR).

   3. Specificity:

   Specificity measures the proportion of actual negatives that are correctly identified. It is also called the true negative rate(TNR).

   5) False Positive Rate:

   The false positive rate is calculated as the ratio between the number of negative events wrongly categorized as positive (false positives) and the total number of actual negative events.

   Advantages:

   1. Reduced risk when multiple distractions are present such as when loud children are in the car.

   2. Safer highway driving.

   3. Accident prevention late at night, when fatigue may lead to lane departure.

   4. Improved protection for teen drivers, who have a tendency to drift in the lane.

   5. More warning of accidents when driving in adverse weather conditions.

   6. Compensates for human error when driving.

   Applications:

   1. Reduce unnecessary information in an image while preserving the structure of image.

   2. Extract important features of image like curves, corners, and lines.

   3. Recognizes objects, boundaries and segmentation.

   4. Plays a major role in computer vision and recognition.

   4) Precision:

   In the field of information retrieval, precision is the fraction of retrieved documents that are relevant to the query. It is also called as positive predictive value(PPV).

6. CONCLUSION

Lane departure warning is an inevitable module in the advanced driver assistance systems. In the last decade several advancements occurred in the lane detection and tracking field. Vision based approach is a very simple modality for detecting lanes. Even though lot of progress has been attained in the lane detection and tracking area, there is still scope for enhancement due to the wide range of variability in the lane environments.

REFERENCES

1. Yang Ye, Xiao Li Hao, Hou Jin Chen, "Lane Detection method based on lane structural analysis and CNNs", IET Transp. Syst., vol. 12, Iss. 6, pp. 513-520, 2018.

2. Yang Xing, Chen Lv, Long Chen, Huaji Wang,Hong Wang, Dongpu Cao, Efstathios Velenis, Fei-Yue Wang "Advances in vision-based Lane Detection: Algorithms, Integration, Assessment, and Perspectives on ACP-Based Parallel Vision", IEEE/CAA JOURNAL OF AUTOMATICA SINICA, VOL. 5, NO. 3, MAY 2018.

3. Cong Hoang Quach, Van Lien Tran, Duy Hung Nguyen, Viet Thang Nguyen, Minh Trien Pham, Manh Duong Phung "Real- time Lane Marker Detection Using Template Matching with RGB-D Camera", 2018 2nd International Conference on Recent Advances in Signal Processing, Telecommunications Computing (SigTelCom).

4. Lioris,Annie Bracquemond, Gildas Thiolon, Laurent Bonic, "Lane Change detection algorithm on real world driving for arbitrary road infrasructures", 2nd IEEE International Conference on Computer Software Applications,2018.

5. K.Mirunalini, Dr.(Mrs).Vasantha Kalyani David, "Techniques for Detection of Lanes on RoadsA Review", International Journal of Pure and Applied Mathematics Volume 118 No. 18 2018, 2835-284.

6. Zhong-xun Wang* and Wenqi Wang, "The research on edge detection algorithm of lane", EURASIP Journal on Image and Video Processing.

7. Mingfa Li , Yuanyuan Li , and Min Jiang, "Lane Detection Based on Connection of Various Feature Extraction Methods", Hindawi, Advances in Multimedia, Volume 2018.

8. Gianni Cario, Alessandro Casavola, Marco Lupia, "Lane Detection and Tracking Problems in Lane Departure Warning System", 2017

9. Li Dang, Girma Tewolde, Xiaoyuan Zhang, Jaerock Kwon, "Reduced Resolution Lane Detection", IEEE Africon Proceedings,2017.

10. Qi Gao, Yan Feng, Li Wang, "A Real time Lane Detection and Tracking Algorithm", IEEE, 2017.

11. Any Gupta, Ayesha Choudhary, "A Real-time Lane Detection Using Spatio-Temporal Incremental Clustering", IEEE 20th International Conference on Intelligent Transportation Systems (ITSC),2017.

12. Shiuh-Jer Huang, Chien-Chang Tsai, "Vehicle Lane Detection and Following Based on vision system and Laser Scanner", Proceedings of the IEEE International Conference on Applied System Innovatin, IEEEICASI 2017 – Meen, Prior Lam (Eds), 2017.

13. Kodeeswari M,Philemon Daniel, "Lane Line Detection in Real Time Based on Morphological Operations for Driver Assistance System", 4th IEEE International Conference on Signal Processing, Computing and Control (ISPCC 2k17), Sep21-23,2017,solan,India 2018.

14. Van Quang Nguyen, Changjun Seo, Heungseob Kim, Kwangsuck Boo, "A study on Detection method of vehicle based on lane detection for a Driver Assistance System using a Camera on Highway", 2017,11th Asian Control Conference (ASCC) Gold Coast Convention Centre, Australia,December 17- 20, 2017.

15. Kawther Osman, Jawhar Ghommam, Maarouf Saad, "Vision Based Lane Reference Detection and Tracking Control of an Automated Guided Vehicle", 2017, 25th Mediterranean Conference on Control and Automation (MED) Valletta, Malta,July 3-6, 2017.

16. Vinuchandran A V, Shanmughasundaram R., "A Real-Time Lane Departure Warning and Vehicle Detection System using Monoscopic Camera", 2017 International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT).

17. Hanwool Woo, Yonghoon Ji, Hitoshi Kono, Yusuke Tamura, Yasuhide Kuroda, Takashi Sugano, Yasunori Yamamoto, Atsushi Yamashita, Hajime Asama, "Lane Detection Based on Vehicle- Trajectory Prediction", IEEE ROBOTICS AND AUTOMATION LETTERS. PREPRINT VERSION, JANUARY, 2017.

18. Xuqin Yan, Yanqiang Li, "A Method of Lane Edge Detection based on Canny Algorithm", IEEE,2017.

19. Joran Zeisler, Fabian Schonert, Marcel Johne, Vladimir Haltakov, "Vision Based Lane Change Detection using True Flow Features", 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC): Workshop.

20. Moon-Hyung Song, Chang-il Kim, Jong-Min Kim, Kwang-Soo Lee, Hyun-Bae Park, Jae-Seok Jeon, Su-Jin Kwag, Moon-Sik Kim, "A Novel Approach to the Enhancement of Lane Estimator by using Kalman Filter", 2017 17th International Conference on Control, Automation and Systems (ICCAS 2017), Ramada Plaza, Jeju, Korea,Oct. 18-21, 2017.

21. Suvarna Shirke, Dr. C. Rajabhushanam, "A Study on Lane Detection Techniques and Lane Departure System".

22. Er RajniMultani, Chetan Marwaha "Efficient lane detection using the hybridization of artificial bee colony & Modified Hough transform", International Journal of Engineering Research and General Science Volume 5, Issue 5, September- October, Journal of Engineering Research and General Science Volume 5, Issue 5, September-October, 2017 ISSN 2091-2730.

23. Fernando Arce a , Erik Zamora b , Gerardo Hernández a , Humberto Sossa a,* "Efficient Lane Detection Based On Artificial Neural Networks", SPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume IV-4/W3, 2017 2nd International Conference on Smart Data and Smart Cities, 46 October 2017, Puebla, Mexico.

24. Miss. GargiPatel1, Miss. MayuriVasava2, Miss. SwetaGoria3, Mr.AkashParikh4, Mr.Nimit Modi "Road Accidental Alert System Based On Lane Change Detection And Eye Blink Detection" International Journal of Advance Engineering and Research Development Scientific Journal of Impact Factor (SJIF): 4.72 Special Issue SILICON-2017, April -2017 e-ISSN: 2348-4470 p-ISSN: 2348-6406.

25. NurShazwaniAminuddin, 2 Masrullizam Mat Ibrahim, 3 NursabillilahMohd Ali, 4 Syafeeza Ahmad Radzi, 5 WiraHidayatMohdSaad& 6 Abdul Majid Darsono "a new approach to highway lane detection by using Hough transform technique", Journal of ICT, 16, No. 2 (Dec) 2017, pp: 244260.

26. Tamanna1, Arushi Bhardwaj2 "Improved Performance Of Fuzzy Logic Algorithm For Lane Detection Images", International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 p-ISSN: 2395-00, Volume: 04 Issue: 07 | July -2017.

27. Qijun Deng, Jiangtao Liu, Dariusz Czarkowski, Mariusz Bojarski, Jing Chen, Wenshan Hu, Hong Zhou, "Edge Position Detection of on-line charged vehicles with Segmental Wireless Power Supply", IEEE 2016.

28. Huang Guan, Wang Xingang, Wu Wenqi, Zhou Han, Wu Yuanyuan, "Real-Time Lane-Vehicle Detection and Tracking System", IEEE,2016.

29. Seongrae Kim, Junhee Lee, Youngmin Kim, "Speed-Adaptive Ratio- Based Lane Detection Algorithm for Self Driving Vehicles", IEEE,2016.

30. Saina Ramyar, Abdollah Homaifar , Ali Karimoddini, Edward Tunstel, "Identification of Anomalies in Lane Change Behavior Using One-Class SVM", 2016 IEEE International Conference on Systems, Man, and Cybernetics SMC 2016 Budapest, Hungary,October 9-12,2016.

31. C. Zheng, C. Libo, Y. Linbo, Q. Qin, Z. Ruifeng, "Lane-Level Positioning System Based on RFID and Vision", 2016

32. Pallavi V. Ingale, Prof. K. S. Bhagat, "Comparative Study of Lane Detection Techniques", International Journal on Recent and Innovation Trends in Computing and Communication ISSN: 2321-8169 Volume: 4 Issue: 5 381 390

33. Mrinal Haloi, Dinesh Babu Jayagopi, "A Robust Lane Detection and Departure Warning System", 2015.

34. GurjyotKaur and Gagandeeop Singh, "A REVIEW OF LANE DETECTION TECHNIQUES", International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395- 0056 Volume: 02 Issue: 03 | June-2015.

35. Anjali Goel, "Lane Detection Techniques – A Review ", International Journal of Computer Science and Mobile Computing, Vol.3 Issue.2, February- 2014, pg. 596-602.

36. Sayanan Sivaraman and Mohan Manubhai Trivedi, "Integrated Lane and Vehicle Detection, Localization, and Tracking: A Synergistic Approach", IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 14, NO. 2, JUNE 2013.

37. Zehang Sun, George Bebis, and Ronald Miller, "On-Road Vehicle Detection: A Review", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 28, NO. 5, MAY 2006.

38. Jung-Hwan Kim, Sun-Kyu Kim, Sang-Hyuk Lee, Tae-Min Lee, Joonhong Lim, "Lane Recognition Algorithm using Lane Shape and Color Features for Vehicle Black Box".

39. Prajakta Yelwande, Aditi Jahagirdar, "Vehicle Lane Detection System for Car Safety", IEEE 5th I2CT, Asian Journal Convergence Technology, Pune, India, 2018.

40. Prajakta Yelwande, Aditi Jahagirdar, "Straight and Curve Lane Detection System for Car Safety using Hough Transform & Sobel Operator", 6th International Conference on Computing in Engineering and Technology(ICCET) (World Academy of Research in Science and Engineering), Mysore 2019.