Raspberry-Pi Based Real Time Lane Departure Warning System using Image Processing

Raspberry-Pi Based Real Time Lane Departure Warning System using Image Processing

Pravin T. Mandlik Prof A. B. Deshmukh

Dept. of E&TC, (VLSI & ES) Assistant professor, E&TC Dept.

Skncoe, Vadgoan Skncoe, Vadgoan

Pune, India Pune, India.

Abstract- Lane detection is an essential component of Advance Driver Assistance system (ADAS). Many different approaches have been proposed till today by researchers but still it is a challenging task to correctly detect the road lanes in various environmental conditions. The main purpose of the system is to detect the lane departure to avoid road accidents and to provide safety for pedestrians. The proposed method detect the road edges using the canny edges detector whereas the feature extraction technique like Hough transform is used in image analysis and digital signal processing. The main input to the system is camera captured images in order to detect and track the road boundaries. This concept of image processing is implemented using Open CV library function on raspberry-pi hardware. This method can correctly detect the roads in various challenging situations. Results shows that the proposed method can detect both the straight and curves lanes correctly.

Index Terms- Advance Driver Assistance System (ADAS), Hough Transform (HT), Lane detection, Lane departure warning system, Raspberry-pi, Open CV.

1. INTRODUCTION

   Over the last few years, there is rapid increase in the embedded electronic subsystems in vehicles, intelligent driver assistance system forms a significant component. The different electronic subsystems such as Vision based advance driver assistance system form the significant contribution toward the intelligent drivers assistance system. The application of the intelligent driver assistance system works towards to form a driver less vehicle. These new up growing technology is trying to create a driver assistance system that go beyond the automatic control system.

   Recently, many studies are conducted based on advanced driver assistance system (ADAS) to avoid car accidents. Mostly, the lane departure warning system (LDWS), this system warns the driver when the vehicles tends to depart of its lane, which is the most basic and important part of the ADAS. Another is the lane keeping system (LKS), this system automatically takes action to keep the vehicles in lane when no action is taken even if the system warn the drivers of the departure. This LDWS, is basically based on lane identification and tracking algorithm using camera, low power is required and it share resources such as central processing unit (CPU), Read Only Memory (ROM) and Random Access Memory (RAM) with the other assistance system for drivers. Also, the system should be of low price, so as to attract the customer. Therefore the LDWS should have lane detection algorithm as compact as much possible, with efficient tracking and less computational time.

   Here the camera is being used to monitor the surrounding of the ego vehicles for both safety of the system and the data analysis of driving. The important task is to understand the traffic scenario and to make real time choice based on the visual information from camera for self-sense driving vehicles. The correct lane stripe identification according to the environmental condition of roads is very important for autonomous cars, also keeping lane track and keep feasible distance with reference to front vehicles. The most important areas is to warn the driver in case of lane change for proper overtaking. The autonomous car is the future but advance driver assistance system (ADAS) is growing in modern cars. Advance driver assistance system (ADAS) plays an important role in providing safety to drivers. It helps to automate the car system and increases the driving experiences. The Advance driver assistance system provides a safe system to reduce the road accidents. The system takes an active step like warning the driver or takes a corrective action to avoid an accident during the dangerous situation.

   In lane departure warning system, the lane detection is the initial step to be taken. There are two types of methodologies used in lane identification: the elements based methodology and the model based methodology. The elements based methodology detects the lane from the images of roads by detecting the low level elements such as lane edges etc. This methodology requires all round painted lanes or solid lane edges, otherwise it cannot detect it. This methodology may experience the ill effects of impediment or clamor. The model based methodology use geometric parameters such as assuming the shape of lane can be presented by straight line or curve. This approach is strong against clamor and missing information [1] [3].

   Rest of the paper is organized as follows, Section II describes the previous work done by researchers. In Section III describes the architecture of the system and Software design of the system. Section IV our experimental setup and results are illustrated. Finally, we conclude in Section V.

2. RELATED WORK

   The literature related to advance driver assistance is mostly in view of utilizing one or more rear camera to detect the lane stripes. There are various sensors like LIDAR, RADAR and so on are utilized for recognizing the things and 3D display of the roads. Likewise here and there camera confronting the driver is utilized to detect the behavior (drowsiness, sleepiness etc.) of the driver while driving.

   To expand the lane identification precision of the method various pre-processing filter are used by the researchers. The methods like hybrid median filter [7], kalman filter [5], Gaussian filter [8], segmentation [10] and gabor filter [14]. In this paper, the canny edges detector is used to detect the edges from the blur images. Blurring the images makes the edges more prominent to detect. So as to lessen the computational time of images the Region of Interest (ROI) is set.

   In reference [1] the B-snake method is proposed for lane detection and accomplished effective following of lanes. It is likewise relevant to the dash and the solid paint line roads also to the marked and the unmarked roads. In reference [3] author used piecewise linear filter for lane detection but it gives little false alarms. The algorithm used is strong in the presence of clamor, shadow, absence of lane painting and various environmental conditions. In reference [4] author used local gradient features for lane detection but this approach gives more false alarms and poor performance in shadow. In reference [5] author used edges feature extraction and grouping along with the Hough transform but it fails to detect lane in heavy traffic and confusing road textures. In reference [10] author used segmentation method to detect the lane but gives high false positive rate. The straight and curves roads lanes are detected using the Hough transform and under various illuminating conditions Hough Transform shows good performance. [15]. Therefore, the Hough transform is selected for identification of lane stripes on roads.

3. SYSTEM ARCHITECTURE

   In this research work, the different cases is analyzed such as captured road images, lanes to be detected can be straight or curvy roads, day or night time and in all-weather condition whether good or bad. Other hand, detecting the lane marking whether solid or dash lines.

   Fig. 1. Block Diagram of system

   1. System Specification

      Raspberry Pi 2 single board computer is used as central module of complete embedded image capturing and processing system. The main signal processing chip unit used in Rapberry Pi is Broadcom 2836 900MHz Chip in which CPU is a 32 bit ARM cortex A7 processor. Camera used in the project is Intex IT-305WC webcam. Raspbian-wheezy operating system image is installed on Raspberry Pi which is a version of Linux. Operating system is the set of basic utilities and programs that makes Raspberry Pi run properly

      as a standalone computer. The operating system on Raspberry Pi board ids installed using a class 4 8GB Samsung Micro SD Card with help of Micro SD Adapter. OpenCV is then installed on Raspbian-wheezy using Micro SD card.

   2. Flow diagram of the lane detection:

   Step 1: RGB to Grayscale conversion of image:

   The color images are composed of three independent primary colors namely red, green and blue components. Here conversion takes place by removing the shades and immersion data while holding the luminance. The RGB values are changed over to Grayscale values by forming the weighted sum of three intensity (R, G and B components) values:

   Y = 0.299 * R + 0.587 * G + 0.114 * B (1)

   Step 2: Filtering Technique:

   Next is to remove the noise present in the images. The captured videos is passed through Gaussian blur filter. It is a type of image blurring filter that uses Gaussian function for transformation of each pixels of the images to a new pixels.

   Step 3: Region of Interest (ROI) selection:

   In determining the computational complexity of lane identification and LDI system, ROI plays an important role to detect it. Here only the selected areas is detected or taken for the next level of processing. These selected ROI images are then used for lane detection using a proposed algorithm. The selection of ROI reduces the processing time of the frames.

   Read in the road images

   Read in the road images

   RGB to grayscale conversion

   RGB to grayscale conversion

   Apply Blur Filter

   Apply Blur Filter

   Canny edges detection

   Canny edges detection

   Subtract road background from road images

   Subtract road background from road images

   Use Hough transform to convert pixels to line parameters space

   Use Hough transform to convert pixels to line parameters space

   Look in Hough space to recognize straight line

   Look in Hough space to recognize straight line

   Lane marking after grouping the cluster lines

   Lane marking after grouping the cluster lines

   Lane departure warning if departure occurs

   Lane departure warning if departure occurs

   Fig. 2. The lane detection flow diagram

   Step 4: Edges Detection:

   Edges recognition alludes to the procedure of identifying and finding the sharp discontinuities in an images. There are different edges identification algorithms such as Sobel, Robert, Prewitt, Laplacian and Canny. In this proposed method, canny edges detection algorithm is used.

   The edges are the important features to be detected in this work. The element of interest is to segregate between the lane stripes and the non-lane edges. Most of the elements of the lane stripes are preserved as edges, which is caused by the canny edges detection function. This edges detection techniques extract the useful information from the vision objects and reduces the amount of data to be processed. If this techniques is used directly on the original images then most of the lane stripes edges information is lost. To reduce the loss of information, the background subtraction is done. The main focus is on the road lanes so the region of interest is set to subtract background. The resultant image after background subtraction is used for edges detection. From this images we can find that lane marking information is preserved.

   Step 6: lane identification and departure warning:

   There many straight lines detected by Hough transform, so we detect the long straight line in the Hough space. Then the lane marking is done and the departure of the lane is detected if the vehicle tends to depart the lane.

4. EXPERIMENTATION AND RESULTS

   1. Experimental setup

      Implementation of the whole system is done on Raspberry Pi 2B board. A USB camera is used to capture the images. System memory is used to store the images and Hough Transform algorithm is applied to detect the road lanes from the images captured by the camera. The Open CV library functions are used for image processing. Open CV serves as an interface on which Hough Transform algorithm is allowed to run along with other image processing functions.

      Step 5: Hough Transform:

      The features of various shape inside an images can be separated using a techniques called as Hough Transform. This technique is generally used for the identification of arbitoary shapes such as straight lines, circles, ellipses, etc. The Hough Transform is implemented on an images after the canny edges detection has taken place so as to obtain the image pixels that are desired ones. So here in this system to detect the lanes marking from the image data, Hough Transform is used. The primary point of interest of the Hough Transform technique allows gaps in feature boundary descriptions and is not affected by noise.

      In general the straight line equation is given by y = mx + c can be represented as point (c, m) in the parameter space or Cartesian co-ordinate system. For the Hough transform we

   2. Results

   Fig. 4. Experimental Setup

   convert the equation to polar co-ordinates i.e. in terms of rho and theta.

   y

   Fig. 5 shows the step by step evolution of the algorithm. These results are taken on Intel Core i3 is central processing unit with a 1.80 GHz processor.A few video sequences are used for the information acquition.

   r

   r = x cos + y sin

   x

   Fig. 3. Hough transform for detecting straight lines

   Where r is the distance from the initial position to closest point on straight line and theta () is the angle between the line connecting the origin and the x axis. The (r, ) plane is referred to as Hough space. The Hough transform detect the straight line in two dimensional array (matrix). Each element of the matrix has values equal to the sum of the pixels that are positioned on line. So the element with the highest values represent the straight lines.

   (a) Original image (b) Gray image (c) Filter image

   1. canny edges detector (e) ROI (f) Lane Detection Fig. 5. Step by step evolution of algorithm

      Fig. 6 shows the correct lane departure warning is issued to the drivers in case of the consistent and discontinouous lane stripes. The proposed algorithm shows that the lanes are detected correctly for the straight and the curved roads.

      1. Right departure (b) No departure

   (c ) Left Departure

   Fig. 6. various departures states results

   The image size is 320 * 240; the images are captured at 30 frames per second. Our algorithm works well in various environmental conditions. Figure 7 shows the experimental results of the system.

   a) No Departure b) Left Departure

   Table I: Comparison of proposed method with other methods

   Methods	False Alaram(%)	Detection rate(%)	Avg. Computational time (ms)
   OTSU Segemenation method	11	91.25	–
   Edge linking method	0	92.45	40
   Fan scanning detection method	10.4	93.48	50
   Proposed method	7.14	93.30	40

   The table I shows the comparison of the proposed method with other algorithm. Various parameters such as false alarm, detection rate and time are used to compare the algorithm. The proposed distance measurement for the lane departure shows less false alarm. In other method, the lane departure is measured using an angular measurement whih result in more false alarm.

5. CONCLUSION

In this paper, we developed an efficient lane identification algorithm and based on that lane departure detection. Using the OpenCV platform the lane detection is done using the Hough transform algorithm and the edges are detected using the canny edges detector. The results proves that the real time based lane identification and tracking is done efficiently. The lane are detected correctly for the straight and the curves roads. The necessary indication is given to driver in case of the vehicles try to depart the lane. The proposed techniques for lane departure warning system is also successfully checked with captured videos.

The future work will focus on expanding the proposed system along with the conceptual clues or the sensors to accurately detect the obstacles and make smart driving vehicles. The road sign detection can also be included to make the system smarter.

ACKNOWLEDGMENT

The author would like to express his sincere thanks to Prof.

A. B. Deshmukh for his support, co-operation and valuable suggestions.

c) Right Departure

Fig. 7. Real time departure state results

REFERENCES

1. Y. Wang, E. K. Teoh, and D. Shen, Lane detection and tracking using B-Snake, Image Vis. Comput. Vol.22, no. 4, pp. 269- 280, Apr. 2004.

2. M. Aly, Real time detection of lane markers in urban streeets, in Proc . IEEE In-tell. Veh. Symp. Einddhoven, Thr Netherlands, pp. 7-12, 2008.

3. C. Mu and X. Ma, Lane detection based on object segmentation and piecewise fit-ting, TELKOMNIKA Indonesian J. Elect. Eng., vol. 12, no. 5, pp. 3491-3500, May 2014.

4. A. Parajuli, M. Celenk, and H. Riley, Robust lane detection in shadows and low illumination condition using local gradient features, open J. Appl. Sci. , vol. 3, no. 1B, pp.68-74,Mar. 2013.

5. Y.Li, A.Iqbal, and N.R.Gans, Multiple lane boundary detection using a combina-tion of low-level image features.In Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on, pp. 1682-1687, IEEE, 2014.

6. J.Wang, T.Mei, B. Kong, and H.Wei, An approach of lane detection based on In-verse Perspective Mapping, In Intelligent Transportation Systems (ITSC), IEEE 17th International Conference on pp. 35-3, IEEE, 2014.

7. S. Srivastava, M. Lumb, and R. Singal, Improved lanr detection using hybrid me-dian filter and modified Hough Transform , Int. J. Adv. Res. Comput. Sci. Softw. Eng., vol. 4, no. 1, pp. 30-37, Jan. 2014.

8. B. Yu, W. Zhang, and Y. Cai, A lane departure warning system based on machine vision, in Proc. IEEE Pacific-Asia Workshop Comput. Intell. Ind. Appl., 2008, pp. 197201.

9. Q.Lin, Y.Han, and H. Hahn, "Real-Time Lane Detection Based on Extended Edge-Linking Algorithm," Proceedings of 2nd International Conference on Computer Research and Development, Seoul, South Korea, pp.725-730, Jun. 2010.

10. V.S.Bottazzi, P.V.Borges, B.Stantic, Adaptive regions of interest based on HSV histograms for lane marks detection. In Robot Intelligence Technology and Applications 2, pp. 677-687, Springer International Publishing, 2014.

11. C.Guo, S.Mita, and D.McAllester, "Lane detection and tracking in challenging environments based on a weighted graph and integrated cues," in Proc. Int. Conf. on IEEE/RSJ Intelligent Robots and Systems,Taipei, Taiwan, Oct. 2010, pp. 6643-6650.

12. Y.-C. Leng and C.-L. Chen, "Vision-based lane departure detection system in urban traffic scenes," in Control Automation Robotics Vision (ICARCV), 2010 11th International Conference on 2010, pp. 1875- 1880.

13. H.Jung, J. Min, and J.Kim,"An efficient lane detection algorithm for lane departure detection," In Intelligent Vehicles Symposium (IV), 2013 IEEE, pp. 976-981.

14. S.Zhou, Y.Jiang, J. Xi, J. Gong, G. Xiong, and H. Chen, "A novel lane detection based on geometrical model and gabor filter," in Intelligent Vehicles Symposium (IV ), 2010 IEEE , June 2010, pp. 59-64.

15. H.Tan, Y. Zhou, Y.Zhu, D.Yao, and K. Li, A novel curve lane detection based on Improved River Flow and RANSAC, In Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on, pp.133-138, IEEE, 2014.