Algorithm for Road Sign Detection for Driver Assistance from Complex Background

Algorithm for Road Sign Detection for Driver Assistance from Complex Background

Priya D. Nagarkar Elec. & comm. of Eng. Dept.

A.D. Patel Institute of Technology New V. V. Nagar, Gujarat-388121

Dr. Heena R. Kher Elec. & comm. of Eng. Dept.

1. Patel Institute of Technology New V. V. Nagar, Gujarat-388121

   Abstract Traffic Sign Recognition (TSR) is an important component of Advanced Driver Assistance Systems (ADAS). The traffic signs enhance traffic safety by informing the driver of speed limits or possible dangers such as icy roads, imminent road works or pedestrian crossings. Traffic sign detection leads to the detection of traffic sign from certain distance and informing driver well in advance. Various methods are studied in order to find suitable technique for effective and efficient road sign detection and recognition. This paper proposes a novel system for the automatic detection of traffic signs. Our system is able to detect all red colour Road Signs. The proposed system detects the road sign by Colour and Shape detection methods. Results show a high success rate. From these results, we can conclude that the proposed algorithm is invariant to translation, rotation, scale, and, in many situations, even to partial occlusions.

   Keywords Intelligent Transport Systems, YCbCr Colour Space, Extent, Shape Detection, Projected Area.

   1. INTRODUCTION

      Thousands of people are injured every year in automobile accidents. Most of those accidents are preventable, especially those caused by missing or incorrect road signs. Municipalities, namely local, state and federal authorities, are responsible for making sure that road signs are properly placed. When they are not properly placed and accidents occur, especially those resulting in injuries. Intelligent Transport Systems (ITS) have great potential to save time, to save money, to save lives, and to improve our environment. Automatic traffic sign detection and recognition is an important part of an advanced driver assistance system. Traffic symbols have several distinguishing features that may be used for their detection and identification. They are designed in specific colors and shapes, with the text or symbol in high contrast to the background. Because traffic signs are generally oriented upright and facing the camera, the amount of rotational and geometric distortion is limited. Information about traffic symbols, such as shape and color, can be used to place traffic symbols into specific groups; however, there are several factors that can hinder effective detection and recognition of traffic signs.

      The field of road sign recognition is not very old; the first paper appeared in Japan in 1984. The aim was to try various computer vision methods for the detection of objects in outdoor scenes. Since that time many research groups and companies are interested and conducted research in the field,

      and enormous amount of work has been done. Different techniques have been used, and big improvements have been achieved during the last decade. The identification of the road signs is achieved by two main stages: detection, and recognition. In the detection phase, the image is pre- processed, enhanced, and segmented according to the sign properties such as color or shape. The output is a segmented image containing potential regions which could be recognized as possible road signs.

      TABLE I

      MEANING OF INDIAN TRAFFIC SIGNS ACCORDING TO THE COLOR AND SHAPE

      COLOR	SHAPE	MEANING
      Red Rim	Circle	Prohibition
      Red Rim Up	Triangle	Danger
      Red Rim Down	Triangle	Yield
      Red	Octagonal	Stop
      Blue	Square	Recommendation
      Blue	Circle	Obligation
      White	Circle	End Of Prohibition
      Yellow	Circle	End Of Prohibition(Construction)

      The efficiency and speed of the detection are important factors which play a strong role in the whole process, because it reduces the search space and indicate only potential regions. In the recognition stage, each of the candidates is tested against a certain set of features (a pattern) to decide whether it is in the group of road signs or not, and then according to these features they are classified into different groups. These features are chosen so as to emphasize the differences among the classes. The shape of the sign plays a central role in this stage and the signs are classified into different classes such as triangles, circles, octagons, etc.

      In Section II, we review previous work and state the improvements that we make. Then, in Section III, characteristics of road sign is explained In Section IV, road sign detection system is introduced , various results are shown in addition to that comparison is carried out. In Section V conclusions are drawn and in Section VI future scope of the system introduced is stated.

   2. RELATED WORK

      Most traffic recognition algorithms divide the problem into three stages: 1) a rough segmentation to determine the location

      of the signs; 2) category determination; and 3) candidate classification to identify the content of the extracted traffic signs using various machine learning techniques. We will concentrate on the classification phase in this paper and give a brief overview of some of the existing techniques used in TSR in the following.

      Three basic detection techniques are Shape based, Colour based and Learning based detection techniques. Shape based detection techniques involved geometric terminologies and standard shapes of road sign are detected. Shape detection techniques are used in [2], [8], [17], [18] and [20]. All the road signs have common colour format. Each colour indicates particular information. Colour detection techniques are used in [1], [5], [6], [10], [11] and [12]. So for detection purpose colour classification is carried out. Learning methods comprises of all soft computing techniques. In [6] group sparse coding approach is used for detection purpose. There are four main techniques in learning based methods, Support Vector Machine, Maximally Stable Extremal Region, Scale Invariant Feature Transform and Histogram Oriented Gradient.

      Support vector machine is highly used for classification purpose. *Histogram oriented gradient method is used in [13]. High efficiency is obtained using learning based methods. Henceforth the are generally used for recognition purpose.

      speed limit as the pictogram. Another exception is octagon with red background with STOP in white and blue circle with red border and cross represents no parking sign.

      1. Regulatory Sign: They are mandatory sign to control the action of the drivers on road. They are used to regulate the traffic flow and vehicles moving on road. Blue circle with white border represents a regulatory sign. While the arrows within it represent the movement of the vehicle on road.

      2. Information Sign: Important information like nearby hospitals, telephone booth, first aid, petrol pumps etc. come under this category. This information helps the driver in emergency in need. White rectangle with thick blue border and the necessary pictogram represents the required pictogram. Parking information is also included in this categoy.

   3. CHARECTRISTICS OF ROADSIGN

      Traffic signs have been designed so that they are easily recognisable from natural and driving environment. The colour for traffic sign are chosen such that, it serves different purposes and is also distinguishable for the driver while driving. The signs are represented by fixed shapes like triangle, circle, octagon, and rectangle. The combined feature of colour and shape are used by driver to distinguish a traffic sign. Hence an automated system also uses the same principle of the colour and shape property of traffic signs. With respect to the road the traffic signs are located at well-defined locations so that the drivers can more or less expect the position of the sign. The road sign may contain text as a string of characters or pictogram or both to represent the meaning of the sign. They are characterised by using fixed text fonts and character heights. There are a number of traffic signs in India categorized as WARNING (40), COMPULSORY (27), REGULATORY (10) and

      INFORMATORY (15). This makes a total of 92 traffic signs all together. These signs are mainly characterized by colour and shape. Figure 1 shows the different types of Indian traffic sign and their description are discussed below.

      1. Warning Sign: A triangle with red coloured border and white background represents a warning sign. Different pictograms in black are used to represent the various warnings. These signs alert the driver with hazard ahead.

      2. Compulsory Sign: Compulsory signs uses circle with red border and white background. These signs restrict the action of drivers depending on the pictogram represented on the sign. Signs with a cross prohibit the driver from certain decision such as no left turn or no U turn etc. Speed limit signs are also included in this category of traffic signs, with

      Fig. 1 Different category of traffic signs [5]

      However identification of traffic sign is still a challenging task due to different geographic and weather conditions like cloudy day, raining or foggy day. The lighting conditions are uncontrollable since it is time dependent and seasonal, for example day light and night fall. More over the signs are of different types. Distance between the sign and video capturing device is a factor. The blurring of the image is dependent on the speed of the moving vehicle. Other problems are sign may be disoriented, damaged, faded or occluded. There may be similar objects as in colour or shape. Hence most sign detection system use both colour and shape as distinguishing feature and pre-processing techniques for image enhancement for coping with the varying lighting conditions.

   4. ROAD SIGN DETECTION SYSTEM

      1. Overview of system

         The proposed system consists of following main stages for road sign detection. Firstly the image of road sign is captured along with complex background. Then the captured image is converted from RGB color space to YCbCr color space. Later red colour is extracted from the image then thresholding is applied in order to remove unwanted elements present in the image. Next step is the removal of noise. Later

         obtained regions are filled and area computation is carried out. Lastly the obtained image have to perform x-or operation with original image. Hence the required road sign is successfully detected.

      2. Detection of Road Sign

         Initially road sign image is captured using camera. This process is termed as Image Acquisition. As we are focusing only on red color road sign. So the RGB color space is converted to YCbCr color space. Red color is extracted from the image by obtaining Cr plane.

         After determination of shape, system will keep objects with any of the standard shape as it is and it will discard all those which does not satisfy any of the range specified. Lastly performing morphological operation to obtain required road sign.

         Y 16 65.481

         128.553

         24.996 R

         Cb 128 37.797 74.203 112.000 G

         (1)

         Cr 128 112.000 93.786 18.214B

         Fig.4. Road sign Detected

         Fig.5. Image 2 Fig,6. Result 2

         Fig.2. Original Image Fig.3. Red Colour Extracted

         Once red colour is extracted, thresholding is applied. So all the unwanted objects are removed. Small components are eliminated by removing noise. From all the obtained objects region filling is carried out in order to compute the area of al objects. Then shape is determined by using eccentricity and extent.

         (2)

         Extent = Area of Object

         Area of Bounding Box

         Fig.7. Image 3 Fig,8. Result 3

         As mentioned earlier, road signs are generally divided into four main shapes. Square, Circle, Triangle and Octagon so firstly detecting only red color objects and then determining its shape will reduce complexity and efficiency is increased. Extents for all particular shape have unique range. Following mentioned are the constant range for standard shapes.

         TABLE II

         EXTENT RANGE FOR SHAPES

         Shape	Minimum Value	Maximum Value
         Square	>0.78	<0.95
         Circle	>0.65	<0.78
         Triangle	>0.3	<0.6

         Fig.9. Image 4 Fig,10. Result 4

         Fig.11. Image 5 Fig,12. Result 5

         Fig.13. Image 6 Fig, 14.Result 6

      3. Comparative Analysis

      The results shown in Table III gives out time consumed by the system to detect road sign effectively. In addition to that projected area is also computed. Projected area means the number of pixels present in the detected road sign. From all the images in the database, selected cases are displayed. Approximate time range from the analysis is concluded from

      1.1 to 1.56 seconds. Time range has to minimised.

      TABLE III

      PARAMETERS COMPARISION

      Image	Time (sec)	Projected Area
      Image 1	1.1	3020
      Image 2	1.1	1965
      Image 3	1.32	8478
      Image 4	1.57	6114
      Image 5	1.49	9596
      Image 6	1.56	1267

      All the techniques studied from literature survey, conclusions obtained by comparing techniques are shown in Table IV. Accuracy obtained is maximum in learning based techniques and minimum in colour based methods. There are four main controlling parameters 1)Timing Constraints 2)Illumination Factors 3)Distance Manipulation 4) Computation Error, timing constraint is the time consumed by system to obtain output effectively. It should be as low as possible. As all the signs are located by roadside so it illumination factors turns out to be very crucial parameter to focus on. In distance manipulation basically from the farthest distance road sign should be able to detect. As observed from the gravity of system it turns to be very important to detect road sign correctly.

      TABLE IV COMPARISION OF EXISTING METHODS

      Parameters	Colour Based	Shape Based	Learning Based
      Accuracy	Low	Moderate	High
      Speed	Moderate	High	Moderate
      Distant Images	Easy	Moderate	Moderate
      Robustness	Less	More	More

   5. CONCLUSION

We have proposed a system for road sign detection. Detction of road sign is effectively carried out using color detection and shape detection techniques. Traffic sign detection based on color and shape is presented in this work. YCbCr color space is used for color segmentation to overcome the illumination sensitive characteristic of RGB space. This detection system is insensitive to broken image or during night mode. Shape detection methods highly efficient outputs, as with calculation of extent and eccentricity all the standard shapes can be filtered out. The system retains high accuracy at almost all the challenging conditions. Our system can detect all of the red signs available in India.

REFERENCES

1. Fatin Zaklouta and Bogdan Stanciulescu, Real-time traffic sign recognition in three stages, Robotics and Autonomous Systems, Elsevier, pp 16-24,2014.

2. Sanket Rege, Rajendra Memane, Mihir Phatak and Parag Agarwal, 2d Geometric Shape And Color Recognition Using Digital Image Processing, International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering,Vol. 2, Issue 6, June 2013.

3. P. Paclok , J. Novovicova , P. Pudil and P. Somol, Road sign classification using Laplace kernel classifer, Pattern Recognition Letters Elsevier, pp 1165-1173, 2000.

4. S.H. Hsu and C.-L. Huang, Road sign detection and recognition using matching pursuit method , Image and Vision Computing Elsevier, pp 119-129, 2000.

5. A. de la Escalera, J.Ma Armingol and M. Mata, Traffic sign recognition and analysis for intelligent vehicles, , Image and Vision Computing Elsevier, pp 247-258, 2003.

6. Huaping Liu , Yulong Liu and Fuchun Sun, Traffic sign recognition using group sparse coding, Information Sciences Elsevier, pp 75-89, 2014.

7. Miguel Angel Garcia, Miguel Angel Sotelo and E. Martin Gorostiza, Traffic Sign Detection in Static Images using Matlab.,IEEE International Conference on Intelligent Robots and Systems pp 212- 215, 2003.

8. Gareth Loy and Nick Bames, Fast Shape-based Road Sign Detection for a Driver Assistance System, IEEE International Conference on Intelligent Robots and Systems, pp 70-75,2004.

9. Reza Oji, An Automatic Algorithm For Object Recognition And Detection Based On Sift Keypoints, Signal & Image Processing : An International Journal (SIPIJ) Vol.3, No.5, pp 29-39 ,October 2012

10. Hasan Fleyeh, Color Detection And Segmentation For Road And Traffic Signs, IEEEConference on Cybernetics and Intelligent Systems, pp 808-813, December, 2004.

11. Chiung-Yao Fang , Chiou-Shann Fuh and Sei-Wang Chen, Road-Sign Detection and Tracking, IEEE Transactions On Vehicular Technology, Vol. 52, pp 1329-1341 SEPTEMBER 2003

12. Saturnino Maldonado-Bascón, Sergio Lafuente-Arroyo, Pedro Gil- Jiménez, Hilario Gómez-Moreno and Francisco López-Ferreras, Road-Sign Detection and Recognition Based on Support Vector Machines, IEEE Transactions On Intelligent Transportation Systems, VOL. 8, pp 264-278, JUNE 2007.

13. Jack Greenhalgh and Majid Mirmehdi, Real-Time Detection and Recognition of Road Traffic Signs, IEEE Transactions On Intelligent Transportation Systems, Vol. 13, pp 1498-1506, DECEMBER 2012.

14. Fatin Zaklouta and Bogdan Stanciulescu, Real-Time Traffic-Sign Recognition Using Tree Classifiers, IEEE Transactions On Intelligent Transportation Systems, Vol. 13, pp 1507-1514, DECEMBER 2012.

15. Mario Muñoz-Organero and Víctor Corcoba Magaña, Validating the Impact on Reducing Fuel Consumption by Using an EcoDriving Assistant Based on Traffic Sign Detection and Optimal Deceleration Patterns, IEEE Transactions On Intelligent Transportation Systems, Vol. 14,pp 1023-1028, JUNE 2013.

16. C.Y. Fanga, C.S. Fuhb, P.S. Yena, S. Cherngc and S.W. Chen, An automatic road sign recognition system based on a computational model of human recognition processing, Computer Vision and Image Understanding ELSEVIER, PP 237-268, 2004.

17. Erkut Kirmizioglu, Hediye Tuydes-Yaman, Comprehensibility of traffic signs among urban drivers in Turkey, Accident Analysis and Prevention ELSEVIER, pp 131-141, 2012.

18. Leonardo Brunoa, Giuseppe Parlaa and Clara Celauro, Improved Traffic Signal Detection and Classification via Image Processing Algorithms, Procedia – Social and Behavioral Sciences ELSEVIER, pp 811-821, 2012.

19. Bram Alefs, Guy Eschemann, Herbert Ramoser and Csaba Beleznai, Road Sign Detection from Edge Orientation Histograms, IEEE Intelligent Vehicles Symposium, pp 993-998, 2007.

20. Elisabet Pérez and Bahram Javidi, Nonlinear Distortion-Tolerant Filters for Detection of Road Signs in Background Noise, IEEE Transactions On Vehicular Technology, Vol. 51,pp 567-576, MAY 2002.