Robust Traffic Sign Recognition with Illumination Variation

Robust Traffic Sign Recognition with Illumination Variation

Satyajit Bhattacharya, Sayan Giri

TIFAC-CORE IN AUTOMOTIVE INFOTRONICS, VIT UNIVERSITY, INDIA

Abstract

Traffic sign are essential to road safety. It helps the driverby giving valuable safety and navigation related information. This paper aims to propose an efficient algorithm that will detect as well as recognize traffic signs in a wide range of lighting conditions from the real time video image of road side which is taken by a video camera placed on the top of a car. Though, our algorithm isbased on colour based (red & blue colour) detection of the traffic sign. The use of Improved Hue Luminance Saturation (IHLS) color space and Normalized Hue Segmentation (NHS) make our algorithm more robust with respect to change in illumination. The detection of the traffic sign has been accomplished using various image processing operations like RGB to IHLS conversion, NHS, filtering, labelling connected pixels and ROI extraction. And the traffic sign recognitionis accomplished by pixel-based template matching which is based upon the correlation between the detected traffic sign in the input image and pre-defined standard traffic sign images stored in the data base. This algorithm had been implemented and tested using Matlab. Experimental result shows that the system is robust to changes in the illumination and also able to identify multiple traffic signs placed at any position on road side. The average identification rate of true sign is around 90.27% and false identification is 9.73%.

1. Introduction

   Driving needs continuous processing of visual information from the road. To avoid accident, driver needs to monitor a lot of traffic signs. For this purpose traffic signs play an important role to provide information about traffic and road conditions which is necessary for a driver to accomplish a collision free driving environment. We know that traffic signs are made in a standardized colours (red and blue), shape (circle, triangle and square) and pictograms.In many cases, the traffic signs use easily recognized symbols or pictures rather than words. They are also situated in a specific position on the road side to provide sufficient

   information like speed limit, bumper, stop etc. In last decade due to heavy traffic, it becomes very difficult for a driver to monitor all the traffic sign along the road, which sometime leads to accident. So, this paper will help to develop an efficient, robust and reliable real time driver assistant system which will be able to detect the traffic sign with minimum error in different lighting conditions.

   This traffic sign detection algorithm has two parts: detection of traffic sign, followed by recognition of the signs. Traffic signs can be detected either by shape or color. Shape based detection algorithms use techniques like the Hough transform to detect lines in order to identify and determine the shape of the road sign [1]. Though shape based detection method are efficient but they are prone to false detection due to presence of objects oftriangular, circular and rectangular shapes on road side that are not traffic signs. Benallaet. al. [2] proposed a color segmentation algorithm for segmenting Red, Green and Blue regions in the given image. This algorithm is well enough for segmenting traffic signs in ideal illumination condition. In non- ideal illumination conditions, the performance becomes relatively poor due to fixed threshold values. In our approach, use of IHLS and NHS compensate for the loss of performance due to fixed threshold values. This makes our system more robust with respect to changes in illumination. Our approach for recognition of traffic sign involves pixel-based template matching.

   In ideal condition (off-line indoor detection of signs with direct front view) traffic sign recognition is not very hard in principle as signs have discriminating color and 2-D shape [3]. But on a real roadit becomes a complex operation due to variety of signs with all different colors, shape and pictographic symbols, complex and uncontrolled road environment (lighting, shadow, occlusion, orientation, distance) which all can be categorized as noise and real-time recognition. In the following section our methodology for traffic detection and recognition system is described.

2. Proposed Methodology

   Fig.1 Block diagram of Traffic sign Recognition Algorithm

   The first section is Image acquisition and pre- processing. Image acquisition involves capturing video from camera and extracting each frame from video for processing. Pre-processinginvolves resizing of each frame to pre-determined size. It also computes the correlationbetween current and the previous frame. If the correlation value is 1, the current frame is discarded under assumption that car is idle and traffic signs from the previous frame are displayed. This reduces the computation pressure from the system, by avoiding processing of same frame continuously.

   The second section consists of Normalized Hue Segmentation (NHS) Block. The pre-processed input image is converted from RGBcolor space to Improved Hue Luminance (IHLS)color space. The RGB to IHLS conversion is done using mathematical relation (1), (2),

   (3) and (4) based on [4].

   Y c = 0.2126R + 0.7152G + 0.0722B(1) S(c) = max(R, G, B) min(R, G, B)(2)

   H(c) = arccos 0.50.5 (3)

   2 + 2 + 2 0.5

   = 360 , > ;(4)

   ,

   This makes the NHS segmentation more robust with respect to changes in illumination. This block plays a crucial role in traffic sign detection. The IHLS image is segmented using appropriate hue and saturation threshold, for either the red or blue colour to extract pixels from IHLS image that matches our criterion [5]. The result is a binary image which shows the areas with colours matching those of road signs.

   The binary output of the NHS block consists of traffic sign pixels along with noise. The noise pixels are obtained from other red or blue colored object in the image than traffic sign. This noise can be randomly distributed. This constitutes salt noise along with larger patches. The noises are eliminated using the following morphological operators: The NHS output image is applied with a 3×3 median filter to remove fine salt noise. The image is morphologically opened by erosion followed by dilation using a 10px disk structural element. The image is morphologically closed by dilation followed by erosion using a 10px disk structural element. These operations remove any isolated objects comparable to size of structural elements, thin protrusion. We consider our signs to consist of symbol surrounded by red or blue lined shape. Hence, for a valid traffic sign, the NHS output will consist of a hole in between traffic sign pixel. This hole is detected by tracing the exterior boundary of objects, as well as the boundaries of holes inside theseobjects.

   In order to extract the ROI from the input image, first step involves connected pixel labelling [6]. Connected components labelling involves scanning an image pixel by pixel in order to identify connected pixel region, i.e. regions of adjacent pixels which share the same set of intensity value V and are in some way connected with each other. The connected components labeling operator scans the image by moving along a row until it comes to a point p (where p denotes the pixel to be labeled at any stage in the scanning process) for which V={1}. When this is true, it examines the four neighbour pixels of p which have already been encountered in the scan (i.e the neighbour to the left of p, above p and the two upper diagonal terms). Based on this information, the labeling of p occurs as follows:

   If all four neighbour pixels are 0, assign a new label to p; else if only one neighbour has V= {1}, assign its label to p; else if one or more of the neighbours have V= {1}, assign one of the labels to p and make a noteof the equivalences. After completing the scan, the equivalent label pairs are sorted into equivalence classes and a unique label is assigned to each class. As a final step, a second scan is made through the image, during which each label is replaced by the label

   assigned to its equivalence classes. Labelling of connected pixel is followed by ROI extraction.It involves measuring of certain properties of each labelled region using interactive segmentation technique [7]. Properties are Bounding Box (the smallest rectangle containing the regions) and Area (actual number of pixel in the region). Bounding Box gives the coordinates of rectangle holding the traffic sign. Area is used eliminate labelled objects with area smaller than maximum allowed area for avoiding error in recognition.

   The last section deals with Recognition of the traffic sign symbol in the extracted ROI. Recognition using template matching uses a database of images.

   Fig. 2 Traffic sign templates

   Fig. 3 Traffic sign templates after eliminating boundary.

   The cropped ROI is pre-processed to eliminate any noise. Now, we compute the correlation between the cropped ROI with each known traffic sign symbol available in database to obtain the correlation image. Since, value of correlation is zero for no match and 1 for exact match. Since, traffic sign database template and cropped image vary in size; correlation image includes mostly zero and highest value at point of best match. For a particular database template maximum value in correlation image is considered. The database template that gives the highest correlation value is

   projected a most fit candidate. So the label corresponding to that template is queried and outputted.

   Consider input image as f(x, y) of size MxN and a template w(x, y) of size JxK. In its simplest form, the correlation between f(x, y) and w(x, y) is

   , = , + , +

   where x=0, 1, 2, M-1 and y = 0, 1, … N-1 and summation is taken for the region where f and w overlaps. In our project template matching is optimized by keeping the size of template and the cropped image very close to each other.

3. Result and Discussion

   We tested our algorithm in several lighting condition with both red and blue traffic sign.

   Fig. 4 Original image (left), NHS output after filtering (centre) and final output image with detected sign.

   Fig.5 Extracted ROI (RGB) and processed ROI for template matching (Binary image)

   Fig. 4 Original image (left), NHS output after filtering (centre) and final output image with detected sign.

   Fig. 5 Test image in afternoon, with sun light falling on the sign (left) and in evening (low light intensity)

   The result obtained from a video is given below:

   TABLE I.

   No. of Test Images	50
   No. of signs	72
   True Recognition	65
   False Recognition	7
   % Recognition	90.27
   % Error	9.73

4. Conclusion

   Our work implements a fast and light weight traffic sign recognition algorithm, which is independent of lighting conditions.The processing time for typicalimage of resolution 540×320 pixels is around 1-

   2 seconds in an Intel x86 machines.The minimum detectable size is 20 x 20 pixels. Our approach uses IHLS color space and Hue segmentation as primary step for Traffic sign detection. Secondary detection includes noise filtering, labelling, ROI extraction. Finally ROI obtained is utilized for recognition. Effort has gone in making the inefficient template matching more accurate for above purpose. The system was tested in different lighting conditions with different color signs.

5. Reference

1. Siti Sarah MdSallah, FawnizuAzmadiHussin, MohdZukiYusoff. Shape-based Road Sign Detection and Recognition for Embedded Application Using MATLAB in IEEE International Conference onIntelligent and Advanced Systems, pp.1-5, June 2010.

2. M. Benallaland J. Meunier. Real time colorsegmentation of road signs in IEEE Canadian Conference on Electrical and Computer Engineering, vol. 3, pp. 1823-1826, May,2003.

3. Garcia, M.A., Sotelo, M.A., Gorostiza, E.M., Traffic Sign Detection in Static Images using Matlab, in Proceedings ETFA '03. IEEE Conference, Vol. 2, pp. 212-215, 2003.

4. Hanbury A., Serra J. Color Image Analysis in 3D- Polar coordinates in Proceedings25th DAGM Symposium, Magdeburg, Germany, pp. 124-131, September, 2003.

5. Valentine Vega, Desire Sidibe, and Yohan Fougerolle. Road Sign Detection and Shape Reconstruction using Gielis Curves in VISAPP 2012, 7th International Conference on Computer Vision Theory and Applications, Roma, Italy, pp. 93-396,

   February, 2012

6. Kenji Suzuki, Isao Horiba and Noboru Sugie. Linear-time connected-component labelling based on sequential local operations In Academic

   Press, Computer Vision and Image Understanding 89 pp. 123, 2003

7. Yuri Y. Boykov, Marie-Pierre Jolly. Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images in Eighth IEEE Proceedings of International Conference on Computer Vision, Vancouver, Canada, vol. 1, pp.105- 112, July, 2001.

.

Authors Profile

Satyajit Bhattacharyareceivedhis B.TECH degree in Applied Electronics & Instrumentation engineering fromUniversity of Calicut, Kerala, India in 2011. Currently, he is persuing his M.TECH degreein Automotive Electronics fromVellore Institute ofTechnology. His area of interest coversImage Processing,Embedded System and Robotics.

Sayan Girireceivedthe B.TECH degree in Electronics & Communication engineering from WestBengal University ofTechnnology, India in 2012. Currently he is persuing the M.TECH degreein Automotive Electronics from Vellore Institute ofTechnology. His area of interest covers Signal Processing, Image Processing and Control System.