Image Segmentation and Shape Analysis for Road Sign Detection

Image Segmentation and Shape Analysis for Road Sign Detection

Pratiksha Manjare

Electronics-Telecommunication Department JSCOE, Pune

India

Prof. S. M. Hambarde Electronics-Telecommunication Department

JSCOE, Pune India

Abstract In recent years, extensive study has been performed to recognize road signs. Many methods have been developed over these years for road sign detection. Signs are used to represent some special circumstantial information of environment. Road signs provides important information for warning, guiding people to make their movements easier, safer and more convenient. This paper presents two different methods for road sign detection and recognition which has based on color and shape. The first method is Gielis curve fitting. In this method algorithm is applied to different shapes like circular, triangular, octagonal. Fitting contour points into a Gielis curve takes a lot of time, which naturally cannot be acceptable in a real-time system. Hence, further investigation and improvement is required to decrease the processing time. It gives 85% accuracy for road sign recognition. The second method is based on Neural network. The Neural network stages were performed to recognize the traffic sign patterns. The first step is to reduce the number of MLP inputs by preprocessing the traffic sign image, and the second step is to search for the best network architecture by selecting a suitable error criterion for training with LMA. Using neural network, road sign classification gives 90% accuracy for different type of signs (circular, triangular, octagonal).

Keywords Road sign recognition, Gielis curve fitting, Neural network.

1. INTRODUCTION

   A road sign gives vital information for driving purpose. For alerting drivers about road conditions to make driving safe, the road sign recognition system is an important tool. The road sign recognition system should be able to first detect, and then identify a set of road signs from within images. There are a number of important issues that need to be taken into consideration. These includes: direction of signs face, status of paint on signs, illumination conditions, placement of multiple signs near each other, torn and tilted signs, variation in signs scale, obstacles such as trees, image sensors properties and car vibrations. In traffic environments, road sign recognition is used to regulate traffic signs, warn drivers, command and prohibit certain actions. The road sign detection and recognition can support the driver, and thus, significantly increase driving safety and comfort. Generally, traffic signs provide the driver with a variety of information for safe and efficient navigation. The identification of various road signs with

   respect to natural background viewing conditions still remains a challenging task. The road sign recognition systems usually have been developed into two specific phases. The first phase is normally related to the detection of road signs in a video sequence or an image using image processing. The second phase is related to recognition of those detected signs, which deals with the interest of performance in a neural network technique. The detection algorithms are normally based on shape and color analysis. The efficiency and speed of the detection plays important role in the system.

2. PROBLEM STATEMENT

   The road sign recognition is a difficult task if aimed at detecting and recognizing sign image captured with unfavorable background. Complex background, weather conditions, lighting and shadows make the task complicated and difficult. An intelligent transport system has following concerns [1-5]:

   • Disoriented or damaged signs making it hard for the system to detect and recognize.

   • Images acquired are usually blurred because of car vibration and speed.

   • Poor visibility because of lighting and bad weather conditions.

   • Positing of road sign is also important for the system to detect the sign. Signs placed near trees often have portions hided by tree branches.

   • Color fading because of constant exposure to sunlight.

   • Presence of objects in backgrounds with likely shape and color.

   For problem solving strategy next section explains two different methods for detection and recognition of road signs.

3. PROPOSED METHODS

   Different countries represent different shape and color of road sign. Detection and recognition of road signs basically depend on two features that is shape and color. According to that two different methods are presented in this section.

   1. Gielis Curve Fitting for Shape Analysis

      Gielis curves and surfaces are used to describe a wide range of natural shapes which have been used in various studies in biology and physics as descriptive tool. The basic principle behind Curve fitting algorithm, in simple terms curve fitting or reconstruction of Gielis curve is proposed as minimization problem of corresponding cost function. For this Levenberg-Marquardt algorithm for Least Square Minimization of Non-linear Models are required. For shape reconstruction it is important to have a formula which can we manipulate with sufficient parameters related to the efficiency of cost function to fit to shape and Gielis curves with various parameters meet the requirements.

      .. (1)

      Gielis curve takes six parameters: a, b, m, n1, n2, and n3. a and b are the stretch parameter along the axes, m is the rotational symmetry of the shape and n1, n2, n3 are the shape coefficients.

      Fig. 1. Flowchart of Gielis curve fitting algorithm

      The algorithm of Gielis curve fitting consisting of different blocks which are used for to detect binary image, contour points and shape of the object.

      NHS segmentation is used to find the pixels in the image that is likely to belong to road sign by color segmentation. To do this, image is transformed to the IHLS (Improved Hue, Luminance and Saturation) color space to provide robustness against changes in lightness and then it is threshold according to hue and saturation, based on look- up table. The first step of the proposed method is the localization of potential road signs in the image through color segmentation. When image is converted to IHLS color space, segment it using the (NHS) Normalized Hue- Saturation method [8].

      Post processing is used to eliminate the noise present in the image and small objects that could cause bad results in the road sign detection by the contours and the convex hull points are determined. It is implemented to remove ambiguity from the input image which comes in the form of noise and other artifacts. Median filter is used for this purpose. Filter is applied to overall image to remove the noisy pixels, pixel drop-outs. The morphological operations are used to obtain proper contour of partially detected and deformed shape of the road signs. The main operations involved in this step are dilation and erosion. Dilation and erosion together performs closing operation which helps in recovering damaged contours of the objects of interest in the image.

      In contour extraction we have taken coordinates of the contour points as input and get the output is contour of the points nearer to the convex hull than the threshold. The next step in the algorithm is rotational offset. Rotational offset means angle required so that contour gets aligned to the Gielis curve before matching start. The rotational offset is very sensitive parameter for Gielis curve fitting recovery. The Gielis shape reconstruction is the final step of the algorithm. For shape econstruction to optimize the Gielis curve parameters to get the best fitting curve for the extracted contour points. For optimization applying the Levenberg-Marquardt algorithm, it optimizes the cost function of the Gielis curve. The cost function is the squared value of the distance of a point to the Gielis curve. The results of road sign shape detection using this Gieis curve fitting algorithm are shown in section IV.

   2. Neural network for road sign detection and classification

   In Neural network method detection is based on image processing and multi-layer perception with back propagation algorithm is used for training the network for classification of road signs. The capability of recognition of neural network increases with increasing the training accuracy. For this each sign is converted to a designated M x N feature matrix. These feature matrices of signs are then fed into the neural network as input patterns. The neural network is trained with the set of input patterns of the digits to acquire separate knowledge corresponding to each road sign. In order to justify the effectiveness of the system, different test patterns of the signs are used to verify the system [7]. Fig. 2 shows the flowchart of overview system which represents the number of steps for detection and recognition of road signs, which gives exact output. As per

   flowchart different techniques are applying for detection sub-module and classification sub-module.

   Fig. 2. Flowchart of overview system

   In detection sub-module consisting HSV transformation, color thresholding, extract size as per criteria selection. In recognition sub-module consisting classification by feed forward back propagation network.

   1. Introduction of RGB and HSV transformation:

      For describing an image RGB (Red, Green, Blue) is basic color space commonly used. RGB is three dimensional in its nature whose values vary from 0 to 255. Pure red value is represented by (255,0,0), pure green value is represented by (0,255,0) and pure blue value is represented by (0,0,255). HSV encodes color information by separating out an overall intensity value V from two values encoding chromaticity- hue H and saturation S [3]. Hue is defined by an angle between 0 and 2 relative to the red-axis, pure red is positioned at angle of 0, pure green is positioned at 2/3 and pure blue is positioned at 4/3. Saturation varies from 0 to 1.0, the corresponding colors vary from unsaturated to fully saturate. Value varies from 0 to 1.0, the higher the value, the brighter the color is. RGB is transformed to HSV using following formulas:

      .. (2)

      . (3)

      . (4)

   2. Color thresholding:

      To detect the red color road sign the color threshold method is used. Thresholding is used to classify pixels of an image into object pixels or background pixels. A pixel is called object pixel if its color is close enough to a reference color, if not it will be background pixel [4]. HSV color range for red road sign is as follows:

      Hue < 0.05 or Hue > 0.95 Saturation > 0.5

      Value > 0.01

      These ranges of values are used as the thresholds to find the red pixel in the image after the HSV conversation.

   3. Image labeling and Criteria selection:

      The image is converted to binary image consisting of only black and white colors, which is shown red pixel will be translated to white color and rest of the colors are black color. The binary image is then labeled according to each pixels 8-connectivity.Then criteria selection will be applied. It will examine labeled cluster size output from the previous stage. If the cluster is between 100 and 6000 pixels, it will be regarded as consisting of a road sign. This stage will get rid of objects like red flower, red building or even the road sign when it is too small and unrecognizable.

   4. Extract and Resize:

      Finally the road sign region is extracted and resize to 30 x 30 pixels and proceed to classification sub-module.

   5. Road sign Classification:

   A road sign image which contains 30 x 30 pixels is proceed to classification sub-module and MLP network is created and trained to classify the type of road sign in this sub-module. The classification sub-module is implemented using MLP (Multi-Layer Perceptron) neuron network. This network consist of a set of secondary units that constitute the input layer, one or more hidden layer of computation nodes, and an output layer of computation nodes [6]. One of the most important types of feed forward network is back propagation neural network. I t is a multi- layer feed forward network using extend gradient decent based delta learning rule commonly known as back propagation rule. In this method, the weights are updated based on the error between input and output pairs. The update can be done either by batch where the adjustment of the weights is done only after all the pairs have been presented to network or sequentially as each training pair is propagated through network. This training process above will be iterated until a certain condition set by the network designer is met. Standard back propagation is gradient decent algorithm, as is the Window-Hoff learning rule, in which the network weights are moved along the negative of the gradient of the performance function. The term back propagation refers to manner in which the gradient is computed for non-linear multi-layer networks.

   The input neurons take in the images features in YCbCr color space. YCbCr has three values corresponding to each pixel. These properties are used to train the network. The road sign is also transformed to YCbCr color space to classify a road sign before feed into classifier. In YCbCr color space Y is luminance component, Cb and Cr are the chroma components. The corresponding YCbCr values can be computed as follows:

   In above equation value of Y is in the range of [0,1], while the value of Cb and Cr is in the range of [-0.5,0.5]. The conversion from YCbCr to RGB color space is computed as follows:

   As per MLP network rules the network architecture is shown following figure to classify the type of road sign. The networks had been trained on 60 sets of training patterns, 20 for each type of road sign. The network has 2700 input neurons and three output neurons. The hidden layer is five neurons.

   Fig. 3. Feed forward back propagation network architecture

   Multi-layered networks are capable of performing just about any linear or non-linear computation, and can approximate any reasonable function arbitrarily well. Such networks overcome the problems associated with the perceptron and linear networks. However, while the

   network being trained might theoretically be capable of performing correctly, back propagation and its variations might not always find a solution.

4. EXPERIMENTAL RESULTS

   1. Road sign shape detection using Gielis curve fitting algorithm

      Gielis curve fitting algorithm which is basically implementation of image processing and Levenberg- Marquardt algorithm (LMA). In image processing we observed NHS segmentation, Noise removal, Contour extraction but for LMA is applying using rotational offset, and shape reconstruction. The shape reconstruction method can successfully identify different road sign types. Table 1 shows the results of road sign shape detection using Gielis curve fitting algorithm.

   2. Road sign classification using Neural network

   Using neural network, the color detection is done by HSV transformation and YCbCr color space. Shape detection based on region properties which is used for labeled region. Classification is based on feed forward network and train the image using by input neuron of YCbCr features of 30 x 30 image pixel range. Feed forward network and nearest neighbor classification which gives more robust and accurate results such as HSV image, binary image, labeled region and message after classification shown in Table 2.

   Table 1. Results ofroad sign shape detection using Gielis curve fitting algorithm

   Table 2. Results of road sign classification using Neural network

5. CONCLISION

In this paper we proposed two methods Gielis curve fitting algorithm and Neural network technique for road sign recognition. The Gielis curve fitting algorithm gives best results by color detection but not giving perfectly for shape detection. It gives 85% accuracy for road sign recognition. The Neural network classification is time consuming task but provided best result for identification of road sign. It gives best result for color and shape detection. This performance shows 90% accuracy for classification.

REFERENCES

1. G. Nicchiotti, E. Ottaviani, P. Castello, and G. Piccioli, Atomatic road sign detection and classification from color images sequences, In S. Impedovo, editor, Proc. 7th Int. Conf. on Image Analysis and Processing,World Scientific, 1994, pp. 623-626.

2. Z. S. a. J. Tian-tian, Intelligent approach of traffic sign recognition based on Color Standardization, presented at IEEE International Conference on Vehicular Electronics and Safety, October 2005.

3. T. K. J. Miura, and Y. Shirai, An Active Vision System for Real-Time Traffic Sign Recognition, presented at Proceeding IEEE Conference on Intelligent Transportation System, October 2010.

4. C. S. F. C. Y. Fang, S. W. Chen, P. S. Yen, A Road Sign Recognition System based on dynamic Visual Model, presented at IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003.

5. G. P. S. Vitabile, G. Pilato, E. Sorbello, Road Sign Recognition using a Dynamic pixel aggregation technique in the HSV color space, presented at Proceeding, 11th International Conference on Image Analysis and Processing Palermo, Italy, September 2001.

6. Simon Haykin Neural Networks : A comprehensive foundation© Prentice Hall; 2nd edition, July 6, 1998.

7. Sanjit Kumar Saha, Dulal Chakraborty, Md. Al-Amin Bhuiyan, Neural Network based Road Sign Recognition, International Journal of Computer Applications (0975-8887) Volume 50- No.10, July 2012.

8. Valentine Vega, Desire Sidibe, and Yohan Fougerolle, Road Sign Detection And Shape Reconstruction Using Gielis Curves, published in International Conference on Computer Vision Theory and Applications, Rome: Italy (2012).