Unmanned Vehicle

Unmanned Vehicle

Senkathir Selvan S1,Saravanan S2,Vignesh Raja3

SVCE

Abstract :- One of most important issue in research for unmanned vehicle is detecting signal lights at crossroads and level crossings. This paper presents new algorithm for signal light detection. It has high detection rate with real time and fast processing at low price. Proposed algorithm realizes driving with reliability in unmanned vehicle.

Keywords: Light Detection, Real-time Signal, Unmanned Vehicle

INTRODUCTION:

Vehicle driving without human is a dream of people. There are many studies about Intelligent Transport System (ITS) and Unmanned Ground Vehicle (UGV). Interest on unmanned vehicle is increasing. One of important issue in UGV is detection of signal light. Which is necessary when train is on through a railway gate with level crossing. RF(Radio Frequency) system is a perfect system for safety crossing, but, for using RF system, all of the level crossings must have RF receiver and sender, and all the compartments of the train should have RF terminal. All of intersection should be changed. This method requires lots of cost. Vision system is cheaper than RF approach. Therefore, vision detection system for signal lights detection should be developed. Also, signal light detection provides help to safety driving. There are many accidents in level crossings. Such accidents are caused by careless attitude of people while crossing the rails. Signal light detection system can support driver to notice about signal light . In this paper we present a new vision algorithm for signal light detection which has real time processing with high detection rate while using low price camera. Using this algorithm, reliability and safety for UGV and drivers will be increasing.

RELATED WORKS:

There have been a lot of studies about unmanned vehicles and signal light detection. Popular one is a contest by DARPA (The Defence Advanced Research Projects Agency)[1]. DARPA open Grand Challenge in 2004, 2005 and 2007. In this contest, cars drive with no driver. Driving environment is dirt road in desert. In 2004, no teams arrived at goal. In 2005, Stanford University Team, led by S. Thrun, won with the car named Stanley equipped with many sensors[2]. However, limitation was Stanley is made for driving in the dirt load. Detection of signboard and signal light in urban road is excepted. In 2007, there are many changes in rule of contest. Urban Challenge was opened. Urban Environment was made artificially. In 2007, difficulty is high, because of very complex urban environment. Team in Canegi Melon University won in 2007.

In this contest, car must detect signboard and signal lights. Urban challenge use environment under control for contest, therefore, good condition for UGV. In 1998, real-time stereo vision system for generic obstacle and lane detection was developed by Massimo Bertozzi and Alberto Broggi[3]. But this system detect only lane on the road and obstacle. This system is for cruising in normal road. There are many researches for signal light detection. First, there is study based on colour tracking by Mahipa. R. Yelal et al[4]. This research is about real-time tracking of signal light. But, limitation is only simulation by using Matlab, and, condition of sample image is excellent. Background of signal lights is clean sky and, no experiment in real urban road. Another study for signal light detection is using Fuzzy AI by Yun-Chung Chung et al[5]. They use position of light, and processes with Fuzzy Map. Using position, it can detect red, yellow and green signal. But time for processing is 0.5~1 sec per frame. Therefore, it does not support real-time processing. There is another research built in real car by Lampros Tsinas and Volker Graefe[6]. In this case, they use HSI colour map for signal light detection. That takes 20ms. That is fast, and experiment in real road, but limitation is that algorithm cannot divide light from cars and another from signal lights. It makes a lot of errors in detection. Therefore, other light sources, they are not signal lights, can be detected in this research. It is a serious error.

System Architecture: This signal detection algorithm is developed by part of Intelligent Vehicle Central Control System. This project is developed for base technology of unmanned vehicle for real railway system.

Architecture of Intelligent vehicle is in figure 1.

All data from each sensor are parsed by each correspondence system. GPS system can track position of train in macroscopic. INS System can detect microscopic position for control train. Lane detection system can detect railway lane . Obstacle detection system detects obstacle in front of train engine. Finally, signal light detection system detects signal light. Therefore, train can stop and go at stations as well as at the level crossings. Each system is connected with steering wheel control system and brake / accelerator control system. Signal detect system consists of several steps. First, extract candidate pixels of signal light from grabbed image, and cluster extraction result for separating each object. To determine the object is signal light or not, applying circularity check for each object. Looking around the object, find frame of signal light. Finally, remove noise and increase reliability by robust filtering.

For clustering steps, this algorithm uses K-means centroid clustering. K-means centroid clustering is introduced by MacQueen in 1967[7]. To get good result, K-means centorid clustering can be run many times recursively, but running one time also get good result in this algorithm. Using K-means centroid clustering, there are two criteria for detection. First thing is about size of object. K-means centroid clustering can recognize object that have smaller size than threshold. Therefore, if threshold has a size of signal lights, large objects that is not signal light, but have same colour are rejected. Second, some small clusters are regard as a object. In K-means centroid clustering, If there are clusters which have distance to each other is less than threshold range, they are regard as an object. In signal light detection, that criterion is critical. Two of light sources that is far from train can be detected as a light source. There are two or more of signal lights. These clusters can be detected as an object, but, they will be rejected in circularity check steps. Figure 4 shows result of clustering. In figure 3, there are two signal lights. In figure 4, clustering step detects objects successfully. The object is drawn by white square.

ALGORITHM:

Colour Extraction: Colour extraction step extracts candidate pixels that may come from signal light from grabbed image. This step selects candidate pixels, using statistical processing. Average value of red, green, blue and value of standard deviation are determined for each sample image of signal light. Select pixels within range that is determined from average value and standard deviation and constant. Pixels of signal lights are extracted from grabbed image in figure 3. Also, pixel from no signal lights are extracted from image, but probability of extracted light pixel is higher than that of extracting error pixel. Generally, error pixels are noise. Its probability is low.

Clustering: Clustering step is to recognize set of pixel for each object corresponding to the lights source by centroid clustering from grabbed images. We can know centroid position, size, and standard deviation about position to this step.

Circularity Check: Object that has a shape of circle is detected in this step. There are many ights sources that have colour like signal light. For example, they are rear light of car, signboard of buildings and etc. These objects are rejected in circularity check. There are many algorithms for circularity measurement. One of popular algorithm is introduced by Haralick[8].

Eq. (1) is circularity measurement by Haralick. If the circularity measure C for the object is high, it is close to circle.

circularity measurement algorithm. First, this algorithm has insensitivity to noise. In experiment, this research uses low price camera. It makes many noises. This algorithm considers object if it has low standard deviation than threshold. That means, if threshold is high, noise will be ignored. Second, if object has small size, this algorithm is insensitive to shape. In figure 6, small square is considered as a circle. If distance from signal lights to train is long, there are many noises. Therefore, object may be not regard as a circle. Therefore, our algorithm is good for detection. Pseudo Code of this algorithm is below:

For all pixels in the each object

If (distance from center < (standard deviation of x position

+ standard deviation of y position from center to each pixel in object))

{

Get average of square of distance for each pixel, Get average of distance for each pixel.

}

If ((average of square of distance)-(square of average of distance) > threshold)

{

Consider this object circle.

}

In this implementation, we do not use Haralicks circularity measurement, but, similarly, this circularity check uses standard deviation. This algorithm uses a virtual circle using standard deviation. Calculate standard deviation using pixels out of virtual circle. If that value is less than threshold, consider that object as circle. Figure 5 shows result of circularity check of sample grabbed image. Circularity of object is higher than threshold. Therefore, they are recognized as circle. Figure 6 is result of circularity check for some sample figures. If object has high circularity, colour of square around object will be green. If object has low circularity, colour of square around object will be red. In this figure, some shapes have various type of noise, and some shapes are not circle. But some of them is recognized as circle. There are some features in this

FRAME CHECK:

This step scans neighbourhood of object, and whether object is a signal light or not. Circularity check distinguishes object that has circular form light source, but, there are many light source that have shape likes signal light. Trains have tail light with circular form. Some of signboards have red circle. They have to be rejected. If object has a red colour, scan right side of object, otherwise scan left side of object. Size is set by that of object. If they have low brightness, object is considered as signal light. Figure 7 shows result of frame check. In this sample image, there are red signal light. Therefore, frame check step watches right side of object. If this object is signal light, its right side will be dark. If object is signboard or rear light of train, brightness of right side is high. Therefore, they are rejected.

ROBUST FILTERING:

For improving quality, Robust Filtering is applied to result. Grabbed image has many noises. Robust filtering reduces noise. Because of noise, result is cracked. Therefore quality of result of frame check is low. It may tell there is a signal light even if there is not. Even if there is signal light, it may tell there is not. But, in case of signal light, result of only few frames may tell there is not. Also, even if there is not signal light, result of few frames show there is. Therefore, if there are many frames with signal light in last several frames, we consider at this moment there is signal light. If there are only few frames in last frames with signal light, consider there is not signal light.

PRIORITY POLICY:

This algorithm has policy for priority between objects, if many objects are detected in same time. It is related with size, position, colour, and shape. First priority is related with size and position. Depending on view point, Camera can see numbers of signal lights. In this case, a near signal light from car has priority. For example, depending on position, camera can watch signal light on the next tracks. Second, priority is related with colour. In India, there are many lights sources in one signal light frame. In this case,

red signal light has priority. In figure 9, there is example of this case in left image of third row. In this algorithm, only circular figure is allowed. And, any other figure is rejected.

EXPERIMENT AND RESULT:

All of computers and other equipments are carried out in real vehicle. Environment is below: First, CCD (Charge coupled device) camera has low price with no special brand. Price is less then $100. ISO and shutter speed cannot be controlled by manual. Size of CCD is 1/4 inches. CCD camera connected with NTSC to image grabber. Also, image grabber has low price. It is connected with USB (Universal Serial Bus) to computer. Computer is Mac Mini made by Apple corp. It has CPU (Central processing unit) of Core Duo with 1.66 GHz and 512MB RAM. Grabbed image has 320×240 of resolution and 30fps of frame rate. Development tool is Visual C++ 6.0, and Open CV library v1.0. Weather was cloudy. Cloudy weather is not a good condition. ISO sensitivity and shutter time of camera is not controllable by manual. Therefore, brightness of image depends on weather and time. If sky is clear and bright, ISO sensitivity becomes down and shutter time becomes fast. Ager becomes good, and noise will be decreasing. Therefore, quality of image is good. The other way, cloudy weather is not a good condition. Colour of signal lights goes to white hole. Number of experiment is twice. Courses are different.

Figure 9 shows result of some sample images.

Image shows result for various environments. First row shows signal light detection for usual environment. A signal light far from train has small size in first image. And near signal light has big size. Both of signal lights are detected well. Second rows shows signal light detection in complex environment. There are many buildings and signboards. But this algorithm works well. Third row shows signal light detection with multiple objects. Red signal light has priority by policy. In right image, there are red and green signal lights, together. Green signal light has priority in this case. Because of, green signal light is near the engine. Therefore, this algorithm decides that this frame is detected green signal light.

Figure 10. Computers and equipments built in car and camera.

Detection success rate is with 93.9%. 124 of 132 signal lights are successfully detected. And, green and red lights are detected well. In policy, red light has priority.

Average of run time is 7ms per frame. This time is faster than any other algorithms in reference. Considering Moores law, this algorithm is faster than other algorithms. And also, this algorithm realizes real-time detection.

CONCLUSION:

Technology for unmanned vehicle is rapidly advanced. Peoples are interested in unmanned vehicle. Therefore, signal light detection is important. There are many research and development for unmanned vehicle. Contest has opened, but, it is artificial and limited. Research for real- time tracking was just simulation. AI based research has slow run time. HSI colour map based research has fast run time. But it has low detection rate. In this paper, we presented the new algorithm for signal light detection. This algorithm has high detection rate. Also, red and green signal lights are detected successfully. Low price camera can be used for this algorithm. Condition for weather is not good, but, detectable. And, all of computers and equipments are built in real vehicle. Future work will be implementation on real-time embedded system. Also, shape like X should be detectable. And, detction rate should be increased up to 99%.

REFERENCES:

1. http://www.darpagrandchallenge.com/

2. Sebastian Thrun et al, Stanley: The Robot that Won the DARPA Grand Challenge, Journal of Field Robotics. 2006

3. Massimo Bertozzi and Alberto Broggi, GOLD: A Parallel Real- Time Stereo Vision System for Generic Obstacle and Lane Detection, IEEE Transactions on Image Processing, Vol. 7, No 1,

   Jan 1998

4. Mahipal. R. Yelal et al, Color-based Signal light Tracking in Real- time Video, AVSS 06 IEEE, 2006

5. Yun-Chung Chung et al, A Vision-based Traffic Light Detection System at Intersections, Journal of Taiwan Normal University:

   Mathematics, Science & Technology 47(1), 67-86, 2002

6. Tsinas and Volker Graefe, Real-Time Recognition of Signaling Lights in Road Traffic. MVA 96, 1996

7. J. B. MacQueen, Some Methods for classification and Analysis of Multivariate Observations, Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley,

   University of California Press, 1:281-297 , 1967

8. R.M. Haralick, A Measure for Circularity of Digital Figures, IEEE Transactions on Systems, Man, and Cybernetics, vol. 4, pp. 394-396, 1974.