Vehicle Detection using Thermal Sensors

Vehicle Detection using Thermal Sensors

Sangeetha N

1. ech VLSI & Embedded Systems Dept. of Electronics and Communication

   ER & DCI IT

   Thiruvananthapuram, India

   Kadar A A

   Principal Engineer Dept. of Electronics and Communication

   ER & DCI IT

   Thiruvananthapuram, India

   Sathyanarayan K Principal Engineer Intelligent Transportation & Networking Section

   CDAC

   Thiruvananthapuram, India

   Abstract Intelligent Transportation Systems (ITS) incorporates advanced technologies into different levels of transportation management to increase traffic efficiency and vehicle detection plays a major role in it. Advanced traffic management subsystems in ITS, such as vehicle actuated traffic control and parking management increasingly rely on vehicle data that reflects real-time traffic conditions. As data measured and collected by vehicle detection systems must be plentiful, diverse, and accurate, these complex data requirements present a challenge to traffic detection systems. Presently, there are two primary categories of detector technologies: intrusive and non- intrusive. Though intrusive detector technologies, such as inductive loops and magnetometers, have been used widely in transportation fields, they have their own disadvantages of higher installation cost and disruption of traffic while installation or maintenance. As the cost of maintenance and installation of these detectors can become uneven, non-intrusive methods like radar, laser, ultrasonic, acoustic, and infrared and video based methods are gaining popularity.

   By deploying non-intrusive detectors, the higher initial cost can offset the costs associated with installation and maintenance of intrusive detectors. Nevertheless, they are not without drawbacks and challenges. Radar based sensor suffers due to unwanted vehicle detection based on reception of side lobe radiation, false detection due to multi-path and limited coverage area. Ultrasonic, acoustic, and infrared sensors suffer performance deterioration in noisy environments; the video and laser sensors are usually vulnerable to weather and light conditions. Many researches on vehicle detection using visible- light cameras have already been proposed. However, the visible- light camera may not be able to capture clear images in night time or places where there is insufficient light as they are sensitive to issues such as the time of day, reflection from headlights, swinging tree leaves, fog, rain and shadow. Since thermal sensors see heat instead of light and thermal imaging can improve the robustness and performance over standard visible light systems and address the issues faced by the visible light cameras, vehicle detection method using thermal imaging sensor is used to realize road traffic controlling under different environmental conditions. In this project, background subtraction and image segmentation based on morphological transformation for detecting vehicles from the live thermal video is proposed. Proposed algorithm segments the image by preserving important edges which improves the performance. Future work will include counting and classification of vehicles.

   KeywordsTraffic; Vehicle detection; Thermal Sensors; Visible light camera; Image processing

   1. INTRODUCTION

      With the increase in number of vehicles, traffic congestion and traffic accidents have become a major challenging problem in our life. Every second, at least one person dies

      due to vehicle accidents. To minimize the traffic accidents, traffic control using modern technologies are essential. Modern technologies such as Intelligent Transportation systems (ITS) have gained popularity and one of the reliable approaches for controlling traffic is vehicle detection.

      During the last decades, vision systems have been rapidly growing. More recently, the vision-based video cameras have gained attention and they are becoming more common in vehicle detection. These systems detect vehicle based on data from video cameras. Video cameras are increasingly deployed used for traffic surveillance application to monitor traffic on major roadways. Hence, the effective utilization of these cameras for data collection is of practical signicance [6]. Vehicle detection under different environmental conditions is a very challenging problem in video processing applications. Based on the movements of vehicles, videos can be categorized into two types: moving vehicles and stationary vehicles. Moving vehicles are detected from video frames by background subtraction. There are various ranges of vehicle detection methods which will be discussed in next section, Literature review.

      Vision-based monitoring systems, capturing visible light in grey scale or RGB images, have been the standard imaging device and can provide rich positional data beyond the capabilities of traditional devices. There are certain limitations associated with the use of visible video cameras. As these cameras rely on the visible light spectrum, the accuracy of detection is sensitive to environmental factors such as day light time, low light condition, shadow, cobwebs and other weather conditions and it degrades the accuracy of extracted data. This is particularly problematic because the increased injury risk associated with night time conditions leads to severe road accidents. This is why visible based sensors may not work under all conditions. To overcome some of these limitations, other sensors have been introduced in vision systems. The performance of thermal cameras is efficient when compared to visible sensors across varied lighting and visibility conditions. In the mid- and long wavelength infrared spectrum (314 m), radiation emitted by the objects themselves and thereby they do not depend on any external energy source. Thermal cameras utilize this property and measure the radiation in parts of this spectrum.

      This paper describes a solution to road traffic problems in cities through the design and implementation of a vehicle detection system using thermal sensor and it is designed using the Scilab simulation software. The rest of the paper is organized as follows: The first section provided a brief introduction to traffic in general and also a brief review is

      given about the project. The literature review is discussed in section 2. In section 3, the vehicle detection approaches are illustrated. Various vehicle detection technologies are presented in section 4. The system design is given in section 5 and finally conclusions are summed up in section 6. While section five describes the system design development, section four presents the results on road traffic systems and section five concludes the work.

   2. LITERATURE REVIEW

      Moving object detection methods can be divided into three main types: temporal difference, optical flow, and background subtraction.

      In [1] referenced by Lipton A.J and et.al, the images are taken from a video sensor and the absolute value is considered here which is obtained by the difference of two successive images in a pixel-by-pixel manner. If the absolute value is greater than the threshold value, then the pixel is regarded as a foreground point, else, it is a background point. Although this method is quick and simple, if the color difference of background and foreground are too similar, the continuous images may be too small to detect. When there is a slow moving objector when the moving object is temporarily stopped, the objects positions in the continuous images do not change significantly; so, there is a chance that the object is likely to be considered as background. So, this causes an incomlete detection of the moving object.

      In [2] referenced by Zinbi Y, the moving objects are detected by analyzing the optical flow change in a video-scene. This method was not able to obtain accurate result because of the indistinct features of the object or due to the movement detection due to noise. To get accurate optical information, a high frame rate is required which leads to high hardware costs. Since, this method is very complex, this method is used not very frequently. One of the hardest challenges in creating efficient and accurate Vehicle Detection Systems is handling varying lighting conditions. Most daytime methods of detection lose their accuracy when applied to night-time detection. Due to this, this system runs two separate algorithms for daytime and night-time detection. In [3], two different approaches of image analysis algorithms have been discussed for detecting vehicle in traffic scenes based on the illumination conditions of day and night time. In daytime images, spatio-temporal analysis is performed on moving templates, while, in night-time images, morphological analysis is performed on headlight pairs of the vehicle. The headlights of motor bikes are ignored since they dont make a huge contribution to the traffic congestion. This method also results in a few false positives due to shadow scenes and overlapped objects.

      In [4] referenced, vehicle detection and vehicle counting is presented in this paper. Identifying objects from a video is a critical task in vehicle detection. The system takes images from visible camera installed at roads in a particular time interval to determine the traffic congestion in the road. Here, the system uses Background subtraction and threshold for detection of vehicles. The image is pre- processed through Median filter, Binary Erosion and Binary Dilation to remove noises and identify moving vehicles from background. The system collects the traffic congestion data is plotted in a web server map. Users can view the information and find an alternate path to their destination. Another work proposed by [5] discusses about the background subtraction technique and filtering method to detect vehicles in complex traffic scenes. Here, the filtering method uses a Histogram which collects information from image sequences. This method has a good performance even when the camera is shaken, traffic congestion and light changes. In [6], a method has been proposed for detecting moving vehicles based on the filtering of swinging trees and raindrops. It discusses about using an adaptive background subtraction which is combined with a shade removal approach to separate out the moving vehicles in background images. The luminance variation, weather changes and image noise are removed from foreground objects to reduce the computational complexity. But, the background calculation and updating process is computationally expensive and does not take sudden changes in the illumination conditions into the account and suffer from problems such as ghost of moving objects and shadowing effects. Some cases may lead to the erroneous recognition of cars or scooters due to the falsely recognized swinging trees. The ideas from these papers paved way to the study of vehicle detection using thermal sensor.

   3. VEHICLE DETECTION TECHNOLOGIES

      With the introduction of visible and thermal cameras, these sensors can be used to detect vehicles more effectively.

      1. Inductive Loop Sensor

         Inductive loop sensors measure the change in the inductance of the loop, as vehicles move near or over the loop, installed in the road surface. When a vehicle with conductive metal passes over the loop, the inductance is reduced, thereby increasing the frequency of the oscillator. The higher frequency is registered by the detector oscillator, and the vehicles presence is registered. They are of low cost, consume low power and measure high accuracy. But their detection measure is poor for small vehicles, damaged by road deterioration or heavy vehicles, major disruption to traffic during installation, sensitivity to temperature fluctuations, affected by metallic road construction materials and high risk of the loop and feeder cable theft. Extreme high and low temperatures affect the operation of Inductive loops, causing them to miss vehicles or generate false detections.

      2. Video Sensor

         Video detection systems analyses the image, looking for changes in the pixel values to determine if there is a vehicle to be detected. They are quick to install, very reliable, video monitoring, flexible setup, optional uni-directional operation and no disruption to traffic during installation. But, the performance of Video Detection systems might be degraded in bad weather or low light conditions. Shadows, vehicle projection into adjacent lanes, day-to-night transition, water droplets and cobwebs on the camera lens can affect performance. Heavy rain can cause video detection systems to miss vehicles, when the field of view is impaired. They are quick to install, very reliable, video monitoring, flexible setup, optional uni-directional operation and no disruption to traffic during installation. But they are of high initial cost and detection accuracy affected by occlusion. But, they are susceptible to obscure issues, as with other non-intrusive detectors. Performance of VID systems might be degraded in bad weather or low light conditions.

      3. Thermal Sensor

      Thermal imaging systems, on the other hand, could detect and clearly identify any vehicle, in all-weather conditions, any time of the day or night. Thermal sensors are sensitive to far infrared wavelengths and thus detect the heat energy naturally radiated by all objects. As heat energy depends on the composition and temperature of the object, thermal detection can be a consistent and reliable solution in all environmental and lighting conditions such as fog, smoke, daytime, night time, etc. Thermal cameras cannot use the same lens materials as visible light cameras because those materials effectively block all of the infrared energy from getting to the detector. This explains why thermal cameras cant be used to see through glasses or windows. Hence, thermal sensor was chosen as the vehicle detector technology for vehicle detection.

   4. SYSTEM DESIGN

      The block diagram of the vehicle detection is as shown in Figure 4.1. The video frames are collected from the thermal video camera and it is given to Computer via a Web Server. Here, a pre-defined virtual loop is drawn in computer with the help of Graphics Interface Unit (GUI). These video frames are processed through certain algorithms to detect the vehicle.

      Fig 4.1 Block Diagram of the Vehicle Detection

      1. Image Acquisition

         The live video frames are captured through the thermal camera and is sent to the Computer via Web server. After the image has been acquired, the video frame is pre-processed with image processing algorithms. The ultimate goal is to detect a vehicle which is one of the challenging tasks in image processing.

      2. Background Subtraction

         Background subtraction is one of the major task used for detecting vehicles in videos from static cameras. This method is also known as foreground detection since it detects foreground objects from the background. A computer, unlike humans, is unable to distinguish background from the foreground which consists of vehicles by considering a single image. More the number of frames, more the quality and more time is consumed. This technique helps in detecting changes in video sequences. The moving pixels obtained from the difference between the current frame and the reference frame is called background image and the common features are removed from previous and current frame. The flowchart for the algorithm is shown below in Figure 4.2.

         Fig 4.2 Background Subtraction Algorithm

      3. Morphological Operations and Segmentation

         The main idea of Segmentation is to identify a connected region in the image, a blob associated wit the target. This method is also known as blob detection or Segmentation. This step plays an important key role in vehicle detection. Feature based segmentation approach is followed where it performs the vehicle detection by extracting and grouping features. The advantage of this approach is that even in the presence of partial occlusion, some of the sub-features of the moving vehicle remain visible. After the stage of segmentation, the Morphological operations will be applied on various frames where the structuring element is used as a template matching for vehicle feature detection.

      4. Virtual Loop Detectors

      An installed thermal imaging camera captures the video frame and sends an input signal to the detection unit via a Web server where detection zones are superimposed onto the video image. These detection zones are known as virtual loop detectors. When a vehicle enters a detection zone, the pixel value within that zone changes. Based on that pixel difference, the vehicles are detected as shown in Figure 4.3.

      Fig 4.3 Virtual Loop Detectors

   5. RESULTS

      Scilab 6.0.2 software is used to extract video frames and are displayed in Graphical Figure. The resultant image is obtained by subtracting the previous frame from the current frame which takes out the common features and is segmented. The images used are captured by using a thermal video camera installed at Manaveeyam veedhi lane, Vellayambalam, Thiruvananthapuram. The algorithm detects the vehicle when the features enters the virtual loop. The loop size determines the detection zone of the loop. Loop length and width depends on the dimension of the target vehicle and on the width of the traffic lane. The below-attached figures, Figure 5.1 to Figure 5.4 shows the vehicle detection in video frames when the vehicles are moving as well as when the vehicle is stationary.

      1. Moving Vehicle Detection

         Fig 5.1 Input image sequences from thermal camera

         Fig 5.2 Moving vehicle Detection

         The moving vehicles are shown in Figure 5.1 where ith frame,i+1th frame and i+2th frame are the image sequences in series. Check whether if any blobs can be extracted from subtracted image and if it comes inside the loop then, moving vehicle is detected and this is indicated by red loop as shown in ith and i+1th frame of Figure 5.2. If the vehicle comes out of the loop then, vehicle is not detected as shown in i+2th frame of Figure 5.2 by turning the loop color to white.

      2. Stationary Vehicle Detection

      When vehicle is stationary as shown in image sequences in Figure 5.3 where ith frame,i+1th frame and i+2th frame are the image sequences in series. (i.e) both input image and reference image are same, the subtracted result does not show

      much pixel differences. Since, the blob area in subtracted image is less than the pre-defined area, check whether any blobs can be extracted from the input image and if it comes inside the loop then, stationary vehicle as shown in ith and i+1th frame of Figure 5.4 is detected and is indicated by red loop color. If the vehicle comes out of the loop then, vehicle is not detected as shown in in i+2th frame of Figure 5.4 by turning the loop color to white.

      Fig 5.3 Input Image sequences from thermal camera

      Fig 5.4 Stationary vehicle detection

   6. CONCLUSION

This paper presents the vehicle detection observed by a thermal camera installed at the traffic junctions. Background subtraction algorithm is developed using open source software, Scilab (version 6.0.2) which provides an effective way of vehicle detection. This paper presents morphological analysis with the use of varying templates size used. The simulation results show that blob analysis is successfully applied. This method can overcome noises that might occur due to motion blur and illumination because thermal camera depends on heat emitted by objects.

REFERENCES

1. A. J. Lipton, H. Fujiyoski, and R. S. Patil, Moving target classification of and tracking from real-time video. Proceedings of IEEE Workshop Applications and Computer Vision, Princeton, NJ, pp. 814, 1998.

2. Zinbi, Youssef & Chahir, Youssef & Elmoataz, Abderrahim, Moving vehicle Segmentation using optical flow with active contour mode, 3rd International Conference on Information and Communication Technologies: From Theory to Applications, Damascus, pp 1-5, 2008.

3. Cucchiara, Rita and Massimo Piccardi. Vehicle Detection under Day and Night Illumination. IIA/SOCO , 1999.

4. Maharjan, Pradip & Shrestha, Ajay, Automatic Vehicle Detection and Road Traffic Congestion Mapping with Image Processing Technique, International Journal of Computer Applications, Volume 114, Issue No.16, March 2015.

5. Nicholas A. Mandellos, Iphigenia Keramitsoglou, Chris T. Kiranoudis, A background subtraction algorithm for detecting and tracking vehicles, Expert Systems with Applications, Volume 38, Issue 3, Pages 1619-1631, 2011.

6. Deng-Yuan Huang, Chao-Ho Chen, Wu-Chih Hu, Sing-Syong Su, Reliable moving vehicle detection based on the filtering of swinging tree leaves and raindrops, Journal of Visual Communication and Image Representation, Volume 23, Issue 4, Pages 648-664, 2012.