Hybrid Approach for Object Detection from Dynamic and Static Backgrounds for Surveillance Systems

Hybrid Approach for Object Detection from Dynamic and Static Backgrounds for Surveillance Systems

Raji priya.R Assoc. Prof Saini Jacob Soman

CSE Department CSE Department

Sree Narayana Gurukulam College Of Engg. Sree Narayana Gurukulam College Of Engg.

Ernakulam Ernakulam

Abstract Object segmentation is the key process in the video analysis to deal with surveillance and security. There are so many algorithms are put forwarded for implementing object segmentation. This paper proposes a hybrid approach for object segmentation in both dynamic and static backgrounds. In this paper the objects from dynamic backgrounds are segmented by using a neural-fuzzy method. The neural stage is based on self organizing map like architecture. The fuzzy stage automatically computes and adjusts the main parameters required for segmentation without human interaction. Also the moving objects from static backgrounds are detected and tracked by using Kalman filter method and morphological operations.

Keywords Object segmentation, Neural-fuzzy approach, Kalman filter.

1. INTRODUCTION

   Now-a-days every city is equipped with surveillance cameras to provide security. Surveillance systems have applications in not only in military or security environments, but also for industrial applications, traffic analysis and in child care applications. In most of the cases the video footage analysis is the responsibility of human. Object segmentation is the important process in video analysis. The security analysts need to inspect different video footages from different surveillance cameras to provide security and to identify abnormal events, abandoned objects and traffic violations.

   The video analysis task in most of the surveillance systems is the responsibility of concerned human. So the performance of the system mainly depends on the performance of the security analyst. For example, if a surveillance system has 100 cameras and 3 operators, only 3% of screens can be monitored at a time [1]. The solution for this problem is providing some sort of intelligence to system. That is intelligent information system (ISS). By providing intelligence the surveillance system can detect abandoned objects [2], abnormal activities and area violations.

   The prerequisite for implementing ISS is object segmentation. Detection of static and dynamic objects from video sequence is the most important process in video analysis. Several segmentation techniques have been proposed in literature. The segmentation techniques are classified into background subtraction method, edge detection method, optical flow method and temporal differencing

   method. Background subtraction method is the basic and simple segmentation technique in which the fore ground objects is separated from background objects from video frame. In this method the difference between the current frame and the reference frame (background frame) is taken and apply predefined threshold for segmenting the objects [3]. The performance of this method depends on threshold values. In Edge detection method [4] the edge of objects are taken in account for tracking and recognizing different objects in video frames. Both canny edge detector and sobel edge detector can be used for object segmentation. Optical flow method [5] is mainly used for non-stationary cameras, that is, camera is also moving. In this method, each pixel is assigned with a 2D velocity vector over each video frames. According to the characteristics of velocity vector the moving objects are detected. Temporal differencing method [6] is based on the temporal and spatial features of the sequences. The goal of this procedure is to divide the image sequence into scenes or shots mainly based on the changes in the appearance.

   This paper proposes a hybrid approach for object detection from dynamic and static backgrounds. An adaptive neural-fuzzy method [7] is used for object detection from dynamic background. The neural stage is based on SOM like architecture [8] and the fuzzy inference system will mimics human behavior for adjusting the threshold for segmentation for different video frames without human intervention. The moving objects from static backgrounds are detected and tracked by using Kalman filtering method and morphological operations.

2. OBJECT DETECTION FROM DYNAMIC BACKGROUNDS

   Surveillance systems are designed to work in both static and dynamic environments. The different objects from dynamic backgrounds are detected by using a neural-fuzzy method. The overview of this method is shown in Fig 1.

   In this approach the neural stage utilizes threshold value computed automatically in the fuzzy stage for segmentation. Both Mamdani and Sugeno fuzzy inference systems are used here.

   A video sequence is given as the input of the system. This video sequence consists of nine frames each of which

   represents different environmental scenarios. The video sample used in our research is shown in Fig 2. Each frame has different illumination and saturation. For each frame the threshold will be different. The fuzzy inference system (FIS) will automatically computes the threshold values according to the illumination and saturation of each frame. This threshold is utilized by the self organizing map (SOM) for detection purpose.

   Fig.1. Neural-fuzzy approach flow chart

   Fig.2. Video used in Neural-fuzzy method

   1. Dynamic background modeling method

      The neural stage is based on SOM like architecture. The proposed method consists of two stages: Initial Background selection, Dynamic Background Update.

      1. Initial Background selection

         In this step, define or select a reference background frame for comparison and later it is updated. The criterion for the selection is that the frame should contain most of the static objects.

      2. Dynamic Background Updation

         This step is based on SOM like architecture. SOM is based on unsupervised learning. That is these network will form classifications from the training data without any external help. Here each video color pixel is mapped into a SOM element. There is one to one mapping between the pixel and neuron. So that the number of parameters required for

         tune will be less. Each pixel is represented by HSV color space so each neuron has three inputs.

         The dynamic update algorithm is as follows,

         Step 1: Initialization: Initial weights of each neuron are assumed to be equal to the corresponding color pixel values of the initial background.

         Step 2: Winner finding: Find the winner neuron by using the Euclidean distance criteria. The criterion is that select a neuron with largest Euclidean distance.

         Step 3: If the Euclidean distance between the pixel and associate neuron is greater than the threshold then the weight of the pixel need to be updated.

         Step 4: Update the weights of neighboring neurons.

   2. Automatic fuzzy threshold computation method

      The segmentation threshold for SOM detection model is computed by using fuzzy inference system. For each frame of the video input has different illumination and saturation. So for each frame the threshold will be different. The FIS [9] will mimics human adjustment of segmentation threshold by this method.

      1. Fuzzy Variable Description

         This FIS has two inputs illumination I and saturation S and one output threshold value. The membership function of each variable is designed according to the analysis of different video scenes. The range of illumination variable is 0.26<=I<=0.64 and saturation variable is 0.12<=S<=0.59. The range of thrshold is between 0 and 1. Triangular membership function is used in this FIS, Fig 3, 4,

         1. The fuzzy values for the inputs and output are low, medium and high.

            Fig.3. Membership function of saturation

            Fig.4. Membership function of illumination

            Fig.5. Membersehip function of output (Threshold)

      2. Fuzzy Rules

   The FIS will act according to the rules defined in the rule base. Mamdani system is simple and there are eight rules are defined for Mamdani systems. The rules are described as follows,

   • If S is low and I is low then output is low

   • If S is medium and I is medium then output is medium.

   • If S is high and I is high then output is high.

   • If S is medium and I is high then output is medium.

   • If S is low and I is medium then output is high.

   • If S is high and I is medium then output is low.

   • If S is low and I is high then output is high.

   • If S is low and I is high then output is low.

   The Sugeno system is developed by converting Mamdani into Sugeno system. Sugeno system uses weighted average to compute the output.

3. MOVING OBJECT DETECTION FROM STATIC BACKGROUND

   Moving object detection and tracking is main aspect many computer vision based applications. It also finds applications in the field of surveillance, traffic pattern recognition, human detection system and medical image processing.

   This paper deals with moving object detection and tracking by a method called Kalman filtering [10]. Fig 6 depicts the flow chart of the proposed method.

   The method takes the video input in which background is static and some of the objects are in motion. Multiple objects can detect and tracked by this method.

   1. Foreground Detection

      The moving objects from the video are segmented by a background subtraction method based on Gaussian mixture models. Morphological operations like erosion, dilation, closing and opening are applied to the resulting foreground mask to reduce noise.

   2. Blob Analysis

      Blob analysis is performed to group connected pixels of the moving objects. The association of pixels into same group is based on motion of the objects. Here the properties of blob can be adjusted according to scenarios.

   3. Kalman Filtering

   The motion of each object is estimated by Kaman filter. Kalman filter is used to predict the tracks location in each and every frame and maintains its track throughout the video. Multiple moving objects can be segmented and tracked by this method.

   Track maintenance is an important step in this method. In each frame some of the detections are assigned to some tracks. Update the assigned track whenever the detections are moving. Each track maintains the count of the frame in which the object detected is unassigned. If the count exceeds predefined threshold the corresponding track is deleted with the assumption that the object left the field.

   Fig.6. Moving object detection via Kalman filter

4. RESULTS

This section presents the segmentation results of the methods discussed. Fig.7 shows Objects of interests are segmented from ninth frame by using neural-fuzzy method. The figure on the upper left corner is the original frame. The figure on upper right corner represents the output of normal background subtraction method. The figure on below left corner gives the output of SOM segmentation and last figure represents the final output of neural-fuzzy method. From the figure it is easy to identify that neural-fuzzy approach will provide better result than the normal background subtraction.

Fig.7. Segmentation of frame 9

Fig.8. Surface generated for threshold in Mamdani and Sugeno system

Fig 8 shows the surface generated for the output segmentation threshold in Mamdani and Sugeno fuzzy system. The segmentation threshold will be different for each of these systems. Because the threshold calculated by different ways in Mamdani and Sugeno system.

Fig.9. (a) original frame (b) segmented moving object

Fig 9(a) and (b) shows the moving objects detected from static backgrounds by using Kalman filter method. Kalman filtering and morphological operations are used for segmenting the moving object from static background video.

REFERENCES

1. E. Wallace and C. Diffley, CCTV control room ergonomics, Police Sci. Develop. Branch (PSDB), U.K. Home Office, Hertfordshire, U.K., Tech. Rep. 14/98, 1988.

2. M. Bhargava, C.-C. Chen, M. S. Ryoo, and J. K. Aggarwal, Detection of abandoned objects in crowded environments, in Proc. IEEE Conf.

   Adv. Video Signal Based Surveillance, 2007, pp. 271276

3. Alan M. McIvor, Background Subtraction Techniques.

4. Canny edge detection method, marcp009,http://www.cse.iitd.ernet.in/~pkalra/csl783/canny.pdf

5. D.J. Fleet & A.D. Jepson, Optical flow estimation.

6. John Y. A. Wangy and Edward H. Adelsonz, Spatio-Temporal Segmentation of Video Data, Proceedings of the SPIE: Image and Video Processing II, vol. 2182, San Jose, February 1994.

7. M.I…C Murguia and S.G. Duarte An Adaptive Neural-fuzzy Approach For Object Detection In Dynamic Backgrounds For Surveillance Systems, IEEE transactions on industrial electronics, vol. 59, no. 8, august 2012.

8. L. Maddalena and A. Petrosino, A self-organization approach to background subtraction for visual surveillance applications, IEEE Trans. Image Process., vol. 17, no. 7, pp. 11681177, Jul. 2008.

9. S. Zhang, C. Zhu, J. K. O. Sin, Fuzzy Logic Fundamentals, Page 62 ,

   March 26, 2001

10. T. Zhao and R. Nevatia, Tracking multiple humans in complex situations, IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 12081221, 2004.