Moving Object Detection and Tracking in Wide Area Surveillance using Hungarian-Kalman Algorithm

Moving Object Detection and Tracking in Wide Area Surveillance using Hungarian-Kalman Algorithm

1. Komagal*1, B.Yogameena#2, J. Keerthiga*3, R. Manimozhi*4, J. Ragavi*5, Gowthaman Nandhini*6,G. Surya*7

   (1-7)Department of Electronics and Communitcation Engineering

   *(1,3-7)Velammal College of Engineering and Technology ,#2Thiagarajar College of Engineering Madurai,Tamil Nadu, India

   Abstract: Pan-tilt-zoom (PTZ) cameras have increased field of interest in video surveillance due to its wide area coverage and its multifunctionality. Tracking a moving object in wide area surveillance is a crucial issue. Most of the existing tracking algorithms have failed to detect moving object reliably in wide area due to scale changes of camera and prediction of the target position. In this paper, moving object detection and tracking is done by HOG features with Hungarian Algorithm to obtain efficient tracking in wide area. First, extracting the moving object features by HOG (Histograms of Oriented Gradient) to obtain foreground/background is done. Then the segmented output applied to Hungarian Algorithm together with the Kalman filter for tracking. Thus the moving objects are tracked well even under different scale change. Experiments were evaluated and performed in challenging surveillance scenarios and precise real-time tracking with pan, tilt and zoom was achieved.

   Key terms: PTZ camera, video surveillance, object detection, hog feature, object tracking, Hungarian algorithm.

   I.INTRODUCTION

   The focus of most research work in video analysis has been on improving the performance of detection, tracking and recognition techniques in a variety of fields. Most modern wide-area camera networks make wide-ranging use of pan- tilt-zoom (PTZ) cameras. For example, a large airport typically contains hundreds or even thousands of cameras many of which have PTZ capability. In practice, these cameras move along predetermined paths are controlled by an operator using a graphical or joystick interface. However, since such cameras are in constant motion, accumulated errors from indefinite mechanisms, random noise, and influence cycling extract any calibration inabsolute world coordinates useless after many hours of continuous operation.

   Even it has multifunctionality it has challenging issues while the camera is on moving. Hence detection and tracking plays a vital role in video surveillance. Due to camera movement there may be more number of failures like reliable tracking, consistently target of the object, multi-view and FOV get changed. To overcome this problem under wide area is addressed in this paper.

   In this paper, moving object detection by HOG features and tracking is done by Kalman filter with Hungarian Algorithm to obtain efficient tracking in wide area. First, to extract the moving object features by HOG (Histograms of Oriented Gradient) to obtain foreground/background. Then

   the segmented output used for Hungarian Algorithm together with the Kalmanfilter tracking. Thus the moving objects are tracked well even under different scale change.

   The organization of the paper is given below. The next section describes related work. Section III explains the overall methodology. Section IV presents the results and discussions obtained from the algorithms and the authors conclusion and future work.

   1. RELATED WORK

      Traditional methods such as frame differencing, optical flow and background subtraction failed to work in real time and are sensitive to noise and illumination changes that are subject to the aperture problem[2][3].In inter-frame differencing, an object is detected if the inter-frame differences in pixel values exceed a given threshold. This method can robustly estimate the camera motion under dynamic scale changes and the people who are moving in groups or independent can be tracked. This method is efficient and fast, but it suffers two well-known drawbacks: foreground aperture and ghosting [4] caused by frame rate and object speed.The Horn-Schunck algorithm of optical flow method can detect the contour of moving object completely.however, it is computationally complex and sensitive to noise, applying to real-time detection of moving targets is difficult. In the surveillance system proposed [5] the corner feature points are detected to represent vehicles for handling the occlusion. But,it is difficult to detect the corner features when image quality is low. For instance, a strategy is to reduce transmission bandwidth by reducing resolution and environmental changes i.e. heavy rain or snow etc. These issues can affect the accuracy and robustness of object analysis.

      Background subtraction involves of maintaining an up-to- date model of the background and detecting moving objects that deviate from that model. Thus, the main problems are sensitivity to dynamic scene changes and the consequent need for background model adaptation via background maintenance.

      Object detecting and tracking are important in image based surveillance system. For object detection, the different approaches being proposed for surveillance, including feature-based object detection, template-based object Detection and background subtraction [6] or inter-frame difference-based detection. There are also many feature-

      based motion detection methods include Harris corners, color and contours [7], which rely on finding corresponding features in successive frames.

      To track the object, feature extraction is an important task. There are features like SIFT [8], SURF [9], which can able to extract features are robust to occlusion and cluster but its does not work well with lightings changes and blur. For the issues of feature sets for human detection HOG is the normalized Histogram of Oriented Gradient (HOG) descriptors provide excellent performance relative to other existing feature sets. They are computed on a dense grid of uniformly spaced cells and they use overlapping local contrast normalizations for improved performance. This detector uses a simpler architecture with a single detection window that appears to give significantly higher performance on moving object images in which the other detectors use combinations of orientation position histograms leads to complexity [10]

      Tracking algorithms like kalman [11], camshaft [12], optical flow [13], and particlefilter [14] are widely used by more number of authors. Hence there will be failures in tracking due to wider scenes.The contour-based people tracking [15] can track only continuously moving edges and cannot track temporarily stopping objects. A camera based position tracking system (PCTS) [16] for person tracking is used. It is only based on motion detection, failed to compare their features and also failure in different tracking conditions with more than one moving object in a scene. Among model- based tracking algorithms, the color-based Mean Shift algorithm too faces difficulty as when large area intense disturbance from background appears around the object, the object can't escape from it.

      Hence the problem of object tracking is formulated as assignment problem [17] and methods should be able to deal with entering and exiting ( beginning and end of tracking ), false points (due to errors in detection) and missing points (due to occlusions). Hungarian tracker is designed to subpoints (center of mass) in excess of time. This is actual in dealing with streaming video having low frame rate in order to reduce transmission bandwidth. Accuracy is improved with illumination balancing followed by tracking. The object can be tracked if it is intensely disturbed in the background.

      Hence the moving object detection is done by HOG features and kalman with Hungarian Algorithm to obtain the efficient tracking in wide area. This will greatly increase the performance for moving object detection and tracking from video using PTZ cameras. These methods can be useful for real-time applications and works well for the detection of fast moving objects.

   2. METHODOLOGY

      The block diagram shown in Fig. 1 provides an overview of our methodology. Initially,the video sequence are given for input, the feature points are extracted using HOG and then classified as foreground or background features.

      Input video sequence

      Feature extraction by HOG

      Foreground/Background segmentation

      Object tracking by kalman filter with Hungarian algorithm

      Fig. 1.Overall block diagram showing methodology

      the moving object regions are obtained using an integration scheme based on foreground feature points and foreground regions, which are obtained using an image difference scheme. A compensation scheme is to increase the regions of moving objects obtained from a single frame, the motion history of the continuous motion contours obtained from three consecutive frames is used to increase the regions of moving objects. Next, object region refinement is applied to improve moving object regions.

      Finally, moving object tracking is achieved using Hungarian Algorithm together with a Kalman filter based on the center of gravity of a moving object region in the minimum bounding box.

      The rest of the methodology is organized as follow a) Feature Extraction b) The moving object detection c) The moving objects tracking.

      1. Feature Extraction:

         The block diagram shown in Fig.2, the feature points are extracted from the Histogram of Oriented Gradients (HOG) feature. Their implementation divides the image window into insignificant three-dimensional regions called cells. Each cell accrues a local 1-D histogram of gradient information or edge orientations of the pixel values in the cell.

         The histogram entries combine to give a unique illustration for the image. Contrast regulation is also carried out across the cells to give better invariance to illumination changes. In the hog feature points are matching. Then it is classified as belonging to foreground or background features with feature matching.

         Video Sequence

         Find Feature Points

         Feature Points Matching

         Foreground and Background Feature Points

         Fig. 2. Feature Extraction

      2. Object detection:

         The block diagram shown in Fig 3, the feature points are extracted then the update scheme of foreground feature points is used to increase the reliability of feature point arrangement.

         Foreground Feature Points

         Background Feature Points

         Update Scheme

         Perspective Transform

         	Foreground Regions from Image Difference
         Integration Scheme

         Updated Foreground Feature

         Moving object Regions

         Compensation of object Regions

         The foreground feature points found in the (k – 1) th frame are extracted.They are used to match with the feature points in the kth frame to obtain additional foreground feature points using Equ. (1) and (2).Finally, these additional foreground feature points are merged with the foreground feature points in the kth frame to obtain the updated foreground feature points using Equ. (3). The feature point matching mainly uses the foreground feature points in the (k-

         1) th frame to find the corresponding foreground feature points in the kth frame.

         f( , , 1 ) = ( + + ) (1) F (1) = {(, , 1)| (, , 1) 1} (2)

         1 = (1) (3)

         Before image difference using Equ. (4) is applied to two consecutive frames to obtain foreground regions, the perspective transform matrix is built to estimate the motion of the background established on the obtained background feature points. When the figure of the obtained background feature points is larger than the number of the obtained foreground feature points, the perspective transform is applied to the (k- 1)th frame to obtain a transformed (k -1)th frame. If the number of the obtained background feature points is smaller than the number of the attained foreground feature points, the perspective transform is not realistic and the image difference scheme is directly applied to the (k-1)th frame and the kth frame using Equ. (5) To obtain foreground regions.

         Fig 4.Foreground regions obtained using image difference scheme: (a) the kth frame, (b) the transformed (k -1) th frame, and (c) obtained foreground regions.

         Object Region Refinement

         Fig 3. Object detection

         1(, , ) = A

         {

         = 255, | (, , ) (, , 1,)| > 0,

         A = 1(, , ) (4)

      3. Object Tracking:

The block diagram shown in Fig. 5,The detected object using hog feature then move to the tracking scheme that uses a Kalman filter [17] together with the Hungarian Algorithm based on the center of gravity of a moving object region in a

= 1

1

(5)

minimum bounding box is proposed. The Hungarian

=1

In the perspective transform, feature matching is primary used to obtain equivalent background feature points between the (k-1) th frame and the kth frame. Next, the corresponding background feature points between the (k-1) th frame and the kth frame are used to obtain a homography matrix using Equ. (13). finally, the image difference scheme is applied to the kth frame and the transformed (k -1) th frame to obtain regions of moving objects. The perspective transform with the obtained homography matrix is applied to the (k-1)th frame for obtaining the transformed (k-1)th frame using Equ. (6)-(11). Finally, the regions of moving objects are obtained using image difference applied to the kth frame and the transformed (k-1)th frame. Because the video sequence was captured by a moving camera, the transformed (k-1)th frame is affected by the ego-motion of the moving camera. Therefore, ego-motion compensation is proposed to obtain more reliable results using the image difference scheme.

Hungarian Algorithm with Kalman Filter

, = And = 1

algorithm [19] is a classical method for the assignment problem such as the 2-D matching of assigning persons to tasks. The classical 2-D Hungarian algorithm is generalized to an n-D Hungarian and applied a 3-D Hungarian algorithm to solve the tracking problem. The additional tracking points increase accuracy. Its development displaces the dominant use of the more computational intensive Kalman filtering and the matching of point triples leads to increased accuracy. To formulate the problem, with the foreground successfully detected we calculate the center of mass using Equ.(12), where p(x,y) is a point in the moving object region in the minimum bounding box and n is the total number of points in the moving object region to serve as a representative of each object/vehicle detected.

Calculate Information of Object Regions

= 1 (6)

Output Obtained

=

xa w,

(7)

p1 p2 p3

Fig 5. Object Tracking

y , , b

1

(, )

(12)

pa = [ a] , pb = [w yb] , Hab = [p1 p2 p3] (8)

=1

1 w,

Where 33 = 1 ,

p1 p2 p3

The search region is defined in Eq (13), Where and are width and height of the minimum bounding box of the moving objective region respectively. Moving objects are searched for within the search region. , where = = 2

11 12 13

31 32 33 1

are set from eperience.

= + (13)

Approaches [20] should be able to deal with entering and exiting (initiation and termination of tracking), false points

x, = x = p1 x+p2 y+p3 2 p1 x+p2 y+p3

(10)

(due to errors in detection) and missing points (due to occlusions). Given a sequence of n frames denoted by Ft1; Ft2 ; . . . ; Ftn . Each frame, Ftk, has a set of points (centers of mass).

y, = y = p1 x+p2 y+p3

(11)

1. RESULTS AND DISCUSSION

   2 p1 x+p2 y+p3

   Next, the obtained foreground regions and updated foreground feature points are integrated to obtain regions of moving objects. Every updated foreground feature point is checked to determine whether it is located in the foreground component; if so, then this foreground component is assigned as a region of moving objects; otherwise, it is not a region of moving objects. Next, object region refinement is applied to improve moving object regions. Finally, the minimum bounding boxes are applied to mark the moving object for moving object detection.

   The algorithms were implemented in Benchmark. A self made dataset that includes two video sequences (car,human). We used to evaluate the performance of moving object detection.

   =

   +

   =

   +

   (16)

   (17)

   = 2 ( × ) (18)

   +

   Fig 6. Existing work based on Kalman Filter

   Fig 6. show the detection and tracking results obtained using the previous method. Fig 7. show the detection and tracking results obtained using proposed method. In order to illustrate the performance of moving object detection the true detection rate TR and the false detection rate FR were adopted as defined in Equ.(14) and (15) respectively, where N is the total number of moving object, TP is the total number of true detection objects and FN is the total number of false detection objects.

   = ( ) × 100 % (14)

   Table (1). Shows the result of the performance evalution for the self made dataset.

   = (

   +

   ) × 100 % (15)

   For tracking scheme based on two model. The one of the model of motion using position, velocity or relative speed and acceleration of an object.

   Estimation process and correction will be managed by kalman filter also predicts the same parameter of each moving object on the field.

   Methods	PSNR	MSE	F	P	R
   Benchmark 1 (INPUT 1)
   Existing work	4.8262	2.1402e	0.4772	0.9682	0.9714
   (kalman)		04
   Proposed work	4.8266	2.1400e	0.4872	0.9689	0.9696
   (Hungarian-		04
   kalman)
   Benchmark 2 (INPUT 2)
   Existing work	6.0495	1.614e0	0.7100	0.9964	0.9571
   (kalman)		4
   Proposed work	6.0499	1.609e0	0.7122	0.9969	0.9596
   (Hungarian-		4
   kalman)

   F R F

   MSE PSNR

   0 5 10 15

   BASE WORK PROPOSED WORK

   Fig 8.Bar chart (%) of performance evaluation for the input dataset.

   Fig 7. Proposed work based on Hungarian-kalman algorithm.

   The another model of motion will maintain the identity of each object on field. The identity of each object will be defined in the first frame and for the next frame, the module will handle it automatically. To implement this module will be used Hungarian algorithm for assignment problem.

   In order to illustrate the performance of datasets, Precision (P), Recall (R) and F- measure (F) were adopted as defined in Equ.(16)-(18)

   In above bar chart shows the proposed method greatly increases the performance of our previous work.

2. CONCLUSION

Most of the existing tracking algorithms are failed to detect moving object reliably in wide area due to scale changes of camera and prediction of the target position. In this paper, moving object detection and tracking is done by HOG features with Hungarian Algorithm to obtain efficient tracking in wide area.. The proposed method works well for tracking overlapping objects and even under different scale changes. Experimental results have demonstrated that this model would be a feasible solution and make tracking accurately.

REFERENCES

1. Lubos Ovsenik, Anna Kazimirova, Jan Turan (2010) Video Surveillance Systems. Acta Electrotechnica et Informatica. Vol. 10, No.4, 46-53

2. S.-C. S. Cheung and C. Kamath (2004) Robust techniques for background subtraction in urban traffic video. in Proc. EI-

   VCIP. pp. 881892

3. S. Elhabian, K. El Sayed, and S. Ahmed Jan (2008) Moving object detection in spatial domain using background removal techniques: Stateof- art. Recent Patents Comput. Sci., vol. 1, no. 1, pp. 3254

4. R.T. Collins, A.J. Lipton, T. Kanade, et al., A system for video surveillance and monitoring, Technical Report, Carnegie Mellon University, 2000.

5. B. Coifman, D. Beymer, P. McLauchlan, J. Malik (1998) A real-time computer vision system for vehicle tracking and traffic surveillance. Transport. Res. PartC: Emer. Technol. 6 (4), 271288.

6. Background subtraction by combining Temporal andSpatio- Temporal histograms in the presence of camera movement Andrea Romanoni · Matteo Matteucci ·Domenico G. Sorrenti

7. C.G. Harris, M. Stephens, A combined corner and edge detector, FourthAlvey Vision Conference, 1988, pp. 147151.

8. D. G. Lowe (2004) Distinctive image feature from scale- invariant keypoints. Int. J. Comput. Vis., Vol. 60, no. 2, pp. 91- 110

9. T. Yusufu, Y. Wang and X. Fang Dec (2013) A Video text detection and tracking system, in proc. IEEE Int. Symp.

   Multimedia (ISM), pp. 522-529

10. Navneet Dalal and Bill Triggs (2005) Histogram of oriented Gradients for Human detection, In proceedings of the IEEE Computer Society Conference Vision and Pattern Recognition (CVPR05)

11. Ning Liu, Hefeng Wu and Liang Lin Jan (2015) Hierarchical Ensemble of Background Models for PTZ- Based Video Surveillance. In the proceedings for IEEE Transaction on Cybernetics, Vol. 45, No. 1

12. Jiude Li June (2015) Research on Camera based Human Body Tracking using Improved Camshift algorithm. In the proceedings for International Journal on Smart Sensing and Intelligent System, Vol. 8, No. 2

13. M. J. Lucena, J.M. Fuertes, J.I. Gomez, N.P. de la Blanca and

    1. Garrido sep (2003) Tracking from optical flow. Proceedings of the 3rd International Symposium, doi: 10.1109/ISPA.1296357

14. P. D. Z. Varcheie and G. A. Bilodeau Feb (2011) Adaptive fuzzy partical filter tracker for a PTZ camera in an IP surveillance System . In IEEE Trans. Instrum. Meas., Vol. 60, no.2, pp.354-371

15. D. Murray and A. Basu, Motion tracking with an active camera, IEEETransactions on Pattern Analysis and Machine Intelligence, vol. 14, pp. 449459, 1994.

16. A. Yilmaz, X. Li, and M. Shah, Contour-based object tracking with occlusion handling in video acquired using mobile cameras, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26,no. 11, pp. 15311536, 2004

17. Kuhn, H.W., 1955. The Hungarian method for the assignment problem.Naval Res. Logis. Quart. 2, 8397

18. R.E. Kalman, A new approach to linear filtering and prediction problems, Transact. ASME J. Basic Eng. 82 (1) (1960) 35 45.

H. W. Kuhn (1955) The Hungarian Method for the Assignment Problem. In the proceedings of Naval Research Logistics Quarterly 2., 83-97 Yiwen Wan, Yan Huang, Bill Buckles Camera calibration and vehicle tracking: Highway traffic video Analytics