Detecting Crowded Zones as an Indicator for Abnormal Events

Detecting Crowded Zones as an Indicator for Abnormal Events

Ismail El Sayad Department of Computer and Communication Engineering

Lebanese International University Beirut, Lebanon

Samih Abdul-Nabi Department of Computer and Communication Engineering

Lebanese International University

Beirut, Lebanon

Abir Youness Department of Computer and Communication Engineering

Lebanese International University Beirut, Lebanon

Hussein Kassem

Department of electrical and electronic Engineering (EENG) Lebanese International University Beirut, Lebanon

Zahraa Kain Department of Computer and Communication Engineering

Lebanese International University

Beirut, Lebanon

Abstract This paper presents a distinct video surveillance system which took place in the Lebanese International University Saida- Campus, which is considered as a very crowded environment, and reveals if there is an unusual event. Our main target is to apply simple procedures that will be present as a futures benchmark. The work is split into three major parts, starting by dividing the video frame into zones, then to compute the magnitude of optical flow in each, and finally to analyze these data and classify it, based on a logical threshold, as normal or abnormal events. We implement our results based on Histogram of Magnitudes for each zone (HOM) and the outcome met our expectations.

Keywords HOM: Anomalous Event Detection, Video surveillance, video analysis, Optical flow, motion analysis Histogram of Magnitudes

1. INTRODUCTION

   Over time and space, technology is widely spreading across the world. It is used to facilitate our living skills in our daily life. Technology has been around surveillance systems for decades. In recent years, video- surveillance systems become a main interest in peoples life such as government agencies, business, and private possessions. Nowadays, people seek for better image quality, less in cost, better in size and scalability. For safety issues, cameras can monitor real-time occurrences, collect data, and come out with analyzing the behavior of people. Monitoring is often performed through consecutive frames which are extracted from the video.

   Figure 1: Employee monitoring the screens to detect any abnormal movement.

   To take advantage on video surveillance, it is an essential thing to propose an algorithm that is simple and fast to detect human activities [1] [2]. Its an approach that combines the needs of the market and the simplicity. The world now is shifting towards mechanization due to the workers load. Moreover, the world is suffering from deficient in safety issues. So why not inventing a new approach that meets these market expectations? It doesnt need to be complex but a simple one.

   Javanbakhti, S., and S. Zinger [3] propose Fast Abnormal Event Detection from Video Surveillance. Their approach of this page is based on block algorithm by analyzing the pixel wise motion context. This procedure indicates abnormal motion variation in regions based on the entropy (systems thermal energy) of Discrete Cosine Transform coefficients.

   They design a general framework based on features directly extracted from motion such as velocity at pixel level based optical flow.

   1. Analyze information collected in frequency domain in each frame.

   2. Compute DCT coefficients for motion magnitudes

   3. Analyzing and recognizing the motion.

   4. Describe the actions and compare the entropies for each block to a given median average values over time.

   As we mention above their approach is somehow fast since it block-based computing and parallel processing. It exactly indicate in which part of frame the event occurs. However, It suffer from a noisy optical flow, limitation in the usage of median filter, and complexity in calculation.

   Andrade et.al [4] observe the crowd optical flow in order to characterize crowd behavior, and they use unsupervised feature extraction to encode normal behavior. In this article, the analysis in based on optical flow, the feature extraction applies HMM (Hidden Markov Model) and spectral clustering. The detection of abnormal events is based on a threshold on the HMM likelihood function. Their approach Filtering reduce to noise of optical flow, spectral clustering is elegant and well-founded mathematically, and it works well when relations are approximately transitive (similarity). However, HMMs fail to represent appliance using a continuously power demand. In spectral clustering, very noisy datasets cause problems and it is expensive for very large datasets.

   Based on the above we suggest a sufficient algorithm that deals with analyzing the content of a video to classify events between normal and abnormal ones, using a simple software algorithm, based on optical flow calculations. We have chosen our university a place to test our results in. Due to the instability of the secure conditions that surrounds the Lebanese International University (Saida Campus) a shown in Figure 2, we aim to solve this issue in order to be aware of anything that could happen to provide the safety of our friends and the students.

   Figure 2: The studied place The Lebanese International University-Saida Campus

2. PROPOSED MODEL

   Our proposed scenario is based on several steps. The below diagram in Figure 3 describes our algorithm.

   Figure 3: A detailed block diagram of proposed model

   1. Dealing with the video

      Video Surveillance Systems have become a major interest in our daily life. It can monitor a specific location for specific targets and mainly to achieve security. We have chosen our university as a place to examine our algorithm and come out with the required results. We have recorded a video in an attractive spot where students usually walk and meet together as shown in Figure 4. We get the full video information to characterize the mobility in this area. As its known, there is a trade-off between the size of the video and its quality. In order to make the size of the video tolerable with simple calculations and with a low power demand, we decreased the quality of the video to handle the load video information in an acceptable manner.

      Figure 4: Snapshot for the video recorded at our university with a grid

   2. Optical Flow in the Zones

      A video is a set of successive image frames. These images are represented as I(x,y,t) where x and y are the pixel position and t represents the time. If an object moves through these frames, this motion results in a time-dependent displacement of the grey values in image sequence. The 2-d apparent motion field in image domain is called optical flow domain. Optical flow [5] works on different assumptions. It works either on the stability of pixel intensities between consecutive frames or on neighborhood-pixels that have similar motion.

      Calculating optical flow is helpful in analyzing human behavior by modeling the global motion information we get through this estimation. Estimating the optical flow vectors can be done through calculating the magnitude and orientation for each cell. This way will ease the differentiation process of the interested objects between the successive frames. Hence, we can detect if this group of people is running, splitting, or normally going to their classes. Optical flow estimation is a challenging task in image analysis and computer-vision. Estimating optical flow vectors is used in video surveillance systems. First, we extract the point of interest in each frame. Then we track these points from frame to frame in order t detect whether these points are merging away from their normal values. The result is a set of four-dimensional vectors V.

      (1)

      where Xi and Yi are the coordinates of feature i Ai is the motion direction of feature i

      Mi is the motion magnitude of feature i.

      It works in with the distance between features i in frame t and its symmetric feature in frame t+1.

      Many pre-existing procedures that work with optical flow suffer from noisy data. Actually, optical flow is noisy. For accurate results, we used a better version in optical flow that handles the noise -Lucas Kanade -It expects that the flow is basically constant in a local neighborhood of the pixel under thought, and explains the essential optical flow conditions for every one of the pixels in that area, by the minimum squares basis [6].

      By joining data from a few close-by pixels, the Lucas Kanade technique can frequently resolve the intrinsic equivocalness of the optical flow condition. It is likewise less sensitive to image noise than point-wise techniques. Then again, since it is a simply local strategy, it can't give flow data in the inside of uniform locales of the image.

      The Lucas Kanade strategy expect that the removal of the picture content between two nearby frames is little and roughly constant inside an area of the point p under thought. Along these lines the optical flow condition can be accepted to hold for all pixels inside a window focused at p.

      (2)

      Where are the pixels inside the window, and are the partial derivatives of the image I

      with respect to position x, y and time t, evaluated at the point

      and at the current time.

      These equations can be written in matrix form (3), where

      (4)

      This system has more equations than unknowns and thus it is usually over-determined. The LucasKanade method obtains a compromise solution by the least squares principle. Namely, it solves the 2×2 system

      (5)

      where is the transpose of matrix A. That is, it computes

      (6)

      where the central matrix in the equation is an inverse matrix. The sums are running from i=1 to n.

      The matrix is often called the structure tensor of the image at the point p.

      So we used the above method in our approach. It computes optical flow locally. This means that a great calculate without having to depend on the whole image.

      Figure 4: Hot cells with high motion

      Figure 5 illustrates hot and normal zones on the principle of motion intensities. The zone which has high motion is marked as a hot cell [6] [7]. The normal zones are regions of the scene where the motion intensities are low. The points of interest are extracted in the marked zones of the scene. Then, these points of interest are tracked using optical flow techniques. The main benefit of the procedure is the fact that it doesnt require a huge amount of data to enable supervised/unsupervised learning [8] [9].

   3. Building Histogram of Magnitudes

      After the calculation of optical flow in each cell, we stored the values of estimated magnitudes in an array. These data should be visualized in order to classify them correctly. With this data, we implement histogram of magnitude for each zone

      [10] [11]. As indicated before, we split the frame into sixteen cells and each cell is treated individually. We label the zones from A to P as shown in Figure 4.

      Figure 5: Labeling the zones

      After that, we build the histograms for each zone and detect where the hot cells are. Being able to do partitioning would solve several problems like optical flow because we will be working on no discontinuities regions.

   4. Implementation/Simulation Tools

      The design consideration of our system was the programming language that suits best our simulation, while various options such as MATLAB and JAVA were employed; our chosen one is C++ using OpenCV [12]. Ii is widely used for segmentation, recognition, and motion tracking. OpenCV includes machine learning library. Moreover, it makes more time and has memory optimization, plus the availability of libraries for image processing. . We tend to have simple and fast approaches taking into consideration low cost enhancements. The software used is visual studio.

   5. Definig a measure for classification of normal and abnormal event

   After estimating the optical flow in each zone, we got all the magnitudes data and we need to analyze it. Choosing an appropriate threshold for classification should be based on comparison between optical flow calculations in normal scenario and abnormal one. So, we are going to take 2 cases and study their optical flow estimations in order to take the suitable threshold to get high accuracy. To make values easier, we set all the magnitude values to a range between 0 and 16

3. EXPERIMETAL RESULTS

   To examine the execution of our method, we used the Histogram of Magnitudes of the optical flow. We used a video recorded in our university to observe the condition. Videos are utilized to show meaningful data to the security team who need to take proper actions in case of a risky situation.

   1. Experimental Setup

      The experiments were conducted using our personal computer with Intel Core i7 using visual studio 2010 and OpenCv 3.3.0 software. After getting the video, we partition the frame into 4×4 non overlapping zones. For each zone, optical flow vector is estimated. The magnitudes of these vectors are represented on histograms.

   2. Implementation/SimulationResults

      Figure 7: Histogram of Magnitude in A zone.

      The recorded data in zone A is displayed in the Figures 7. It shows a low number of motions. The magnitude ranges between 0 and 16, while the frequency represents the number of Optical flow points recorded with their corresponding magnitudes. Based on this Zone A is categorized as a normal zone.

      Figure 8: Histogram of Magnitude in B zone.

      Figure 9: Histogram of Magnitude in C zone.

      Figure 10: Histogram of Magnitude in D zone.

      Figure 11: Histogram of Magnitude in G zone.

      Figure 12: Histogram of Magnitude in H zone.

      Figure 13: Histogram of Magnitude in K zone.

      Figure 14: Histogram of Magnitude in L zone.

      Figure 15: Histogram of Magnitude in M zone.

      Same interpretation as zone A, for the different zones shown above (B, C, D, G, H, K, L and M) as shown in Figures 8, 9, 10, 11, 12, 13, 14 and 15 they are classified as normal zones based on the results of their histograms and recorded data.

      Figure 16: Histogram of Magnitude in E zone.

      Figure 17: Histogram of Magnitude in F zone.

      Figure 18: Histogram of Magnitude in I zone.

      Figure19: Histogram of Magnitude in J zone.

      As shown in figure 16, 17, 18 and 19; there is high number of motion intensities with high magnitudes that ranges from 8 to 16 in Zone E (Same in zones F, I and J). The above figures show high number of motion with high magnitude. Also, the large number of motion ranges between 8 and 16. Based on this, Zone E, F, I and J are labeled hot zone and can be considered as an indicator to an abnormal event (i.e. students are fighting).

   3. Definig a measure for classification of normal and abnormal event

   After estimating the optical flow in each zone, we got all the magnitudes data and we need to analyze it. Choosing an appropriate threshold for classification was based on comparison between these optical flow calculations in normal scenario and abnormal one. We set all the magnitude values to a range between 0 and 16. We took the magnitude value = 8 as a threshold. If the estimated optical flow magnitude is below 8, than we will mark the case as a normal one. Else, it will be an abnormal event and actions should be taken.

4. CONCLUSION

Nowadays, the use of video surveillance systems became a must in every moment of our daily life, in order to maintain security and provide us with the maximum achieved level of safety. To take benefit on videosurveillance, an efficient method is implemented to detect abnormal events which may occur in crowded scene at LIU Saida campus. This pure software algorithm works with analyzing the content of a video to classify events between normal and abnormal ones, based on optical flow calculations, in each zone of the whole frame, using Lucas-Kanade method. The magnitude histograms were very sufficient and efficient proofs to show the importance of this algorithm in detecting hot cells and then classifying the event. In fact, this implemented method provides an efficient solution for event detection; and it can also be merged in the future with a hardware technique, for example, using a microcontroller to implement a security alert, which can control automatically the door of the auditorium, to protect and maintain the students safety.

ACKNOWLEDGMENT

We would like to sincerely thank our supervisor and the dean and the rest of the people who showed us care, guidance, and support. Were thanking God, without him, we wouldnt have made it. Thanks God every moment, were very thankful for giving us the opportunity to continue our education, making our dreams turn into reality, and giving us the strength to preserve our work.

Thanks for our parents who inspired, loved, encouraged, and helped us to realize how meaningful our work is. We are eternally grateful for the trust they have put in us which gave us the chance to work harder.

We thank the kind of inspection, loving revelation and timely guidance, which we have received from our supervisor Dr. Ismail El- Sayyad. We appreciate your faith in our abilities and will certainly do our very best to meet your expectations. You are truly inspirational at your staff.

REFERENCES

1. Andrade, Ernesto L., Scott Blunsden, and Robert B. Fisher. "Modelling crowd scenes for event detection." Pattern Recognition, 2006. ICPR 2006. 18th International Conference on. Vol. 1. IEEE, 2006.

2. Mehran, Ramin, Alexis Oyama, and Mubarak Shah. "Abnormal crowd behavior detection using social force model." Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009.

3. Javanbakhti, S., and S. Zinger. "Fast abnormal event detection from video surveillance." Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV).

4. Andrade, Ernesto L., Robert B. Fisher, and Scott Blunsden. "Detection of emergency events in crowded scenes." (2006): 528-533.

5. Lucas, Bruce D., and Takeo Kanade. "An iterative image registration technique with an application to stereo vision." (1981): 674-679.

6. Bouguet, Jean-Yves. "Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm." Intel Corporation 5.1-10 (2001).

7. Ihaddadene, Nacim, and Chabane Djeraba. "Real-time crowd motion analysis." Pattern Recognition, 2008. ICPR 2008. 19th International Conference on. IEEE, 2008.

8. Sun, Deqing, Stefan Roth, and Michael J. Black. "Secrets of optical flow estimation and their principles." Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010.

9. Wong, King Yuen, and Minas E. Spetsakis. "Tracking, segmentation and optical flow." Dept. of Computer Science Centre for Vision Research York Technical Report CS-2003-09 (2003).

10. Chaudhry, Rizwan, et al. "Histograms of oriented optical flow and binet- cauchy kernels on nonlinear dynamical systems for the recognition of human actions." computer vision and pattern recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009.

11. Li, Ang, et al. "Histogram of maximal optical flow projection for abnormal events detection in crowded scenes." International Journal of Distributed Sensor Networks 11.11 (2015): 406941.

12. Bradski, Gary, and Adrian Kaehler. "OpenCV." Dr. Dobbs journal of software tools 3 (2000).