Real Time Snake Detection and Alert System

Real Time Snake Detection and Alert System

Indrajith K R (Student)

Dept of CSE, Sandur Polytechnic,

Yeshwantnagar,

Vinay Kumar C M (Student)

Dept of CSE, Sandur Polytechnic,

Yeshwantnagar,

Abstract : Snake interactions may harm public safety and biological diversity in both human and animal environments. A Snake Detection and Alert System using AI and ML technologies has generated interest as a potential solution to this problem. The technology aims to correctly recognize snakes and send timely alerts, minimizing human-snake conflicts and increasing relationships. It achieves this through real-time video processing, object identification techniques such as YOLO, and automated decision-making. Current research and works in this field have demonstrated the ability to identify snakes in video footage using machine learning methodologies. Several studies have been using deep learning techniques, such as YOLO allows you to accurately and quickly identify snakes. Further research is needed to better understand how modern systems adapt to different contexts. Real-time processing requirements may also face challenges in environments with restricted resources. The flexible Real-Time Snake Detection with Alert Systems Using Deep Learning presented in this study will solve this gap. Real- time snake detection and alarm generation from a model's video feed are among the objectives. With the support of these objectives, the project seeks to develop a comprehensive and useful solution that will help in the protection of both humans and wildlife in places where snakes are prevalent.

Keywords YOLOV8 , CVAT , snake detection, triggering system, safety

I INTRODUCTION

Detecting snakes in video footage is challenging due to their diverse species, concealment abilities, and various natural settings. Recent advances in machine learning have enabled highly precise snake detection algorithms.

CNNs are often utilized for snake detection. CNNs are effective deep learning algorithms for identifying images. Using labelled snake photos, students can learn how to distinguish between snakes and other items. Once trained, a CNN can recognize snakes in real-time video footage. CNN will scan each frame of the footage for snake-like behaviors. More investigation confirms the existence of snakes in the identified sites. New machine learning algorithms increase improvements in snake detecting technologies. The latest developments have enabled highly precise snake detection

systems, which can be used for a variety of purposes such as wildlife monitoring, insect control, and public safety.

There are various reasons for detecting snakes and alerting them, including:

Public safety: Snakes can pose a threat to small children and pets. Snake detecting systems can alert people about snakes in their nearby areas, allowing them to take actions to avoid them Wildlife Monitoring: Snake detecting devices can be used for monitoring wildlife numbers and movements. This knowledge can help protect snake populations and improve understanding of their environment and behavior.

Insect Control: Snake detection systems can detect and monitor poisonous or crop-damaging snakes for pest control purposes. All this data allows for designed insect control actions. Snake detection devices can provide scientific study data on snake behavior and ecology.

Scientific Research: Researchers can use this knowledge to improve conservation strategies and gain a better understanding of snakes.

1. LITERATURE SURVEY

   Agi Prasetiadi, Condro Kartiko, Elisa Usad, January(2020), Using Convolutional Neural Networks for Reptile Recognition.

   The paper addresses the lack of public interest and awareness of reptiles in Indonesia, which stems from cultural misconceptions. It highlights the importance of reptiles in ecosystems and human activities and proposes a solution using deep learning techniques to recognize and classify reptile species accurately. The proposed solution involves the development of a specialised Convolutional Neural Network (CNN) model trained on a dataset of reptile images, achieving a reasonable accuracy of 64.3% for detecting 14 species. Additionally, the paper explores the possibility of deploying a reptile recognition application on Android smartphones for wider accessibility. In conclusion, the research contributes to computer vision and machine learning applied to wildlife

   conservation and public education by providing a practical solution to increase public awareness and appreciation of reptiles.

   Bhandari et al., (2020), Enhancement and Recognition of Objects for Night Vision Surveillance.

   The paper discusses research on improving object recognition in low light and no light conditions for surveillance using Infrared Imaging, image enhancement techniques, and convolutional neural networks. It addresses limitations of traditional surveillance systems and proposes a comprehensive solution to enhance object visibility and accuracy in challenging lighting environments. Key contributions include system development, implementation of a light-sealed box setup, introduction of a system block diagram, identification of camera calibration points, comparison of image enhancement algorithms, and evaluation of system performance. Overall, the paper aims to advance technology for enhancing security surveillance capabilities in low light scenarios.

   Rajabizadeh M, Rezghi M, (2021), A comparative research on image-based snake detection using machine learning.

   The paper "A Comparative Study of Image-based Snake Identification using Machine Learning" falls within the research area of image classification and machine learning applied to wildlife identification, specifically focusing on snake species identification using image analysis techniques. It addresses the problem of automated snake image identification by comparing the accuracy of various machine learning algorithms. The study aims to improve snake bite management by enabling automatic identification of snake species, contributing to public health and safety, especially in regions like Lar National Park, Iran. The paper's key contributions lie in its comparative analysis, performance evaluation of machine learning algorithms, demonstration of improved accuracy, and practical application of deep learning models for snake image identification.

   Mohd Junedul Haque , Mohd Muntjir, (2017), Night Vision Technology.

   The paper on Night Vision Technology delves into the realm of low-light visibility solutions, focusing on the technologies, characteristics, and applications of night vision systems. It addresses the problem of limited visibility in dark or low-light conditions by elucidating various night vision technologies, including image enhancement and thermal imaging. By providing insights into these technologies' working principles, the paper aims to equip readers with a comprehensive understanding of how night vision devices enhance visibility in challenging environments. Additionally, the paper highlights the characteristics of night vision systems and showcases their diverse applictions, spanning military operations, law enforcement, wildlife observation, surveillance, and search and rescue missions. Through a clear explanation of image enhancement and thermal imaging mechanisms, the authors empower individuals to effectively utilize night vision devices, ultimately contributing to

   improved situational awareness and operational efficiency in low-light scenarios. As a review or overview paper, it consolidates existing knowledge on night vision technology, offering readers a broad understanding of its fundamentals and practical significance.

   Ghaith Al-refai, Hisham Elmoaqet, Mutaz Ryalat , Mohammed Al-refa, (2023), Detecting Objects in Low-Light Environments with YOLOv7.

   The research paper "Object Detection In Low-Light Environment Using YOLOv7" delves into the domain of computer vision and deep learning, specifically targeting object detection challenges in dark situations. It emphasizes the critical importance of accurately detecting objects in scenarios with poor lighting, for instance, in surveillance systems and driverless cars. By leveraging the capabilities of YOLOv7, the latest release in the YOLO series, and introducing innovative training techniques like 'bag of freebies', the paper aims to enhance detection performance in low-light environments. Utilizing the Extended Efficient Layer Aggregation Network (E-ELAN) as the backbone network further enhances the model's ability to adapt to low- light conditions without compromising performance. Rigorous evaluation using the Ex Dark dataset demonstrates the effectiveness of YOLOv7 in accurately detecting objects under challenging lighting conditions, thus advancing the field of object detection in l0w-light environments.

   Naresh et al., (2023), A Novel framework for detecting harmful snakes using the YOLO algorithm.

   This paper "A Novel Framework for Detection of Harmful Snakes Using YOLO Algorithm" presents a research study focused on detecting harmful snakes in agricultural fields to mitigate the risks of snake bites to humans and livestock. The paper introduces a machine learning model that utilizes the YOLO algorithm for real-time snake detection based on images captured by cameras. By training the model on a diverse set of snake images, the system can identify different snake species and provide early warnings to farmers. The proposed framework aims to enhance agricultural safety by leveraging advanced technologies and deep learning techniques for efficient and accurate snake detection, ultimately contributing to the well-being of farmers and livestock in agricultural environments.

   Zulu CL, Dzobo, (2023), Real-time power theft monitoring and detection system with a dual- connected data capturing system .

   This paper discusses a real-time power theft monitoring and detection system using smart meters and GSM modules. It addresses challenges faced by power utilities, such as unauthorized connections and meter manipulation. The system involves measuring power distribution and consumption using current and voltage sensors, with data sent to cloud storage for analysis. This technology aims to improve efficiency and combat electricity theft.

   Yang & Sinnott (2021), Deep Learning-Based Snake Detection & Classification

   In this paper various deep learning models were explored for the detection and classification of Australian snakes in a mobile environment. Models such as YOLOv3, Tiny

   YOLO, SSD VGG16, SSD MobileNet, Faster RCNN ResNet, and RetinaNet were trained and evaluated. Faster RCNN ResNet showed the highest accuracy with over 80% mean average precision (MAP), while Tiny YOLO exhibited the fastest inference speed. However, Tiny YOLO sacrificed some accuracy for speed. The study also compared offline (embedded on the device) and online (cloud-based) deployment scenarios, finding that online processing with Faster RCNN incurred higher latency due to network communication. Overall, model selection depends on the trade-off between accuracy and speed requirements in a given application context.

2. AI AND INNOVATION

   The snake identification and alert system can considerably benefit from AI and ML integration. This improves the solution's durability, effectiveness, and domain-specific adaptivity. The method uses machine learning to accurately identify snake species from a large video sample. Adaptation is crucial when multiple snake species live together. based on artificial intelligence real-time video processing improves safety in snake-prone areas by quickly evaluating incoming frames and giving timely alerts to users. Using computer vision-based object recognition techniques helps identify and pinpoint snakes in video frames, reducing false positives and alert tolerance. Furthermore, the ML-based decision-making features validate detections for high accuracy and reduce unnecessary alarms.

   Computer vision plays a crucial role in detecting snakes via video footage and alarm systems. To detect snakes and send alerts, the system must be able to decode visual data from camera frames. Computer vision, specifically object detection algorithms, is used to locate and identify snakes in video frames. Popular algorithms for this problem include YOLO (You Only Look Once).

   These approaches are suited for real-time video processing, as they can detect snakes with bounding boxes in real-time or near real-time. Real-time technology enables proactive preventive measures, such as informing emergency personnel or medical institutions during snake encounters. The AI model's versatility allows for future updates and improvements, ensuring the system's long-term functionality. The process begins with collecting and analyzing video footage of multiple snake species. Preprocessing procedures, such as frame extraction and data augmentation, increase the diversity of the dataset. The YOLOv8 object detection engine uses annotated data to identify snakes in video frames. The system's real-time video processing allows for speedy examination of incoming frames and prompt alerts for snake detection.

3. METHODOLOGY

   The following succinctly describes the study's methodology:

   1. Setting up the dataset: Put together a group of annotated photographs of snakes. Establish borders around the photos' most relevant objects. In total, almost 1000 images were chosen.

   2. Initial-processing: During the preprocessing phase, compress the photographs to a scale appropriate for the YOLO model. Choose a sensible range, such as 0 to 1, for the pixel values. Create the dataset and the training and testing subsets.

   3. Model selection: Choose a suitable YOLO model variant (such as YOLOv3, YOLOv4) according to the precision and speed requirements. The Yolov8 model was selected.

   4. Model Training: Using the chosen architecture, initialize the YOLO model and load pre trained weights, if needed. Use techniques like the Adam optimizer or stochastic gradient descent (SGD) on the labelled dataset to train the model. During training, the model learns to distinguish between capsules and tablets by improving the loss function, which measures the variation in bounding boxes between what was anticipated and what was observed.

   5. Modifying Hyperparameters: Modify the number of training iterations, batch size, anchor box sizes, and learning rate of the YOLO method to improve the model's performance. Iterative experimentation and validation using the test dataset may be required. One hundred epochs have been employed.

   6. Model Evaluation: The trained model's Confusion matrix, recall, accuracy, and average precision are evaluated against the test dataset.

   7. Real-Time Processing: Assemble a pipeline that uses the models for classifying snakes and recognizing objects on each frame to process video frames in real time.

   8. Alert Generation: Develop a system to send out alerts and notify the appropriate people when snakes are found.

4. INSIGHT INTO THE YOLOV8 ALGORITHM

   YOLOv8 is a deep learning technique that employs convolutional neural networks (CNNs) to detect snakes in photographs. They can be trained on a dataset of labelled photographs to learn what distinguishes various items. Region suggestion networks are an algorithmic tool for identifying areas of interest in images. Additional analysis is performed on these zones to determine whether or not an object is there. It follows a multi-scale training methodology. This implies that the algorithm was trained using a variety of photo sizes. As a result, the system can distinguish objects at different scales, increasing its accuracy. Data augmentation is used to improve the accuracy of the algorithm. Data augmentation is a technique that involves creating new photographs from current ones.

   The YOLOv8 model uses a CNN with 21 layers separated into 5 stages. The initial step involves detecting large objects, whereas the final stage focuses on little objects. The concept uses predetermined boundary boxes, known as anchor boxes,

   to aid in item identification. The YOLOv8 method initially generates a grid of cells from the image. The system predicts the possibility of snakes arriving in each cell. The system calculates the bounding box of any snakes present. To detect the snake, predictions from each cell are merged. The technique uses non- maximum suppression (NMS) to filter out duplicate or unneeded detections.

   The YOLOv8 architecture consists of the following modules: Backbone: Convolutional layers that extract pertinent characteristics from the input image make up the backbone. The architecture of YOLOv8 is derived from ResNet50, but it has a few extra layers.

   SPPF: The Spatial Pyramid Pooling (SPP) module increases the accuracy of item recognition across scales and expands the receptive field of the network.

   C2f: A new module, the C2f module was added to YOLOv8. This method combines contextual information with backbone data to improve detection accuracy.

   Real-time object detection: Bounding boxes and class probabilities are estimated for each item in an image by the detecting heads. Five detection heads, one for each object, are a characteristic of YOLOv8.

   Loss function: This is how the YOLOv8 model is trained. The IoU loss and the categorization loss are combined to form the loss function.

5. HIGH LEVEL DESIGN

   A system architecture is the conceptual model that defines the structure, behavior, and more views of a system

   Fig 1: Fig 1: Architecture Diagram

   Fig 1 displays the system architecture for enhanced object detection in snakes, which includes numerous critical components that work together to provide the needed functionality.

   This is an overview of the system architecture. Image/Video Input: The system receives an image or Video input with snakes for item detection. CVAT annotation: Input data will be annotated Using CVAT, annotators can indicate Snake areas of interest (ROIs) in videos Image frames.

   YOLOv8 Model Training: Using annotated data Used to train the YOLOv8 deep learning model. Looking at the annotated ROIs, YOLOv8 is Snake recognition skills were taught.

   Real-time object detection: Once the model has It has been trained and is now deployed to conduct real-time

   Object detection using new input data. The system Detects capsules and pills using the learnt YOLOv8 model to the input images or videos ,Frames.

   Snake Detection alarm: The system delivers an alarm when a snake is discovered. Python code is used to send notifications to certain individuals, such as a WhatsApp message.

   Image annotation:

6. EXPERIMENTAL DATASET

   The experimental dataset includes 1000 images of snakes from various kinds. To make them more helpful, these photos were highly assigned with the Computer Vision Annotation Tool (CVAT). The annotations focused on identifying the snakes that were there. The pictures have been labelled and converted to YOLO format for further analysis and application.

   Fig 2: presents the dataset with annotations

   Figure 2 presents the dataset with annotations with bounding boxes around each item. These boxes indicate whether an object belongs to the "Snake" class.

7. RESULTS

   The results of the Real-Time Snake Detection with Alert Systems Using Deep Learning are presented below:

   Fig 3: Snake Detected from the video footage

   Figure 3 shows the frame in which the snake was spotted, displaying it to the viewer and drawing a bounding box around the area of the image where the snake was discovered. This project supports live monitor

   Fig 4: WhatsApp alert

   Figure 4 explains how to use the Pywhatkit Python library. It has helped with the creation of WhatsApp alerts. Pywhatkit supports a variety of features, including the ability to send messages, images, and URLs via WhatsApp. It illustrates the alert that was issued to the designated in-charge officer.

   Performance and assessment measures can be used to assess the result.

   1. Measures of Performance:

      Performance measurements for YOLO-based object identification include validation equivalents, boxes, categorization, and deforming convolutional damage as training metrics, and a confusion matrix that

      summarizes classification results. While Average Precision of the mean(mAP50) assesses general detecting ability at a 0.5 IoU limit, Precise (B) measures the boundary boxed accuracy.

      a .Confusing Matrix:

      The several types of findings are listed in a table that represents the confusion matrix:

      The accurate identification of the snake from video footage is shown in the figure below. Maximum accuracy is represented by the darker colour. The training model's accuracy is great due to the snake- shaped box on the x-axis and the dark colour on the y- axis, resulting in accurate predictions.

      Fig 5: Confusion matrix for snake detection

      Figure 5 shows the confusion matrix for snake detection. The rows relate to actual classes or basic facts. Every row correlates with the model's The expected class. In a particular combination of actual and expected classes, the number of It is symbolized by the junction of every single column and row.

      A confusion matrix evaluates the model's prediction outputs, including True Positives (TP), False Negatives (FN), False Positives(FP), and True Negatives(TN).

      • True Positives(TP): Number of instances correctly identified as "snakes."

      • Incorrect negatives (FN): a number of cases incorrectly classified as "snakes".

      • False Positives (FP) refer to situations where "snakes" were wrongly forecasted as"not detected" in film.

      • True Negatives (TN): the number of incidents. that were accurately identified as "not detected" in snake- free films.

      The confusion matrix can be used to assess model performance metrics such as accuracy, precision, recall,andF1-score.

      Accuracy is (TP + TN) / (TP + TN + FP + FN) (2)

      The second equation illustrates a categorization model's accuracy by dividing the number of correctly predicted cases (including true positives and negatives) by the total number of suggestions.

      Precision=TP/(TP+FP)(3)

      Precision refers to the fraction of events correctly labelled as positive by the model (true positives), as shown in Equation(3)

      a. F1 Confidence Curve.

      When evaluating the effectiveness of classification modelsThe F1 Confidence Curve is a visualisation of how the F1 score fluctuates with confient standards

   2. Evaluation Metrics:

   The PR Curve and the F1 Confidence Curve are utilized as evaluation tools.

   Figure 6: F1 Confidence Graph

   The figure 6 depicts the F1 Confidence Level for the model constructed with YOLO. The solid line displays all classes, while the thin blue line indicates snakes. The above chart displays a model's score for F1, divided down by class and confidence level.

   The thin and thick lines overlap, making the projected figures of the figure more realistic. Recall and precision are balanced by the single scalar statistic known as the F1-score. If the number of samples that are both positive and negative is inconsistent the

   effectiveness of the model is evaluated. It represents the harmonic mean of precision/recall.

   F1 = 2 * (Precision * Recall)/(Precision + Recall) (1) As demonstrated in equation (1), the F1 score can be determined by multiplying Precision and Recall by two and dividing the result by the sum of Precision and Recall

   a. Curve of Precision Recall

   A graphic representation called The Precision-Recall (PR) Curve is used to assess a categorization algorithm's accuracy The PR curve illustrates how recall and accuracy are affected when the classifier's decision threshold is

   Fig 7: Precision – Recall Curve

   Figure 7 shows the Precision-Recall (PR) Curve, which depicts the YOLO-trained model's performance in recognizing and distinguishing between snakes.. Indicating the model's capacity to correctly identify snakes, The graph illustrates how the model's accuracy and recall change with confident level. Precision is

   defined as the ratio of genuine positive predictions compared to projected positives, including negative results. Recall is the ratio of accurately predicted positive outcomes to all actual positive outcomes, which includes fake negatives as well as real positives.

8. CONCLUSION

Real-Time Snake Detection and Alert Systems Deep Learning is an innovative and advanced technique for improving human safety. The technology uses AI and machine learning to evaluate real-time video streams and accurately detect snakes.

The system's foundation is the YOLOv8 algorithm, which quickly detects and localizes snakes in video frames. Extensive training on multiple datasets leads

to great accuracy in detecting snake species, making it relevant in various contexts. Real-time video processing notifies users of impending snake encounters, prompting them to take necessary precautions. This research lays the groundwork for future advancements such as multi-species detection, behavior analysis, and adaptive learning to create a comprehensive system.

REFERENCE

Reptile Recognition based on Convolutional Neural Network. IJITEE. 2020;9(3S):112-115. doi:10.35940/ijitee.C1026.0193S20

Bhandari A, Kafle A, Dhakal P, Joshi PR, Kshatri DB. Image Enhancement and Object Recognition for Night Vision Surveillance. Published online June 10, 2020. Accessed April 25, 2024. http://arxiv.org/abs/2006.05787

Rajabizadeh M, Rezghi M. A comparative study on image-based snake identification using machine learning. Sci Rep. 2021;11(1):19142. doi:10.1038/s41598-021-96031-1

Junedul M, Muntjir M. Night Vision Technology: An Overview. IJCA. 2017;167(13):37-42. doi:10.5120/ijca2017914562

Object Detection In Low-Light Environment Using YOLOv7. Published online September 29, 2023. doi:10.21203/rs.3.rs- 3365905/v1

Naresh E, Babu JA, Darshan SLS, Murthy SVN, Srinidhi NN. A Novel Framework for Detection of Harmful Snakes Using YOLO Algorithm. SN COMPUT SCI. 2023;5(1):52. doi:10.1007/s42979- 023-02366-z

Zulu CL, Dzobo O. Real-time power theft monitoring and detection system with double connected data capture system. Electr Eng. 2023;105(5):3065-3083. doi:10.1007/s00202-023-01825-3

Yang Z, Sinnott R. Snake Detection and Classification using Deep Learning. In: ; 2021. doi:10.24251/HICSS.2021.148