Sound Triggered Patrolling And Surveillance Robot Using Deep Learning

Sound Triggered Patrolling And Surveillance Robot Using Deep Learning

SOUND TRIGGERED PATROLLING AND SURVEILLANCE ROBOT USING DEEP LEARNING

Anju Babu1

1Student, Dept. of Computer Science and Engineering, Mangalam College of Engineering, India,

Anjana Prakasp

2Student, Dept. of Computer Science and Engineering, Mangalam College of Engineering, India,

Anna Mariya E S3

3Student, Dept. of Computer Science and Engineering, Mangalam College of Engineering, India,

Jayakrishnan B4

4Assistant Professor, Dept. of Computer Science and Engineering,

Mangalam College of Engineering, India,

Abstract Technology plays an inevitable role in human lives. The development of various technologies has altered the way we live and work. The advent of computer systems and internet has introduced a modern alternative to satisfy our day-to-day requirements. In present and future scenario robots have largest research area in different fields. In Present generation, an autonomous robot has been a popular technology which is widely used in many areas. Robots are used to share the work and act more autonomously in performing the jobs faster than humans. Usually, Robots are more intelligent with endless energy levels and more precise in handling the jobs perfectly. Surveillance is the term which is as old as human civilization, only its mode and technology has changed and evolved. The kind of surveillance adopt here is a patrolling robot which It has the ability to monitor sound in the premises. The robotic vehicle moves at particular intervals and is equipped with night vision camera and sound sensors and it starts moving towards the sound on its predefined path. It then scans the area using its camera to detect any human faces detected. It captures and starts transmitting the images of the situation immediately.

Keywords autonomous robot,night vision,sound sensor

1. INTRODUCTION

   Technology plays an inevitable role in human lives. The development of various technologies has altered the way we live and work. The advent of computer systems and internet has introduced a modern alternative to satisfy our day-to- day requirements. Majority of people rely on internet and computers to perform certain task, most significantly technology altered our way of life while also making it more comfortable. As the world slowly moves towards modernization, peoples are looking for innovative ways to make life easier and more efficient. The security sector has made progress in the development of existing technology but has yet to take full advantage of the technology. When societies first began developing, nearly all productions and efforts were the result of human labors. With technological advances machines were slowly developed. For the last

   decades, many researches had been done to develop the autonomous robot in order to perform risky, dangerous and continued task and hence replace human in certain jobs. Throughout the time, the autonomous robot has been developed according to the progress in different ability such as self-maintenance, task performance, sensing the environment, outdoor autonomous position-sensing and navigation, and indoor position sensing and navigation. Today, the security monitoring job is required and important for the most of place due to safety issues. The monitoring job generally needs to be done in 24 hours time to ensure that any unwanted incident will not happen and because of that, the autonomous robot is needed to be design so that it can replaced the human work which is currently doing the security monitoring manually by checking around the area continuously. The concept of integrating robotics into security sector is relatively new trend, and is mostly efficient. The robot is an autonomous rover, fitted with ultrasound sensor. The ultrasound sensors are what helps the robot in becoming autonomous, detecting obstacles, and avoiding them according to the algorithms set through the raspberry pi, which is the brain of the robot. The robot has the advantage of using a smart camera, having the functionality of human detection, tracking and night vision, which can be controlled wirelessly via an IOT website, making it easy to use and monitor. The emergence of security robots is a milestone in the evolution of security systems-an emerging stage of technological development that brings the entire industry to new standards of best practices expected in the profession of securing people and property. Autonomous mobile robots designed for outdoor use can reduce and overtime, completely eliminate the need for human workers to ensure the safety of large facilities.

2. LITERATURE REVIEW

   In the paper titled Decentralized Control of a Heterogeneous Human- Robot team for Exploration and Patrolling presents a decentralized connectivity- maintenance control framework for a heterogeneous human- robot team. The algorithm is able to manage teams of an arbitrary number of mobile robots and humans to

   collaboratively achieve exploration and patrolling tasks. The human user becomes physically part of the team and receives information about the team's connectivity through haptic or audio feedback. The proposed approach has been tested in both virtual and real environments and has shown to be effective in a wide range of scenarios. Haptic feedback was preferred over audio feedback by users. The framework is ready for deployment in high-impact applications such as surveillance, search-and-rescue, and disaster response.

   In the paper titled Low illumination image enhancement for night time UAV pedestrian detection presents a convolutional neural network model. The Minkowski distance measurement index of enhanced images is increased to 0.975, and the detection accuracies in terms of F-measure and confidence coefficient reach 0.907 and 0.840, respectively, which are the highest to improve pedestrian detection using unmanned aerial vehicles (UAVs) at night, this article presents an image enhancement method. The method involves mapping the image brightness to a desirable level, using block-matching and 3-D filtering for image denoising and sharpening in YCbCr color space, and performing pedestrian detection using a compared to other image enhancement methods. This developed method has potential applications in night-time UAV visual monitoring in smart city contexts. The aim of this paper is to improve the object detection accuracy of UAV cruise at night by developing a hybrid low-illumination image enhancement method that integrates the optimal HTC and improved BM3D algorithms.

   The paper titled Robot Assisted pedestrian regulation based on deep reinforcement learning presents Pedestrian regulation can improve crowd safety in densely populated areas. In recent studies, mobile robots have been used to regulate pedestrian flows through passive human-robot interaction (HRI). This paper presents a robot motion planning problem for the optimization of two merging pedestrian flows moving through a bottleneck exit. To address the challenge of representing complex human motion dynamics under HRI, a deep neural network is used to model the mapping from the image input of pedestrian environments to the output of robot motion decisions. The robot motion planner is trained using a deep reinforcement learning algorithm, which improves the learning capability for complex dynamic problems by avoiding hand-crafted feature detection and extraction. The proposed approach is validated in simulated experiments, and its performance is evaluated. The results show that the robot is able to find optimal motion decisions that maximize pedetrian outflow in different flow conditions, resulting in a significant increase in pedestrian-accumulated outflow compared to cases without robot regulation and with random robot motion.

   In the paper titled Night time pedestrian and vehicle detection based on a fast saliency and multifeatured fusion algorithm for infrared images demonstrates, in recent years, the detection of pedestrians and vehicles at night has become a important subject in computer vision applications. Traditional camera-input algorithms often perform poorly in low light environments. Infrared images, however, can be used to identify pedestrians and vehicles due to their

   brighter appearance and salient features. This paper proposes a method for rapidly identifying areas of interest in infrared images at night using a fast saliency map. This map is used to refine and separate the target area to obtain accurate candidate bounding boxes for pedestrians and vehicles. A multi-feature fusion algorithm and support vector machine (SVM) are used to determine whether the extracted target area contains pedestrians or vehicles. Experimental results show that the proposed method is effective at classifying and detecting pedestrians and vehicles at night and can meet the real-time requirements of actual road scenarios.

3. PROPOSED SYSTEM

   The Night Patrolling Robot is an independent robot capable of roaming freely without any human assistance. After the robot input in a particular location, the robot travels to its workplace by maintaining the shortest distance and simultaneously scanning the area for any irregularities. With its ability to navigate obstacles and scan for irregularities, the Night Patrolling Robot is a reliable solution for safeguarding the area and minimizing human errors. In case the robot encounters an obstacle or an intruder, it notifies the control room. Figure 1 shows the flow chart of the proposed system.

   Figure 1: The proposed flow diagram

   The system uses camera and mic mounted on robotic vehicle for securing any premises. The robotic vehicle

   moves in a particular path and is equipped with camera and sound sensors.it uses a predefined line to follow its path while patrolling. If any suspicious sound detected while moving its path, it moves to that location. The system uses IR based path finding system for patrolling assigned area. If it detects any suspicious sound while patrolling it moves to that location where sound is detected. Then the system scans the area using camera to identify any human face. The system uses IR sensor to avoid obstacles along the path. It monitors the area and detect any problem using camera. Robotic vehicle then captures the image and transmit the image or Realtime video through an IOT website to an authorized person. Images or video displays to the authorized user with alert sound.

   Microphone sensor mounted onto the robot capture sound from the crowded area and send to the processor. Sound captured by the microphone sensor compared with a predefined sound to find a match. Once the match is found command to navigate the robot is sent to the motor. Robot is them moved to that location where the sound is recognized. The motor driver program receives the function parameter from the sound detection command and then sends the signal to motor driver circuit which rotates the DC motor. Figure 2 shows the structure of the proposed system.

   Figure 2: Conceptual Model

   IMPLEMENTATION

   Initially, the robot navigates within a designated area. When a suspicious sound is detected by sound sensor equipped with microphones, the tensor flow algorithm compares the sound with pre-recorded sounds. If there is a match, it sends data to the Arduino. Around 300 screaming audios are used in creating the model. Librosa, a python package for music and audio analysis is used to extract various features from audio signals including MFCC (mel-frequency cepstral coefficients). MFCC are a set of features that represent the

   spectral characteristics of an audio signal. The microphones are implemented on a Raspberry Pi or PC, and the Arduino is connected with motors and motor drivers. DC motors and motor drivers are utilized to control the robot's motion, including the movement of its wheels, arms, or other parts. The microcontroller sends signals to the motor driver, which amplifies and converts them into the necessary voltage and current for the DC motor to rotate. By controlling the voltage and current, the speed and direction of the motor can be controlled with precision, enabling accurate control of the robot's movement. The Arduino is also connected with a magnetometer, which acts as a compass to direct the robot towards the location of the detected sound. Using a navigation algorithm, the robot moves for 20 seconds towards that direction. Additionally, an ultrasonic sensor is connected to the Arduino to detect obstacles in the robot's movement path. If an obstacle is found, the robot stops moving and takes a different direction to reach the location of the detected sound. Finally, the Arduino sends data to the PC or Raspberry Pi. Once the robot reaches the location, the camera starts capturing live video that can be accessed by authorized individuals connected to the same network.

   HARDWARE REQUIREMENTS

   1. Raspberry pi

      Raspberry Pi is a series of small, low-cost, single-board computers that can be used for a wide range of projects and applications. The Raspberry Pi consists of a credit card- sized circuit board that contains a system-on-a-chip (SoC) processor, memory, and I/O interfaces, including USB, Ethernet, and HDMI ports. It is used for screaming sound detection and live streaming.

   2. Arduino uno

   Arduino Uno is a microcontroller board based on the ATmega328P microcontroller. The Arduino Uno board is designed to operate with a voltage range of 5 to 12 volts. It has an on-board voltage regulator that can regulate the input voltage to a stable 5 volts, which is used to power the microcontroller and other components on the board. Arduino is used for navigation purpose of this robot.

   C)Ultrasonic sensor

   In a robot, ultrasonic sensors can be mounted in various positions, such as on the front, back, or sides of the robot. By using multiple sensors, the robot can detect objects in its environment from different directions, which is helpful for navigation and avoiding obstacles.

   D)Dc motor Driver(L298)

   The L298 is a popular motor driver IC used to control DC motors in robotics and other projects. It can drive two DC motors independently and supports a wide range of operating voltages and current levels. The L298 consists of two H-bridges, which allow the motor to be driven in both directions (forward and reverse).

   1. Sound sensor

      sound sensor, also known as a microphone, is an electronic component that converts sound waves into an electrical signal. Sound sensors are commonly used in robotics and other electronic projects to detect sound and trigger specific actions or responses.

   2. Voltage Regulator

      A voltage regulator is an electronic device that is used to maintain a constant voltage level at its output. It is designed to provide a stable voltage output despite variations in the input voltage or load conditions. Voltage regulators are used in a wide range of applications, including power supplies, battery chargers, and electronic circuits.

   3. WIFI module

   A Wi-Fi module is a device that allows a robot to connect to a wireless network using Wi-Fi technology. Wi-Fi modules can be used to transmit and receive data, allowing robots to communicate with other devices or access the internet. Some Wi-Fi modules are designed to be low power and are suitable for use in battery-powered robots, while others are more suited to robots that are plugged into a power source.

4. EXPERIMENT AND RESULT

   The experiments and results are demonstrating the potential of the night patrolling robot as a reliable ad cost- effective solution for enhancing security surveillance. Figure 3 represents the proposed model, which is fitted with microphones and ultrasonic sensors located at the front of the robot. The Arduino drives the robotic vehicle, while a Raspberry Pi equipped with a camera is utilized for scream detection and live streaming.

   Figure 3: Proposed Model

   Figure 4 illustrates the screaming detection process of the proposed system; Sound is detected by using microphone and then the sound is compared with the trained sounds by using tensor flow. If the trained sound matches the received

   sound wake word is detected. And robot moves to that direction in which the sound is detected to start live streaming.

   Live streaming occurs at the location where the screaming sound is detected. Authorized person can view the visuals from the area where danger is detected using an IoT website. Figure 5.1 shows the IP address displayed on the command prompt of the live streaming website, while Figure 4.2 displays the live streaming visuals.

   Figure 4: Screaming Detection

   Figure 5.1: command prompt showing the IP address of live streaming website.

   Figure 5.2: live streaming

5. CONCLUSION

In conclusion, patrolling robots are a promising solution for improving security and surveillance. They have the potential to enhance public safety, reduce crime, and provide real-time monitoring of critical infrastructure. Additionally, patrolling robots can be deployed in hazardous or difficult-to-access areas, reducing the risk of harm to human security personnel.

However, there are also challenges to be addressed in developing and deploying patrolling and surveillance robots. These include ensuring their reliability, autonomy, and ability to adapt to changing environments. The cost of development and maintenance can also be a barrier to widespread adoption.

Overall, patrolling robots have significant potential to improve security and surveillance, and ongoing research and development efforts are needed to further refine these technologies and address the challenges they pose.

RERERENCES

[1] M. Aggravi, G. Sirignano, P. R. Giordano and C. Pacchierotti, "Decentralized Control of a Heterogeneous HumanRobot Team for Exploration and Patrolling," in IEEE Transactions on Automation Science and Engineering, vol. 19, no. 4, pp. 3109-3125, Oct. 2022.

[2] W. Wang, Y. Peng, G. Cao, X. Guo and N. Kwok, "Low-Illumination Image Enhancement for Night-Time UAV Pedestrian Detection," in IEEE Transactions on Industrial Informatics, vol. 17, no. 8, pp. 5208-5217, Aug.

2021.

[3] Z. Wan, C. Jiang, M. Fahad, Z. Ni, Y. Guo and H. He, "Robot-Assisted Pedestrian Regulation Based on Deep Reinforcement Learning," in IEEE Transactions on Cybernetics, vol. 50, no. 4, pp. 1669-1682, April 2020.

[4] T. Xue, Z. Zhang, W. Ma, Y. Li, A. Yang and T. Ji, "Nighttime Pedestrian and Vehicle Detection Based on a Fast Saliency and Multifeatured Fusion Algorithm for Infrared Images," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 9, pp. 16741-16751,

Sept. 2022.