Mine Depth Analyzer for Safety With Intelligence Swarmbots using OpenCV with TensorFlow

Mine Depth Analyzer for Safety With Intelligence Swarmbots using OpenCV with TensorFlow

Dr. R. Vinothkanna. ME.,Ph.D.,1, M. Deepika2, R. Elakkiya3, T. M. Majeetha Thasleema4, E. Menaka5

Professor, Department of ECE, Vivekanandha College of Engineering for Women, Elayampalayam, India.1

UG scholars, Department of ECE, Vivekanandha College of Engineering for Women, Elayampalayam, India.2,3,4,5

Abstract- In This paper, The underground mining are contain hazardous gases. The soil present there may contain chemical components. These may affect the mining workers while they enter to work underground. To save the workers from this problem, the swarmbots is designed. The swarmbots contains, obstacle avoidance, pH value (chemical level) sensing, poisonous gas sensing, soil nature detection camera and transceiver. The swarmbots can travel automatically by using its obstacle detection property. The chemical sensor placed in the swarmbots used to sense the chemical (arsenic, mercury and sulphuric acid) level present in mining. The gas sensor is used to detect the presence of hazardous gas in mining. The camera provided in the swarmbots used to detect the chemical (sulphide) with changes in the soil and high rate in unprotected waste rock mine tailing has very acidic water which additionally leaches metals and the surrounding rock produce chemical explosion. All these operations controlled by using Raspberry Pi microcontroller cum microprocessor in swarmbots. These data sent to control room (server) by using Wi-Fi module. The control room has receiving Wi-Fi module connected with another Arduino microcontroller. That fetches data and shows us in TensorFlow images graphical view and convolution neural network (CNN) image of training datasets Model for image classification will be generated using Machine Learning (ML) in TensorFlow. This paper proposes the maintaining routine inspection of miners by intelligence swarm bots that has interfaces with TensorFlow image classifier of prediction of chemical data analysis with help of chembot(chemical sensor). The highly swarm communication is reliable to inspect the underground mine with self-coordination, self-assembling and high speed in task performance.

Keywords: swarmbots, mining, chemical sensor, Raspberry Pi, Arduino microcontroller, TensorFlow, CNN, Wi-Fi module, ML.

1. INTRODUCTION

   Mining is that the extraction of valuable minerals or other geological materials from the planet . These deposits form a mineralized package that's of economic interest to the miner. In Sub-surface (underground) mining consists of digging tunnels or shafts into the earth to reach buried ore deposits. It contain hazardous gases explosion. Ore, for processing, and waste rock, for disposal, are delivered to the surface through the tunnels and shafts. The soil present there may contain chemical components. Disposal of mine tailings is typically the only biggest environmental concern facing a tough rock properties and toxic chemicals used to extract the valuable materials of ores which cynide for gold to extract the ore body

   hard rocks naturally contains dangerous chemicals that leaches metal mine, and creates very long-term environmental liabilities which future generations must manage Hazardous to arsenic and mercury. Many hard rock mines(including most gold mines) extract minerals that are bound up with sulphide compounds. These compounds produce vitriol on contact with air and water, a process that happens at a really low rate in undisturbed rock, at a better rate in unprotected waste rock. To prevent disaster and detection of hazardous chemical explosion that present in hard rock and soil of mine to save the workers from this problem, the swarmbots is designed. The swarmbots contains, obstacle avoidance, pH value (chemical level) sensing, poisonous gas sensing, soil nature detection camera and wi-fi transceiver. The swarmbots can travel automatically by using its obstacle detection property. The gas sensor is used to detect the presence of hazardous gas in mining. The camera provided in the swarmbots used to detect the chemical (sulphide) changes in the soil and high rate in unprotected waste rock mine tailing has very acidic water which additionally leaches metals and the surrounding rock produce chemical explosion. All these operations controlled by using Raspberry Pi microcontroller cum microprocessor in swarmbots. A Convolutional Neural Network(CNN) Model for image classification will be generated using Tensorflow. This model will be trained with input images. Class of the input image is already known. A model will perform calculation and it will update itself accordingly the purpose of the project. The trained model will then be used for classification. These data sent to control room (server) by using Wi-Fi module. The control room has receiving Wi-Fi module connected with another Arduino microcontroller. That fetches data and shows us in TensorFlow images graphical view. There is also a need of a skilled analyst to perform this experiment. These facilities are available in established government and private laboratories situated in distant places which takes extensive time for the result. Imaging methods are non-destructive in nature and also has the advantage of giving results in real time.

2. PREVIOUS RELATED WORK

   In this section, we provide a brief survey of related work. The survey does not span the entire field of swarm robots. Instead, it focuses on systems for which detection of acidity in the rocks can be detected. In this we identify four different mechanisms used by this system : mining, swarm robots,

   swarm robots can detect the poiseness gas in the mining, swarm communication, deep learning for digital soil mapping.

   1. Mining:

      1) chemical detection in mining: Boularbah et al. 2006[1] demonstrated Phytotoxkit (plants test) and Microtox_(bacterial test), in the assessment of toxicity of bottom sediments, composts, sewage sludge, and soils. The general trend observed was an increase in metal toxicity measured by the biotest with increasing available metal contents in soils. All the soil samples were classified into toxicity which means that they are toxic and present severe danger. Biotest are a good complement to chemical analyses in the assessment of quality of soils as well as in properly managing them.

   2. swarm robots:

      1. Algorithm in swarm robots for detecting , pH, and temperature in mining: Branch and Fit [2] demonstrated the optimization algorithm which is otherwise known as Stable Noisy Optimization by Branch and Fit . It combines both local and global searching to find the maximal value in the available search space in the most efficient manner. In this example the maximal value sought was the highest yield, the search space defined over ranges of concentration, pH, and temperature, and efficiency in this case is conducting the least amount of experiments. By using this algorithm the swarm robots can detect the pH and temperature of the rocks.

      2. Fisher in 1935[3] did the orthogonality, and factorial experiments in order to increase the sensitivity of designed experiments. He indicated that the key factor in DoE is to apply valid and efficient experiments that will produce quantitative results to support decision making. One of the biggest advantages of DoE is that it allows researchers to decide which reactions and conditions to focus on. This can be achieved through the generation of a mathematical model/design space which exposes a relationship between factors affecting a process and the output of that process.

      3. Swarm robots can search a poisoness gas in mining : Kennedy and Eberhart [4]. demonstated a bio-inspired global optimization technique called particle swarm optimization (PSO) was used as the principal algorithm. Each robot was placed in pre-defined positions with a target position corresponding to a single target. The robots were programmed to search in spirals until the target was found by any one of the robots. Once the target was detected the robots attempted to reach the target using the PSO algorithm.

      4. swarm communication: swarm robots will communicate via infra red. Thus the Lego mindstorms control brick[6] with other simple electronics and sensors to make a prototype swarm robot that could communicate via infra red. The infra red communications system was adequate only for very short range communications and we wished our robots to be able to work together over distances of around 100metres (ie over car park sized areas). The Cybot [7] toy system is marketed by Eaglemoss Publications In that We would ideally like to enable our swarm robot nodes with a differential global positioning system (GPS) to allow them to communicate their

         positions to each other and to a base station. The use of the pseudo global positioning system information as described in allow swarm nodes to lock onto specic neighbours and to provide periodic corrections to their positions.

      5. Human robot interaction design: Jiatong Bao and Huatao Zhang[9]The small teleoperated robot is used for detection, rescue,search and other purposes. As the robot is away from the operator when at work at a place that the operator cannot see, the operator needs to know information about the robots environment, position and movement through a human robot interaction interface. Meanwhile the robot can be controlled to detect nuclear radiation and chemical agents, and be navigated by the human robot interaction interface. So human robot interaction is a very important part of a mobile robot system, and research into human robot interaction should be implemented throughout the entire process of robot system design

   3. Deep learning for digital soil mapping:

      McBratney et al. (2003)[8] Demonstrated the DSM methodology. Following the ideasofDokuchaev(1883)and Jenny(1941),they described the scorpan model as the empirical quantitative relationship of a soil attribute and its spatially implicit forming factors. Over that LeCun et al., 1990 demonstrate the use of the convolutional neural network (CNN) model that incorporates contextual information surrounding an observation to signicantly improve the prediction accuracy over conventional DSM modelsCNNs are based on the concept of a layer of convolving windows which move along a data array in order to detect features(e.g.edges)of the data by using different filters.

   4. Using Chemicals to detect Calcium carbide:

      1. Nasim Shah[10]Researchers use different chemical reaction to detect the presence of calcium carbide on the fruit. Firstly, fruit is washed using water after that they perform the test on the water. Bio-Functioning Gold particle is added to the water and the color of the water is examined. If the color of the water changes after adding gold particle then CaC2 is present.

      2. Analysis of physicochemical properties of soil samples:(Dan Pennock & Braidek, 2006)[12],The soil survey represents the association between soil classes and landscape units established in the field by judicious selection of sampling points. The concept of the limit of quantification is that the measurements reported the level for high standard used for the quantification and not mere detection. Chemical analysis is conducted in a commercial laboratory to determine physicochemical parameters of soil samples collected in and around the magnesite-mining region. Soil differs from the parent material in the morphological, physical, chemical, and biological properties. In addition, soils differ among themselves depending on the different genetic identity factors (INM, 2011; APHA,1999; Lad & Samant, 2015). The soil analysis results include the testing of chemical parameters such as pH, lime status, texture, EC, nitrogen, phosphorous, potassium, Fe, Mn, Zn, and Cu.Providing data for big data and

         ML applications with chem/bio sensor networks: 1)Pioneered by Badugu et al[11] 130 nearly 20 years ago, a smart contact lens can monitor ocular glucose through fluorescence changes. The initial design has been further developed to encompass ions such as calcium, sodium, magnesium, and potassium. Since the early breakthroughs in the 1960s and 1970s, which led to the development of a plethora of electrochemical and opto chemical diagnostic devices, the vision of reliable and affordable sensors, capable of functioning autonomously over extensive time periods (years) to provide access to continuous streams of real-time data, remains unrealized.

      3. Detecting Multiple Odor Sources :

      Javier G. Monroy and Jose Luis Blanco [14] The second experiment was designed to test the behavior of the MCE-nose in the case of multiple gas sources in a more realistic robotic scenario. The scenario consists of a long corridor where three equal-sized small cups filled with acetone were placed at 2 meters from each other. Figure 13 displays the experiment setup, and a picture of the MCE-nose integrated in the PatrolBot platform. For the experiment the PatrolBot was commanded to move in a straight line at a constant speed of 20 cm/s.

   5. Static Semaphores Detection With TensorFlow:

   Athul Motty, Yogitha[13],We collected images of semaphores from google. The training database was then generated by manually tagging the objects (semaphores) in the images using Label . LabeI Img saves the annotations as .XML files in a PASCAL VOC format. A ready-made script is available for creating TFRecords (TensorFlow record format). All the images collected are of the dimensions below 640 X 480 to increase model training efficiency. The training directory consisted of a total of 130 pictures along with its xml annotations where in 80 percent is contained in train folder and the remaining 20 per cent in the test folder. The .XML file is used to generate the TFRecord which is the input to the TensorFlow trainer. TFRecord is being served as the input to the TensorFlow training model. The .XML data is used to create a CSV file that contains all the data for training and testing.

   controlled with the help of a remote which interacts with the robot through wireless communication. This approach emerged on the field of artificial Swarm intelligence, as well as the biological studies of insects, ants and other fields in nature, where swarm behaviour occurs. The main objective of Swarm robotics is to reduce the work load and increase the efficiency of the system. It consisted of a group or swarm of robots which were guided by a server robot using various algorithms programmed in its micro-controller, in other words the server carried out certain computations and distributed the work among various client robots. machine learning (ML), two computing fields that combine to yield machine intelligence. ML can accelerate the answer of intricate chemical problems and even solve problems that otherwise wouldn't be tractable. In bio-inspired computation, computer scientists define procedures that mimic mechanisms observed in natural settings. Examples of chemical genes include fraction of individual components in a given material, polymer block sizes, monomer compositions, and processing temperature. The genome refers to the set of all the genes in a compound, while the resulting properties of a genome are named a phenotype. This strikingly demonstrates that chemical sensors are avoided when longer term, reliable and accurate measurements are required from remote locations and hostile environment. This paper proposes the maintaining routine inspection of miners by intelligence swarm bots that has interfaces with TensorFlow image classifier of prediction of chemical data analysis with help of chembot(chemical sensor). The highly swarm communication is reliable to inspect the underground mine with self coordination, self- assembling and high speed in task performance.

3. PROPOSED WORK

   The mining industries are still facing a lot of problems because explosions in coal mines and gas leakages are leading to the pumping up of injured working personnels. This act cause the event of robots to exchange humans work. The underground mining may contain hazardous gases, Disposal of tailing, Acid mine drainage and Hazardous Properties are produced in mines. The soil present there may contain chemical components. These may affect the mining workers while they enter to work underground. To save the workers from this problem, the swarmbots is designed. The mine detecting robot detects the mines and poisonous gases with the help of sensors and takes a clear picture and video of the incident with the help of a camera. The movement is

   Fig. 1. Working detection of hazardous chemicals in mine by swarmbots using OpenCV with TensorFlow

   1. Transmitter

   2. Receiver

4. BLOCK DIAGRAM

   Fig. 2. Swarmbots transmission

   computer vision. It does not require heavy pre processing of input images and it can be executed on GPU using parallel processing computation technique resulting in faster building and training of models. CNN model consists of layers which can be classified into three types called input layer, the hidden layer and an output layer. Hidden layer is made up of convolutional layer, pooling layer, connected layer, loss layer, etc. each designed to perform specific operations. Working of the model is iterative and it divides the image into several portions and performs an individual operation for each portion. The output of one node is input to the next layer. Performing this operation repeatedly and updating properties of each node i.e. weight, the bias will train the model. This model will then be used to classify images into different classes. To classify the image, the models input layer takes the image as input then convert the image into the 2- dimensional array of variables and gives it to the next layer i.e. hidden layer. Based on what the node is, it will perform its function. Then comes the output layer which tells us the class to which the input image belongs to. For convolution operation of the input image matrix, convolutional layer uses the kernel of size 3×3 along with 32 filters and Rectified Linear Unit (ReLU) as the activation function. Proposed system uses 2 such convolutional along with two pooling layers to reduce the spatial size hence reducing the parameters that needed to be processed. Max Pooling Layer with a filter size of 2×2 was used for this model. This layer takes in values from 4 nodes as input and passes the maximum value as output. Output layer i.e. final layer uses Soft max function as activation function to classify input images into different

   Fig. 3. Server receiving display with help of swarmbots

5. COMPONENTS REQUIRED

   1. Hardware requirements

      • Raspberry Pi 3+

      • Arduino UNO

      • Ultrasonic Sensor

      • Raspberry Pi Camera(5MP)

      • Chemical sensor

      • Temperature and humidity sensor

      • Gas sensor

      • Display

      • Battery

      • PH Strip

   2. Software requirements

      • Raspbian jessie OS

      • OpenCV with TensorFlow

      • Language: Python

      • Arduino Uno

      • Language: Python

6. IMPLEMENTATION DETAILS

   A. Algorithm and software tools

   1. Convolutional Neural Network (CNN): The convolutional neural network is a class of deep learning neural network. This is widely used in image classification, feature extraction,

      classes.

      Fig. 4. CNN Architecture

      As shown in above fig.4.the CNN Architecture consists of Convolution and max-pooling layers which act as a feature extractor and the fully connected layer which performs non- linear transformations on the extracted features and acts as the classifier along with the output layer.

   2. Machine Learning In Sensor Application:

      Chemistry is essential for chemical sensing and biosensing, as well as in intelligent systems, for a variety of reasons, including the development of new materials for building innovative chemical(and electrochemical) sensing

      technologies Electronic noses (e-noses) are on illustrative example of the use of ML methods in sensing and biosensing. Moreover, it is likely that most, reliable and accurate measurements are required from remote locations and hostile environment. The proposed e-nose, called Multi- Chamber Electronic Nose (MCE-nose), comprises several identical sets of MOS sensors accommodated in separate chambers(four in our current prototype),which alternate between sensing and recovery states, providing, as a whole, a device capable of sensing changes in chemical concentrations faster. This e-nose is fixed to the swarmbots with their other sensors is called MCE-nose. All these operations controlled by using Raspberry Pi microcontroller cum microprocessor in swarmbots. These data sent to server by using wi-fi module. The inspection miners has receiving wi-fi module connected with another Arduino microcontroller. Combining the chamber with swarmbots to give accuracy in detection of a hard rock waste tailing disposals are watched. In the proposed system a CNN(Convolutional Neural Network) is used for image classification. CNN model is trained using training dataset. It produce the outputs of images in TensorFlow data model. the input images are taken through SWARM BOTS and that image is tested and matched through training dataset which is trained in machine learning model by using the TensorFlow framework. Using the trained model output will be predicted with accuracy.

   3. TensorFlow: TensorFlow is an open source software library for high-performance numerical computation. Its flexible architecture allows easy deployment of computation across a spread of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Googles AI organization, it comes with strong support for machine learning and deep learning and thus the flexible numerical computation core is used across many other scientific domains. A CNN Model for image classification are getting to be generated using Tensorflow. This model are getting to be trained with input images. Class of the input image is already known. A model will perform calculation and it will update itself accordingly the aim of the project. The trained model will then be used for classification.

   4. OpenCV: Once the system has been trained to detect objects, it's further trained to trace the thing as long because the latter stays within the range of the camera. Our implementation focuses on recognizing chemicals detect in tailing hard rock using the TensorFlow API interface with robot OS . OpenCV may be a picture and video processing library; it's bindings in C, C++, Java and Python. face recognition and detection, optical character recognition, advanced robotic vision, license plate reading, photo editing – altogether these domains of image and video analysis, OpenCV finds use. In our implementation, we deploy OpenCV with python binding to capture images from a video input received from our device .

7. RESULT

   The model is trained by training dataset with train accuracy and validation accuracy in 500 steps and with each and every step of coaching the accuracy of the training and thus the validation increases and provides the last word validation accuracy 97% in conjunction with the training accuracy 100%. update itself accordingly the purpose of the project. The trained model will then be used for clasification.

   Fig.5. Visualizing accuracy of model using Tensor board

8. CONCLUSION

   We have proposed a system that attempts to automate the swarmbots to inspect the mine for safety of miners. We used TensorFlow image Detection API, an open source framework for tasks pertaining to image detection, for training and testing an CNN images detection model with a view to detecting chemical hazardous reaction in mine. we observe that a more evolved use of the TensorFlow machine learning API system. Thus the paper proposes the maintaining routine inspection of miners by intelligence swarmbots that has interfaces with TensorFlow image classifier of predictions of chemical data analysis with help of chemical sensor.The manual inspections of miner time taken is reduced with help of swarm performed task with speed of communication.

9. REFERENCE

1. Chemical properties and toxicity of soils contaminated by mining activity, Baran Agnieszka, Czech Tomasz , Wieczorek Jerzy, Boularbah et al. 2006[1] Accepted: 21 May 2014/Published online: 6 June 2014 The Author(s) 2014. This article is published with open access at Springerlink.com.

2. Designing Algorithms To Aid Discovery by Chemical Robots, Alon B. Henson, Branch and Fit Piotr S. Gromski, and Leroy Cronin WestCHEM, School of Chemistry, University of Glasgow, Glasgow G12 8QQ, United Kingdom.

3. Swarm Robotics: A Research Project with High School Students as Active Participants, Chiraag Nataraj*, sanjeev reddy, Mark woods, B.Samanta, and C.Nataraj This paper was presented at the American Society of Engineering Education Annual Conference, Fisher in 1935 Louisville, KY, June 2010, Paper No. AC 2010-1655.

4. An Architecture for Swarm Robots K.A.Hawick, H.A.James, J.E.Story, Kennedy and Eberhart and R.G.Shepherd Computer Science Division, School of

   Informatics University of Wales, Bangor, North Wales, LL57 1UT, UK {hawick,heath}@bangor.ac.uk Tel: +44 1248 38

   2717, Fax: +44 1248 36 1429.

5. International Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-7, Issue-6, March 2019 508 Published By: Blue Eyes Intelligence Engineering, Lego mindstorms control brick & Sciences Publication Retrieval Number: F2472017519/19©BEIESP.

6. Swarm Robots: Eaglemoss Publications R.G.Shepherd Computer Science Division, School of Informatics University of Wales, Bangor, North Wales, LL57 1UT, UK

   {hawick,heath}@bangor.ac.uk Tel: +44 1248 38 2717, Fax:

   +44 1248 36 1429.

7. Small Teleoperated Robot for Nuclear Radiation and Chemical Leak Detection Regular Paper Kui Qian1, Aiguo Song, Jiatong Bao and Huatao Zhang School of Instrument Science and Engineering, Southeast University, Nanjing, China School of Energy and Power Engineering, V Yangzhou University, Yangzhou, China * Corresponding author E-mail: a.g.song@seu.edu.cn Received 2 May 2012; Accepted 15 Jun 2012 DOI: 10.5772/50720 © 2012 Qian et al.; licensee InTech. This is an open access article distributed under the terms of the CreativeCommonsAttributionLicense(http://creativecommons. org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Open access.

8. Using deep learning for digital soil mapping JoséPadarian,BudimanMinasny, McBratney et al. andAlexB.McBratney Sydney Institute of Agriculture and School of Life and Environmental Sciences, the University of Sydney,NewsouthWales,AustraliaCorrespondence:JoséPadari an(jose.padarian@syndey.edu.au)Received:14 August 2018 Discussion started: 3 September 2018 Revised: 18 January 2019 Accepted: 7 February 2019 Published: 26 February 2019.

9. Identification of Artificially Ripened Fruits using Machine Learning, Harshad Vaviya K. J. Somaiya Institute of Engineering and Information Technology, Sion, Mumbai University of Mumbaiharshad.v@somaiya.edu Vijaykumar Vishwakarma K. J. Somaiya Institute of Engineering and Information Technology, Sion, Mumbai University of Mumbai vijaykumar.v@somaiya.edu Ajaykumar Yadav K. J. Somaiya Institute of Engineering and Information Technology, Sion, Mumbai University of Mumbai ajaykumar.yadav@somaiya.edu Nasim Shah K. J. Somaiya .

10. Soil fertility analysis in and around magnesite mines, Salem, IndiaC R Paramasivam, Dan Pennock & Braidek, &SiddanAnbazhagan.(2019,Geology,Ecology,andLandscapes)

    ,DOI:10.1080/24749508.2019.160840.

11. A survey on Big Data and Machine Learning for Chemistry ,Jose F Rodrigues Jr, Larisa Florea , Maria C F de Oliveira ,Dermot Diamond , Pioneered by Badugu et al Osvaldo N Oliveira Jr Institute of Mathematical Sciences and Computing, University of São Paulo (USP), São Carlos

    Institute of Physics, University of São Paulo (USP) 13560-970 São Carlos, SP, Brazil Dublin City University (DCU) Glasnevin, Dublin 9, Dublin, Ireland [junio, cristina].

12. The Multi-Chamber Electronic NoseAn Improved Olfaction Sensor for Mobile Robotics, Javier Gonzalez- Jimenez , Javier G. Monroy and Jose Luis Blanco Department of System Engineering and Automation, University of Malaga, Campus de Teatinos, 29071 Malaga, Spain; E-Mail: jgmonroy@uma.es Department of Civil Engineering, University of Malaga, Campus de Teatinos,Fax: +34-952-133-

    361. Received: 19 April 2011; in revised form: 19 May 2011 / Accepted: 30 May 2011 / Published: 7 June 2011.

13. Flag Semaphore Detection Using Tensorflow and Opencv Athul Motty, Yogitha A, R Nandakuma,International Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277- 3878, Volume-7.