Leaf Disease Detection & Correction using YOLO V7 with GPT3 integrated

Leaf Disease Detection & Correction using YOLO V7 with GPT3 integratedccccc

Sajitha P, Alwin John, V L Devika, Gayathri S V, Nafla Sakhir

Department of Electronics and Communication Engineering Ace College of Engineering

Trivandrum-695027 Kerala, India

Abstract – Leaf disease is a major problem in agriculture, as it can lead to significant crop yield losses. In recent years, deep learning techniques have been increasingly used for the detection and classification of plant diseases. YOLO (You Only Look Once) is a popular object detection algorithm that can detect and classify objects in real-time. In this study, we present a leaf disease detection system using YOLO v7. The proposed system is designed to automatically detect and classify leaf diseases in real-time using a deep convolutional neural network (CNN) and provides correction methods using GPT-3 for the detected diseases. The system is trained on a large dataset of plant images to accurately classify the type of disease present in the leaves. The YOLO v7 algorithm is used for its fast and efficient object detection capabilities. The system achieved high accuracy in detecting multiple classes of leaf diseases, including powdery mildew, rust, and blight. The proposed system has the potential to be used as a tool for early detection and correction of leaf diseases, which can help farmers to take preventive measures and reduce crop losses.

KeywordsDeep Learning, Convolutional Neural Networks, Computer Vision, Plant disease detection, Object detection and Correction, PlantLeaf dataset, You Only Look Once, YOLO v7, GPT- 3

effective disease management and control. Traditionally, farmers and agronomists have relied on visual inspection and manual diagnosis of plant diseases, which can be time- consuming and subjective. Recent advances in computer vision and deep learning have provided new opportunities for automated plant disease detection. Deep learning-based approaches have shown promising results in image-based plant disease detection, providing high accuracy and efficiency in detecting multiple types of plant diseases.

In this study, we propose a novel approach for leaf disease detection using the YOLO v7 algorithm and provides correction methods using GPT-3. The YOLO v7 algorithm is a state-of- the-art object detection framework that can detect and classify objects in real-time. Our proposed method achieves high accuracy and efficiency in detecting multiple types of leaf diseases and gives methods for correcting and preventing the detected diseases, making it a valuable tool for plant disease management and control in agriculture.

1. LITERATURE REVIEW

   1. INTRODUCTION

Agriculture is one of the most important sectors in any country's economy as it plays a vital role in the development of the nation. It is the backbone of human civilization, providing food, clothing, and shelter to billions of people worldwide. Since the beginning of human civilization, agriculture has played a crucial role in the development of societies, allowing humans to settle in one place and form communities. Over time, agriculture has evolved from a simple subsistence activity to a complex industry that relies on technology, science, and innovation.

In modern times, agriculture faces several challenges, including climate change, food insecurity, and the overuse of chemical pesticides and fertilizers. One of the key challenges facing in agriculture is the prevalence of plant diseases, which can significantly impact crop yields and quality. Plant diseases can be caused by fungi, bacteria, viruses, or other pathogens, and can spread rapidly if left undetected and untreated.

Leaf diseases are among the most common types of plant diseases and can cause significant damage to crops. Early detection and diagnosis of leaf diseases are essential for

A large number of experiments have been carried out to implement an effective solution for early plant leaf disease detection with advanced technologies and AI agriculture. Apu Shill and Md Asifur Rahman [4] proposed a computer vision approach using the real time object detection algorithm called YOLO. Among YOLO they have incorporated YOLO V3 and YOLO V4. Around seventeen various classes of diseases in thirteen selected plants have been studied in this research using plantdoc dataset. But in this model, as the number of classes were more, they lacked accuracy. Achyut morbekes, ashi parihar and rashmi jadhav [2] proposed a system that makes use of a novel approach of the real time image detection technique to detect different diseases in crops, YOLO. But to view the resulting image with a bounding box, OpenCV installation and setup is required.

Md Janibul Alam Soeb, Md Fahad Jubayer and Islam Md. Meftaul [5] aims to present an artificial intelligence based solution to the problem of tea leaf disease detectionYOLO v7. They have done the research using the dataset containing 4000 images of five types of diseases from four prominent tea gardens in Bangladesh. The proposed model detected five distinct types of tea leaf diseases and differentiated between healthy and diseased

leaves. Midhun P Mathew and Theresa Yamuna Mahesh [3] proposed this paper which focuses on disease detection in bell pepper plant using deep learning approach. The major diseases that affects the bell pepper is bacterial spots and it spreads rapidly. This paper mainly describes a model that can help the farmers to identify these disease with low additional coasts.. With the help of YOLO V5, they are able to detect the disease even small spots on leaf. It takes the full image in a single instant and predict bounding boxed and class probability. The main motive of this paper to make easier for the farmers to detect the bacterial spot at an earlier stage. Monalika Padma Reddy and Deeksha A [6] proposed the object detection in mulberry leaf using YOLO. The mulberry leaf is the most important component when it comes to Sericulture since the quality and quantity of leaf have a direct impact on cocoon bearing. Through this research they are able to detect the diseases at an earlier stage. The techniques proposed in this paper are CNN(Convolution Neural Networks) and YOLO. The image is taken and divided into several grids which then classifies into different diseases with the help of these techniques. Using YOLO, the infected parts of the leaf is detected and boundary box is drawn around it, later CNN is used for disease classification. The above stated three models are confined to a particular type of plant.

1. METHADOLOGY

   In this section the methodological aspect of the plant disease detection model in terms of YOLO v7 and GPT3 has been discussed.

   1. YOLO v7

      YOLO v7 refers to the seventh version of the YOLO (You Only Look Once) object detection algorithm. YOLO is a popular real- time object detection algorithm that is known for its speed and accuracy. YOLO v7 builds upon the previous versions of YOLO, incorporating various advancements in its architecture and training techniques to improve its performance. One of the key advantages of YOLO v7 is its exceptional speed, capable of processing images at an impressive rate of 155 frames per second. This significantly surpasses other state-of-the-art object detection algorithms. YOLO v7 is particularly well-suited for real-time applications that require swift and responsive detection, such as surveillance systems and self-driving cars.

      In terms of accuracy, YOLO v7 performs favorably when compared to other object detection algorithms. It has been evaluated on benchmark datasets lik COCO (Common Objects in Context), where it achieves competitive results in terms of average precision and intersection over union (IoU) metrics.

      YOLO v7 incorporates advancements such as extended efficient layer aggregation, model scaling for concatenation- based models, trainable bag of freebies, planned re- parameterized convolution, and a coarse-to-fine approach for auxiliary and lead loss. These improvements contribute to its superior performance in object detection tasks.

      1. Performance of YOLO v7 Object Detection

         The performance of YOLO v7 was assessed by comparing it to previous versions such as YOLO v4, YOLO v5, and YOLOR, which served as baselines. All models were trained using identical settings. YOLO v7, the latest iteration, demonstrates the most favorable balance between speed and accuracy among state-of-the-art object detectors. In terms of speed and accuracy, YOLO v7 surpasses all previous object detectors, providing a performance range from 5 FPS to an impressive 160 FPS. When utilizing a GPU V100, the YOLO v7 algorithm achieves superior accuracy compared to other real-time object detection models, operating at a speed of 30 FPS or higher.

         Compared to YOLO v4, YOLO v7 achieves a 75% reduction in parameter count and requires 36% less computation, while simultaneously attaining a 1.5% higher average precision (AP). Similarly, compared to the edge-optimized version YOLO v4- tiny, YOLO v7-tiny decreases the parameter count by 39% and computation by 49%, while maintaining the same AP.

      2. YOLO v7 Architecture

         The architecture of YOLO v7 draws inspiration from earlier YOLO model architectures, specifically YOLO v4, Scaled YOLO v4, and YOLO-R.

         1. Extended Efficient Layer Aggregation Network (E- ELAN)

            The computational module at the core of YOLO v7 is called E- ELAN, an acronym for Extended Efficient Layer Aggregation Network. This architecture plays a crucial role in enhancing the learning capabilities of the YOLO v7 model. It achieves this by employing a technique called "expand, shuffle, merge cardinality," which enables continuous improvement in the network's learning ability without compromising the original gradient pathway.

         2. YOLO v7 Compound Model Scaling

            Model scaling is primarily employed to modify essential characteristics of a model in order to create models that align with diverse application demands. An essential component of model scaling involves optimizing attributes such as the model's width (number of channels), depth (number of stages), and resolution (input image size). By utilizing a compound scaling approach, it becomes possible to preserve the properties of the model as intended during the initial design phase. This enables the maintenance of an optimal structure throughout the scaling process.

   2. GPT3

   GPT-3, short for Generative Pre-trained Transformer 3, is an advanced language model created by OpenAI. Its primary objective is to generate text that closely resembles human language by predicting the next word in a given sequence. GPT- 3 has been trained on an extensive dataset of text and has

   exhibited remarkable performance across various natural language processing tasks, including language translation, summarization, and question answering. What distinguishes GPT-3 from earlier language models is its immense size and versatility. With an impressive parameter count of over 175 billion, it currently stands as the largest language model available. This vast scale enables GPT-3 to produce text that is more coherent and contextually relevant. Furthermore, GPT-3 can handle a wide array of language-related tasks without necessitating fine-tuning or specialized training, making it an influential tool for both generating and comprehending natural language.

   To employ GPT-3, users typically need to register for access to OpenAI's API, which allows them to send text prompts and receive generated text in return. The API can be integrated into a diverse range of applications and platforms, including chatbots, virtual assistants, and content creation tools. Researchers and developers continue to explore methods for enhancing language models like GPT-3, while simultaneously examining the ethical ramifications of their implementation. Some experts believe that language models like GPT-3 could represent a significant stride toward the realization of artificial general intelligence (AGI), which would possess human-level competence in performing a broad spectrum of cognitive tasks.

   Here, our work started by collecting dataset. The dataset named PlantLeaf was collected from plant fields by our team.

   Fig 1. Methodological approach of our work

   After the collection of dataset, it was labelled using roboflow and later exported in YOLOv7 format in order to start with the training process. It also includes integration of GPT3 API to provide correction methods for the diseases.

2. EXPERIMENTAL ANALYSIS

   1. Dataset

      To train and test our model, we collected a dataset of images of various plants and their corresponding diseases. The dataset named PlantLeaf was collected in real-life scenarios by our team under the supervision of plant pathologists, ensuring that it reflected the diverse range of conditions and situations that farmers and growers face when monitoring plant health. The PlantLeaf dataset consists of images of plants affected by

      Fig 2. Sample images from PlantLeaf dataset

      various types of diseases, including early leaf blight, rust & leaf spots like cercospora & tikka. The images were captured using

      high-quality cameras with different angles and lighting conditions to make the dataset diverse and challenging. We also made sure to capture images of healthy plants to serve as negative examples in the training process. The dataset was

      manually labelled by our teammembers, and each image was annotated with bounding boxes around the diseased areas using

      the Roboflow tool. The resulting dataset contains a total of 2,750 images, with approximately 1,900 images for training and 850 images for testing and validation.

   2. Annotation

      Image annotation involves the process of categorizing and labeling data in various formats, such as images, videos, or text files, to enable machines to comprehend and interpret the input data. The annotated dataset plays a crucial role in supervised Machine Learning (ML) as it serves as the foundation for training models, allowing machines to process the data and generate accurate results. The annotation process played a pivotal role in preparing the dataset for training the model. With the help of Roboflow, the project team meticulously annotated the leaf images, marking and labeling crucial regions associated with various leaf diseases. These annotations included precise delineation of disease spots, lesions, and affected areas, providing ground truth labels that facilitated the learning process of the YOLO v7 model. Roboflow's annotation tools ensured accuracy and consistency, enabling the model to recognize and classify leaf diseases with precision during inference.

      Fig 3. Statistics for PlantLeaf dataset

      Some images clearly depict single class, either healthy or diseased leaves, while some images have both healthy and diseased leaves in the same frame which results in a total of 6,630 annotatins in 2750 images.

   3. Training

      The training process of a custom YOLO v7 model involved several stages, as outlined below:

      1. Environment configuration: The initial step involved setting up the environment for YOLO by obtaining the YOLO v7 repository and installing the necessary plugins. This ensured that the programming framework was ready to execute instructions for object identification training and inference.

      2. Data acquisition and format: Real time images was collected to create new dataset and named it PlantLeaf. The dataset was uploaded to Roboflow for annotation.

      3. Pre-processing and augmentation: The uploaded data underwent pre-processing steps and augmentation, which were selected in Roboflow.

      4. Data splitting: Roboflow automatically divided the data into training, testing, and validation sets.

      5. Annotation format selection: After annotating the images, the YOLO v7 PyTorch format was chosen.

      6. Snippet: Roboflow provided a key or PIP package.

      7. Training on Google Collab: The model was trained using the free training environment provided by Google Collab, which likely operated on a Tesla P100 GPU (batch = 16, epoch = 150).

         The data.yaml file contained information about the exclusive dataset and the location of the YOLO v7 images. These steps outline the process involved in training a custom YOLO v7 models.

   4. Performance evaluation metrics

      To assess the effectiveness of the models, all the YOLO variants were tested on images that were not included in the training process. Precision, mean Average Precision (mAP), Loss, and Recall were employed as evaluation metrics in this study to conduct a comparative analysis. These metrics help assess the

      effectiveness and accuracy of the system in identifying and correcting leaf diseases.

      Accuracy: Accuracy measures how well the system correctly identifies and classifies leaf diseases. It is calculated by dividing the number of correctly identified diseases by the total number of samples.

      Precision: Precision determines the proportion of correctly predicted positive disease samples out of the total predicted positive samples.

      Precision = TP (1)

      TP + FN

      Recall: Recall (also known as sensitivity or true positive rate) measures the ability of the system to correctly identify positive disease samples out of the total actual positive samples.

      Recall = TP (2)

      TP + FP

      F1 Score: The F1 score is the harmonic mean of precision and recall and provides a single metric to evaluate the overall performance of the system.

      F1 = 2*Precision * Recall (3) Precision + Recall

      Confusion Matrix: A confusion matrix provides a tabular representation of the system's performance by comparing the predicted disease classes against the actual classes. It helps identify true positives, true negatives, false positives, and false negatives, allowing for a more detailed analysis of the system's strengths and weaknesses.

      Mean Average Precision (mAP): mAP is commonly used in object detection tasks to evaluate the accuracy of bounding box predictions. It considers precision at various levels of recall, and the average precision is calculated across multiple disease classes.

      N

      mAP = 1/NAP (4)

      i=1

   5. Results

   1. Detection & Classification

      The developed model demonstrated a high accuracy rate of 96% in accurately detecting and classifying leaf diseases. This accuracy metric showcases the effectiveness and reliability of the implemented methodology.

      Additionally, a comprehensive classification report was generated, offering a detailed assessment of the model's precision, recall, and F1-score for each disease class. This report provides valuable insights into the model's performance in

      terms of correctly identifying instances of leaf diseases and distinguishing them from healthy leaves.

      Fig 4. Classification Report

      Overall, the results obtained in this project highlight the effectiveness of YOLO v7 in leaf disease detection, with an impressive accuracy of 96%. The generated confusion matrix and classification report provide additional insights into the model's performance and can guide future improvements and optimizations.

      Fig 5. Images after disease detection

   2. Correction

   Furthermore, for the detected leaf diseases, we employed GPT- 3, a powerful language model, to provide correction methods for the detected diseases. By leveraging the natural language processing capabilities of GPT-3, we were able to generate suggested remedies and treatments for each identified disease.

   Using the detected disease labels as input, we utilized GPT-3 to generate text-based recommendations for farmers. The generated suggestions encompassed a range of corrective measures, including cultural practices, chemical treatments, and biological control methods. These recommendations aimed to assist in mitigating the impact of leaf diseases and promoting healthier plant growth.

   Fig 6. Correction methods suggested by GPT3 for ground nut tikka leaf spot

   By integrating the capabilities of both YOLO v7 for disease detection and GPT-3 for remedy suggestions, our project provided a comprehensive solution for leaf disease management. This combined approach enhances the practicality and usability of the system, enabling users to not only identify diseases accurately but also access valuable guidance on how to address them effectively.

3. FUTURE DIRECTON

   Dataset Expansion and Diversity: Increasing the size and diversity of the training datasets can improve the detection and correction capabilities of the integrated system. Collecting more labelled leaf images representing a wide range of plant species, diseases, and environmental conditions will enhance the model's generalization and accuracy.

   Real-time Monitoring and Mobile Applications: Expanding the integration to enable real-time monitoring of plant leaf diseases using mobile applications can provide on-the-spot detection and correction recommendations for farmers. This would facilitate rapid response and timely intervention, resulting in improved disease management and crop yield.

   Collaborative Learning and Model Updates: Establishing a collaborative platform where farmers, agricultural experts, and researchers can contribute labelled data and share their experiences can help improve the detection and correction models continuously.

   These advancements have the potential to revolutionize plant pathology practices, benefit farmers worldwide, and contribute to sustainable and efficient agricultural systems.

4. CONCLUSION

This paper introduces deep learning and reviews recent research on plant leaf disease recognition using deep learning. Integrating YOLO v7 and GPT-3 offers a promising solution for accurately identifying and correcting leaf diseases. YOLO v7 excels at detecting and localizing leaf diseases with high precision and recall. GPT-3 enhances the system by generating detailed descriptions, recommendations, and corrective measures. The integrated system automates disease identification, provides information, and offers actionable recommendations for disease management. It has the potential to enhance agricultural practices, reduce crop losses, and promote sustainable farming techniques. Farmers can input

images of diseased leaves, and the system detects diseases and provides reliable recommendations for correction. This system helps farmers quickly identify and treat diseases, potentially saving crops and increasing yields. The research showcases significant improvements in accuracy and holds potential for developing more precise plant disease detection models in the future.

REFERENCES

[1] A. B. H.-Y. M. L. Chien-Yao Wang, "YOLO v7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors"

[2] Morbekar, A., Parihar, A., & Jadhav, R. (2020, June). Crop disease detection using YOLO. In 2020 international conference for emerging technology (INCET) (pp. 1-5). IEEE.

[3] Mathew, M. P., & Mahesh, T. Y. (2022). Leaf-based disease detection in bell pepper plant using YOLO v5. Signal, Image and Video Processing, 1-7.

[4] Shill, A., & Rahman, M. A. (2021, July). Plant disease detection based on YOLO v3 and YOLO v4. In 2021 International Conference on Automation, Control and Mechatronics for Industry 4.0 (ACMI) (pp. 1-6). IEEE.

[5] Soeb, M. J. A., Jubayer, M F., Tarin, T. A., Al Mamun, M. R., Ruhad, F. M., Parven, A., … & Meftaul, I. M. (2023). Tea leaf disease detection and identification based on YOLO v7 (YOLO-T). Scientific Reports, 13(1), 6078.

[6] Reddy, M. P., & Deeksha, A. (2021). Mulberry leaf disease detection using YOLO.

[7] Ponnusamy, V., Coumaran, A., Shunmugam, A. S., Rajaram, K., & Senthilvelavan, S. (2020, July). Smart glass: real-time leaf disease detection using YOLO transfer learning. In 2020 International Conference on Communication and Signal Processing (ICCSP) (pp. 1150-1154). IEEE.

[8] Liu, J., & Wang, X. (2020). Tomato diseases and pests detection based on improved YOLO V3 convolutional neural network. Frontiers in plant science, 11, 898.

[9] Xue, Z., Xu, R., Bai, D., & Lin, H. (2023). YOLO-tea: A tea disease detection model improved by YOLO v5. Forests, 14(2), 415.

[10] Nayar, P., Chhibber, S., & Dubey, A. K. (2022, December). An Efficient Algorithm for Plant Disease Detection Using Deep Convolutional Networks. In 2022 14th International Conference on Computational Intelligence and Communication Networks (CICN) (pp. 156-160). IEEE.

[11] Cao, L., Zheng, X., & Fang, L. (2023). The Semantic Segmentation of Standing Tree Images Based on the YOLO V7 Deep Learning Algorithm. Electronics, 12(4), 929.

[12] Jhatial, M. J., Shaikh, R. A., Shaikh, N. A., Rajper, S., Arain, R. H., Chandio, G. H., … & Shaikh, K. H. (2022). Deep Learning-Based Rice Leaf Diseases Detection Using YOLO v5. Sukkur IBA Journal of Computing and Mathematical Sciences, 6(1), 49-61.

[13] Li, J., Zhu, X., Jia, R., Liu, B., & Yu, C. (2022, June). Apple-YOLO: A

Novel Mobile Terminal Detector Based on YOLO v5 for Early Apple Leaf Diseases. In 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC) (pp. 352-361). IEEE.

[14] Chau, D. H., Tran, D. C., Vo, H. N., Do, T. T., Nguyen, T. H., Nguyen,

B. Q., … & Nguyen, V. D. (2022, April). Plant Leaf Diseases Detection and Identification Using Deep Learning Model. In The 8th International Conference on Advanced Machine Learning and Technologies and Applications (AMLTA2022) (pp. 3-10). Cham: Springer International Publishing.

[15] Kumpala, I., Wichapha, N., & Prasomsab, P. (2022). Sugar Cane Red Stripe Disease Detection using YOLO CNN of Deep Learning Technique. Engineering Access, 8(2), 192-197.

[16] Mia, J., Bijoy, H. I., Uddin, S., & Raza, D. M. (2021, July). Real-time herb leaves localization and classification using YOLO. In 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT) (pp. 1-7). IEEE.

[17] Agbulos, M. K., Sarmiento, Y., & Villaverde, J. (2021, July). Identification of Leaf Blast and Brown Spot Diseases on Rice Leaf with YOLO Algorithm. In 2021 IEEE 7th International Conference on Control Science and Systems Engineering (ICCSSE) (pp. 307-312). IEEE.

[18] Son, C. H. (2021). Leaf spot attention networks based on spot feature encoding for leaf disease identification and detection. Applied Sciences, 11(17), 7960.

[19] Lin, J., Bai, D., Xu, R., & Lin, H. (2023). TSBA-YOLO: An improved tea diseases detection model based on attention mechanisms and feature fusion. Forests, 14(3), 619.

[20] Haque, M. E., Rahman, A., Junaeid, I., Hoque, S. U., & Paul, M. (2022). Rice leaf disease classification and detection using YOLO v5. arXiv preprint arXiv:2209.01579.

[21] Mathew, M. P., & Mahesh, T. Y. (2021, June). Determining the region of apple leaf affected by disease using YOLO V3. In 2021 International Conference on Communication, Control and Information Sciences (ICCISc) (Vol. 1, pp. 1-4). IEEE.

[22] Nasution, S. W., & Kartika, K. (2022). Eggplant Disease Detection Using Yolo Algorithm Telegram Notified. International Journal of Engineering, Science and Information Technology, 2(4), 127-132.

[23] Dai, G., Hu, L., & Fan, J. (2022). DA-ActNN-YOLO V5: Hybrid YOLO

v5 Model with Data Augmentation and Activation of Compression Mechanism for Potato Disease Identification. Computational Intelligence and Neuroscience, 2022.

[24] Hazarika, A., Sistla, P., Venkatesh, V., & Choudhury, N. (2022, June). Approximating CNN computation for plant disease detection. In 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC) (pp. 1117-1122). IEEE.

[25] Maski, P., & Thondiyath, A. (2021, July). Plant Disease Detection Using Advanced Deep Learning Algorithms: A Case Study of Papaya Ring Spot Disease. In 2021 6th International Conference on Image, Vision and Computing (ICIVC) (pp. 49-54). IEEE.