Automatic Fruit Classification and Freshness Detection

Automatic Fruit Classification and Freshness Detection

Anna Biju Kuriakose

Department of Computer Science and Engineering

Ilahia College of Engineering and Technology

Muvattupuzha, India

Feba Siju

Department of Computer Science and Engineering

Ilahia College of Engineering and Technology

Muvattupuzha, India

Merin K Aji

Department of Computer Science and Engineering

Ilahia College of Engineering and Technology

Muvattupuzha, India

Thasli Rahim

Department of Computer Science and Engineering

Ilahia College of Engineering and Technology Muvattupuzha, India

Chithra Rani P R

Department of Computer Science and Engineering

Ilahia College of Engineering and Technology Muvattupuzha, India

Abstract- Fresh fruit supply chain includes classification and detection of fresh fruits. It is a growing field all over the world due to demand for fresh fruit. Now it is more needed because of the increase selling of vegetables and also for protecting our health from such toxic based chemicals. Meanwhile transportation and distribution of fresh fruit suffers from spoilage and wastage. In this Scenario such disadvantages can be avoided by continuous monitoring of fresh fruit using this. Fresh fruit get easily spoiled by fruit borne diseases. This can be overcome using this method and monitoring during transaction approach using smart logistics. The purpose of the work mainly focused on reduce or remove the wastages.

Keywords- Deep Learning (DL); Convolution Neural Network (CNN); VGGnet; YOLOv3

1. INTRODUCTION

   Automatic fruit recognition and freshness detection system with audio output by using machine vision is introduced. Here, fruit recognition algorithm based on Convolution Neural Network (CNN) is proposed. Deep learning is an advanced machine learning which aims to achieve high accuracy in image recognition and classification. This system can be used to predict the type of fruit by building a convolution neural network. Convolutional Neural Networks (CNNs) is the most popular neural network model being used for image classification tasks. We use VGGnet which is CNN architecture, to detect type of image. Here, the dataset consists of Apple types (Apple A, Apple B, Apple C, Apple D, Apple E, Apple F). A method for detecting the freshness of apple is also included. After detecting the apple type, we use YOLOv3 a deep neural network model to detect freshness of apple. YOLOv3 is pre trained model for image segmentation. We use this model for detecting freshness of fruit. The output of the system will in the form audio.

2. RELATED WORKS

   In [1], Youhak lee, Chulhee lee and Hyuk-Jae lee.Here

   proposed an improved YOLOv3-based neural network for De- identification technology. The existing YOLOv3 is a network with fast speed and performance recently. Most surveillance system using CCD cameras simultaneously store images from cameras installed in multiple locations. In such an environment, the use of deep learning requires a method of detecting objects through a single inference engine in a plurality of image. If the inference engine hardware is used for each camera channel, the cost of building a surveillance system increases significantly. Therefore, in the field of surveillance systems, a network structure with a high detection speed is required even if the detection performance is slightly degraded. This paper proposes a method to increase the detection speed by reducing the existing YOLOv3 network Architecture.53 feature extractors, Darknet-53, are reduced to 24 layers.

   In [2], Jin-Sung Kimast object detection is important to enable a vision-based automated vending machine. This paper proposes a new scheme to enhance the operation speed of YOLOv3 by removing the computation for the region of non-interest. In order to avoid the accuracy drop by a removal of computation, characteristics of a convolutiona layer and a YOLO layer are investigated, and a new processing method is proposed from experimental results. As a result, the operation speed is increased in proportion to the size of the region of non-interest. Experimental results show that the speed is improved by

   3.29 times while the accuracy degradation is 2.81% in mAP- 50.

   In [3], Saad Albawi,Tareq Abed Mohammed. The term Deep Learning or Deep Neural Nework refers to Artificial Neural Networks (ANN) with multi layers .Over the last few decades, it has been considered to be one of the most powerful tools, and has become very popular in the literature as it is able to handle a huge amount of data. The interest in having deeper hidden layers has recently

   begun to surpass classical methods performance in different fields; especially in pattern recognition. One of the most popular deep neural networks is the Convolutional Neural Network (CNN). It take this name from mathematical linear operation between matrixes called convolution. CNN have multiple layers; including convolutional layer, non-linearity layer, pooling layer and fully-connected layer.

   In [4], According to Dingan Liao, Hu Lu,Xingpei Xu, Quansheng Gao. Image segmentation is an important technique in image analysis. Existing methods in image segmentation rely on an artificial neural network to extract the feature of the image. In this study, we propose an image segmentation method based on deep learning features and community detection. We propose the use of a pre-trained convolution neural network (CNN) to extract the deep learning features of the image

   In [5], According to Shuying Liu, Weihong Deng.Since Krizhevsky won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012 competition with the brilliant deep

   convolutional neural networks(D-CNNs), re-searchers have designed lots of D-CNNs.However, almost all the existing very deep convolutional neural networks are trained on the giant ImageNet datasets. Small datasets like CIFAR-10 has rarely taken advantage of the power of depth since deep models are easy to overfit. In this paper, we pro-posed a modified VGG-16 network and used this model to fit CIFAR-10. By adding stronger regularizer and using Batch Normalization, we achieved 8.45% error rate on CIFAR-10 without severe overfitting.

   In [6], According to F.A. Ghisi , A.L.da Silva.This paper concludes that despite the vast distribution of technology throughout the industry.The producers who are able to acquire

   and utilize the new and innovative technologies have been able to build long term relationships.

   In [7], According to Antonio Lazzaro , Marti Boada [7] A battery-less Near Field Communication(NFC) tag is used for grading the ripeness of fruit.The measurement of different samples obtained on different days is used to train the classification algorithms.

   In [8], According to Amitangshu , Krishna kant.The integrated IoT-based online monitoring approach using smart logistics can address the critical needs of reduc-ing food waste.Further

   advances are needed to derive actionable intelligence from the col-lected data in real-world conditions.

   In [9], According to Siyuan Lu , Zhihai Lu , Soriya Aok. It proposed to use CNN for fruit classification.Here three state of the art approaches are there:voting based support vector machine,wavelet entropy,genetic algorithm.

   In [10], According to PL.Chithra , M.Henila [10] Here images having white background alone can only be considered for testing this algorithm. Using KNN classifier. Sample images should be acquired at 360degs in order to obtain 100% accuracy.

3. PROPOSEDSYSTEM

   Fig. 1. The Proposed System

   In the proposed system we are using VGGnet architecture for detecting type of apple from image. First of all, VGGnet architecture will be trained using fruit images and the model will be saved. After that YOLOv3 model will be loaded with pre-trained weights. Then we will have two models, one of detecting type of apple and the other for detecting freshness of the apple. Then when we input image of the apple, the image will be preprocessed and then VGGnet model will be used to predict type of apple. Then the image will be passed to YOLOv3 model for detecting the freshness of the apple. After that the output of both models will be converted to audio.

   1. Data-Preprocessing

      Data pre-processing is a data mining technique that involves transforming raw data into an understandable format. Real-world data is often incomplete, inconsistent, lacking in certain behaviors or trends, and is likely to contain many errors. The data is gathered by Training Type detection model Apple images and corresponding labels are load Image augmentation takes place. Train VGGnet architecture with images and labels. Image is preprocessed.

   2. Model Creation

      Load and trained the VGGnet model and then use VGGnet model to predict type of apple. YOLOv3 model is loaded and Load pre-trained weights to the YOLOv3 model. Use YOLOv3 model to predict freshness of preprocessed image. A model represents what was learned by a machine learning algorithm. Here mainly GUI creation takes place. GUI will be created using tkinter Option to choose image Output will be displayed in the same window.

   3. Analysis And Testing

      Software testing will be one of the most critical factors that determine the success of a machine learning system. In machine learning systems, humans provide desired behavior as examples during training and the model

      optimization process produces the logic of the system. A typical software testing suite will include:

      The unit tests which operate on atomic pieces of the codebase and can be run quickly during development, regression tests replicate bugs that we've previously encountered and fixed. The integration tests which are typically longer-running tests that observe higher-level behaviors that leverage multiple components in the codebase.

   4. Prediction

      Prediction refers to the output of an algorithm after it has been trained on a historical dataset and applied to new data when forecasting the likelihood of a particular outcome. The algorithm will generate probable values for an unknown variable for each record in the new data, allowing the model builder to identify what that value will most likely be. Use YOLOv3 model to predict freshness of preprocessed image.

   5. Overview

      The main goal of this system is to classify fruits and to detect the freshness of various fruits. By going through various stages of process we made a system. At initial stage we preprocess the fruit images and resize it. Then it split to train and to test the images.

      Then the process of image augmentation and architecture creation takes place. VGGnet (Visual Geometry Group), The original purpose of VGG's research on the depth of convolutional networks is to understand how the depth of convolutional networks affects the accuracy and accuracy of large-scale image classification and recognition.

      At the second stage we trained the classification model. And also, by using the fruit images we trained the VGGnet and then we detect the freshness of the fruit. By loading the YOLOv3 model (DNN model) The output is in the form of Audio. The accuracy of machine learning algorithm is highly depended on quality and quantity of data used for training.

      1. MODULES

         The main modules in the proposed system are:

         1. Data Preprocessing

            Initially, at this stage we pre-process the fruit images according to the preference. The datasets contained the images of different fruits. Here we use 5 types of apples (A, B, C, D, F). Therefore, the datasets were passed through the following preparation stages: Preprocessing methods such as data cleaning, transformation of variables, data integration and attribute selection had to be applied.

         2. Data Integration

            Data Integration means to gather the data from the multiple sources into single repository. Redundancy is the common problem occurred when integrating data. In this step, multiple files are integrated into one file.

         3. Data Cleaning

            In this phase, missing and noisy data is handled to achieve data consistency. The dataset occupied by this study not have any missing and outliers etc.

         4. Data Transformation

            Data transformation is the process of changing the format, structure, of data. We predict the data and after resize it. Then Split to train and test images.

         5. Training Model

            This is the stage where the algorithm is trained by feeding datasets. Training the model. Here we Train the VGGnet using the fruit images. This is the stage where the learning takes place. In addition, we detect the freshness of the fruits by loading YOLOv3 model (DNN model). The weights of the model must be initialized randomly. This way the algorithm will learn to adjust the weights accordingly. The training model is used to run the input data through the algorithm to correlate the processed output against the sample output. The result from this correlation is used to modify the model. save the model.

         6. Prediction

   Prediction refers to the output of an algorithm after it has been trained on a historical dataset and applied to new data when forecasting the likely hood of a particular outcome. Load model and predict the apple type and then detect whether the apple is fresh or not using YOLOv3 model and the output of the system is an audio

   1. RESULTS AND DISCUSSION

      This is the first display view of our result. Here we can see that the labels upload image and result. And also there are two icons with named classify and freshness in order to classifying the fruits and detect the freshness.Here we upload the image we want to know and the corresponding result may appear on the other screen.

      Here we can see the image that we are upload from the dataset. And check the freshness then we can identify that the fruit we upload is an undamged fruit and the freshness of the fruit are labeled there.

      Here too we are uploaded an damaged fruit and they shows as damged fruit detected.

   2. CONCLUSION

One of the essential and most challenging issues for the supplying of fresh fruits is highly demanded. We are trying to solve this problem by using this system. The prediction of freshness of the fruits are done here.

*In order to save our health from the chemicals applied in the grocery field we introduced a system by using certain process and algorithms here.

*The project is aimed to create new and efficient system.

*The system will detect fruit type and freshness from images.

*The system will detect fruit type and freshness from images.

ACKNOWLEDGMENT

Apart from the effort of us, the success of this project preliminary report depends largely on the encouragement and guidelines of many other. We take this opportunity to express my gratitude to the people who have been instrumental in the successful completion of this project. We would like to show my heartfelt gratitude towards Prof. DR ABDUL GAFUR M, Principal , Ilahia College Of Engineering and Technology for granting me the permission to work this project .Also, we would like to show my greatest gratitude towards our head of department of computer science & engineering Dr. Arun E and project guide Ms. Chithra Rani P R aND project coordinator Joyal Ulahannan, Meenu Varghese, Neha Beegam P E for their valuable advice and guidance. Finally , We express my gratitude and thank to all our teachers and other faculty members of the department of computer science & engineering, for their sincere and friendly cooperation in completing this project.

REFERENCES

1. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In Computer VisionECCV 2014, pages 818833. Springer, 2014.

2. C.Szegedy,W.Liu, Y.Jia, P.Sermanet, S.Reed, D. Anguelov,D.Erhan, V.Vanhoucke, and A.Rabinovich. Going deeper with convolutions. arXiv preprint arXiv:1409.4842,2014. 1

3. R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 580587. IEEE, 2014. 1

4. I. J. Goodfellow, D. warde-Farley,M. Mirza, a. Courvile, and Y .

   Bengio, Maxout network.ICML,2013.4

5. Mirza Muhammad Ali Baig, Mian Ihtisham Shah, Muhammad Abdullah Wajahat, Image Caption Generator with Novel Object Injection IEEE 2018

6. Rakshith Shetty, Marcus Rohrbach, Lisa Anne Hendricks,Mario Fritz, Bernt Schiele, Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training IEEE 2017

7. Bo Dai,Sanja Fidler, Raquel Urtasun, DahuaLin,Towards Diverse and Natural Image descriptions via a Conditional GAN IEEE 2017

8. Chuang Gan, Zhe Gan, Xiaodong He, Jianfeng Gao, Li Deng, StyleNet: Generating attractive Visual Captions with Styles IEEE 2017.

9. Jiasen Lu , Caiming Xiong , Devi Parikh, Richard Socher, Knowing When to Look: Adaptive Attentionvia A Visual Sentinel for Image Captioning IEEE 2017.