Fruit Recognition using Image Processing

Fruit Recognition using Image Processing

Ms. Anisha M Nayak Department of Computer Science and Engineering

Srinivas Institute of Technology, Valachil

Mr. Manjesh R Department of Computer Science

and Engineering Srinivas Institute of Technology,

Valachil

Ms. Dhanusha Department of Computer Science

and Engineering Srinivas Institute of Technology,

Valachil

Abstract: The ability to identify the fruits based on the quality in food industry is very important nowadays where every person has become health conscious. There are different types of fruits available in the market. However, to identify best quality fruits is cumbersome task. Therefore, we come up with the system where fruit is detected under natural lighting conditions. The method used is texture detection method, color detection method and shape detection. For this methodology, we use image segmentation to detect particular fruit. Fruit Detection project is implemented in MATLAB image processing toolbox. The project is implemented for both Real time and Non-Real time. The proposed method has four stages: First is Pre-Processing and second is Feature Extraction and third is Segmentation and fourth Recognition. In case of Non- Real time, the first stage is used to browse the image, second stage is extraction of the features from images using Grey Level Co-occurrence Matrix (GLCM), RGB and Color Histogram. System will convert the image from RGB to grayscale image for further processing. The color histogram represents the distribution of colors in an image. Since image is captured under different illumination condition. In the third stage, the three extracted image is obtained in the form of red, green and blue. In the fourth stage, the extracted features are used as input to Support Vector Machine (SVM) classifier. Then name of the fruit is output is obtained.

I.INTRODUCTION

Recognizing different kinds of vegetables and fruits is a difficult task in supermarkets, since the cashier must point out the categories of a particular fruit to determine its price. The use of barcodes has mostly ended this problem for packaged products but given that most consumers want to pick their products, they cannot be pre-packaged, and thus must be weighed. A solution is issuing codes for every fruit, but the memorization is problematic leading to pricing errors. Another solution is to issue the cashier an inventory with pictures and codes, however, flipping over the booklet is time consuming. Automatic classification of fruits via computer vision is still a complicated task due to the various properties of many types of fruits. The fruit quality detection technique which was based on external properties of fruits such as shape, size and color.

The proposed method is based on the use of Support Vector Machine (SVM) with the desirable goal of accurate and fast classification of fruits. Support Vector Machines (SVMs) is a classification method based on machine learning theory. SVMs have significant advantages because of their high accuracy, elegant mathematical tractability, and direct geometric interpretation. Besides, they do not need a large number of training samples to avoid overfitting. The task here is to automatically detect and classify the fruits image acquired from database. Assuming that the different images are present and some are overlapped on one another. The proposed work mainly gives a review that what steps are performed throughout the entire process to detect particular fruit. Since image is captured under different natural condition. The framework mainly consists of two phases. In the first phase textural features are extracted from fruit and in the second phase fruit is classified as detected fruit. The measurements obtained from the study of textural feature are given as input to the SVM classifier for training in order to classify it. Finally, system will detect objects and will display as an output. The objective of Fruit Recognition using image processing is to design a incremental model to recognize the fruits based on size, shape and colour of the fruit ignoring external features like environment, noise and background. This just focus the image of particular fruit and identify the fruit. An approach of classification using Support Vector Machine Classifier that has very good working efficiency produces the accurate results. The system helps to improve the performance. Maintaining the project is easy and manageable.

1. RELATED WORK

   In [1], they have recognized nine different classes of fruits. Fruit image dataset are obtained from web as well as certain images are acquired by using mobile phone camera. These images are pre-processed to subtract the background and extract the blob representing fruit. For representing fruits and capturing their visual characteristics, combination of color, shape and texture features are used. These feature datasets is further passed to two different classifiers multiclass SVM and KNN. The color image is firstly

   converted to grayscale by GLCM (Gray Level Co- occurrence Matrix). The image is further converted to binary image. Further, Morphological operations are used to fill the holes and extract the largest blob or object from the image which would further be considered as fruit. After that this largest blob is cropped and the binary values are replaced with original intensity values. From the experiments it can be concluded that the combination of color texture and shape gives better or comparable results in most of the cases than when any two categories of features are used. Also, the second conclusion which can be made is that KNN gives better results for this case than SVM.

   In [2], has different steps of the training process in this research which are as follows: Initially collect fruits image, then feature extraction process using FCH & MI method to get the characteristic of fruits image then transformed into vector feature form which will be stored in the database. Later clustering process is done using the K- Means Clustering method on the vector of the fruits image in the database. The steps of the testing process in this research are as follows: Open file image query to detect fruits. The next step is to get the feature of the face image then transformed into the vector feature form same as training process. Then, the process of recognition using the KNN method by calculating the distance between the new fruits image features and features of the existing on the database by using Euclidian distance which then matched with the clustering results.

   This paper [3] is based on the use of speeded up robust feature. The method extracts the local feature of the segmented image and describes the object recognition. The basic steps are to create a database of image to be classified. Then image pre-processing done by means of various image processing techniques to improve the quality of the image and later several filters are applied to de-noise the image. Finally, image classifiers are used for classification. Image is converted from RGB image to intensity image. Based on speeded up robust technique local feature is extracted and described. To characterize the texture of the input image, statistical measurement of randomness. Other features extracted such as object recognition, image registration, recognizing parameter and image retrieval. Objects and boundary lines of images are obtained by image segmentation. Then feature extraction like shape, size, color and texture of fruits are calculated using algorithm. Then for disease classification pattern matching is applied. The system also includes specific skin defect detection algorithms not only to locate them, but also to determine their distribution, which can affect to their assignment to a standard category.

   In this paper [4], two-dimensional fruit images are classified on shape and color based on analysis methods.

   Used a method to increase the accuracy of the fruit quality detection by using artificial neural network [ANN]. The first step is to get the image of fruit. Image of the fruit samples are captured by using regular digital camera with white background with the help of a stand. In the second step the image of the fruit is loaded into the matlab to include the feature extraction of each and every sample in dataset for training of neural network. In third step features of the fruit samples are extracted. In fourth step neural network is used for training the data. In fifth step fruit sample is selected for testing from database. In step sixth testing is performed by using ANN training module button. Finally, ANN based results are obtained where user has the option to select the sample of fruit which it wants to test and finally want to obtain it.

   In paper [5], Green and orange color fruits are selected under different lighting conditions. The steps of the process are Image Segmentation it means that the original image is divided into many parts. Its purpose is to cluster the pixels which have the same features on the image. In detection fruit, it is purposed to separate the fruits and background in the image. The method which has been preferred to use is k-means algorithm. K-means algorithm requires iteration. Firstly, clustering centres are determined and the data is clustered according to these centres. The centres can be valued of pixels which are selected randomly. There are 3 dominant colors in the image where the fruits are orange color, because of this, the number of cluster sets are 3. The green color ones are clustered 6-8 sets, because the dominant color is green in the image. Filtering operators get to delete some details which are unwanted or to be cleared according to the aim. By this process, the pixel values are calculated again. Determining of Round objects for finding easily the round objects, the image is transformed to binary image.

   This paper [6], presents a novel approach to fruit detection using deep convolutional neural networks. The task of fruit detection using image obtained from two modules: colour (RGB) and Near-Infrared (NIR). Methodology Fruit segmentation is an important step in order to distinguish the fruits from the background. This section, describes the DCNN approach, Faster R-CNN, which forms the basis of our proposed method Fruit Detection using a Conditional Random. The CRF uses both colour and texture features. The colour features are constructed by directly converting the RGB values to the HSV colour space. Visual texture features are extracted from the NIR channel. NIR images are used to calculate texture features. Three sets of visual texture features are used: (i) Sparse Auto-encoder (SAE) features (ii) Local Binary Pattern (LBP) and (iii) a Histogram of Gradients (HoG). Each feature captures a different property, such as the

   distribution of the local gradient, edges and texture, respectively. It uses Faster R-CNN using deep convolutional neural networks on large-scale image classification and detection. Fine-tuning consists of updating, or adapting, the model parameters using the new data. Late fusion combines the classification decisions from the two model. The VGG network is modified and adapted to receive RGB and NIR information simultaneously.

   In [7], the K-Nearest Neighbours algorithm is the methodology that has been used as a classifier to classify fruit based on mean color values, shape roundness value, area and perimeter values of the fruit. The area and perimeter values of fruit are estimated in term of pixels values, whereas the area of a fruit can be estimated by counting the total number of pixels that are enclosed by the detected fruit boundary area. The user selects a scalar value to resize the fruit image so that its size is approximately the size of fruit in real time environment. After selecting a desired scalar value, the algorithm will compute the new area and perimeter values for the fruit image. After inserting training data, system is ready. The Euclidean distance determines the distances between the features values of the test input fruit with stored training fruits. The recognition system that has been developed is able to recognize all the test fruit images which are being selected by user or system tester from the fruit selection menu on the system.

   This paper [8] elaborated the fruit quality and propose a novel classification method based on a multi-class kernel support vector machine (kSVM). First, fruit images were acquired by a digital camera, and then the background of each image was removed by a split-and-merge algorithm. Second, the color histogram, texture and shape features of each fruit image were extracted to compose a feature space. Third, principal component analysis (PCA) was used to reduce the dimensions of feature space. First, we use Image Segmentation with Spilt and Merge Algorithm which will remove the background area and based on a quadtree partition of an image. This method starts at the root of the tree that represents the whole image. If it is found inhomogeneous, then it is split into four son-squares. Conversely, if four son-squares are homogeneous, they can be merged as several connected components (the merging process). Then the histogram is merely the count of pixels that have each possible color. Third step is using Unsers Texture Features, Gray level co-occurrence matrix and local binary pattern are good texture descriptors. It eliminates the components in the data set that contributes the least variation.

2. PROPOSED WORK

   This system is built using various methods and features such as:

   RGB:

   It is also referred to as truecolor image which defines Red, Green and Blue color components for each individual pixel. This RGB array is of class double where each color component is a value between 0 and 1. This can be stored along the third dimension of data array.

   GLCM (Gray Level Co-occurrence Matrix):

   It is statistical method that examines the texture that considering the pairs of pixels with specific values. It mainly consists of statistic feature like contrast which measure the local variation, correlation which measure the joint probability, energy which provides the sum of squared elements and homogeneity which measures the closeness of the distribution.

   Color Histogram:

   It controls the appearance and behaviour of image. It converts color image into HSV image and preserves the hue and saturation components. The values are extracted and plotted in the graph. The intensity matrix is obtained from the HSI image matrix. This matrix is updated with histogram equalized intensity matrix.

   Color moments:

   Color moments are very much useful for color indexing purposes. It considers only the first three color moments as feature in image retrieval applications. It can be used to compare the two images based on color.

   HOG feature:

   The histogram of oriented gradients (HOG) is a feature used in vision and image processing for object detection. The image is divided into small connected regions called cells. Since it works on local cells, it is invariant to geometric transformations.

   HSV Feature:

   The Hue Saturation Value (HSV) represents the color, dominance of color and brightness. Therefore, the color detection algorithm can be used to search in terms of color position and color purity. It is used to detect the pixels.

   SVM (Support Vector Machine):

   It is a supervised learning algorithm which can used for binary classification or regression. It is a coordinate of individual observations. It is based on decision planes which defines decision boundaries. It also separated the set of objects having different class.

   The system is bult on two different environments namely using Real Time Fruit and Non-Real Time Fruit.

   Non-Realtime: Non-real time, or NRT, is a term used to describe a process or event that does not occur immediately. A non-real time system is one in which we cannot guarantee the response time of a task. These systems are non- deterministic in nature and we cannot predict the behaviour of the system with respect to time. For example, communication via posts in a forum can be considered non- real time as responses often do not occur immediately and can sometimes take hours or even days. In Non-Real time process begins with. training the data where features are extracted and stored in the dataset. Fruit image is then browsed. In the further process testing takes through pre- processing, image segmentation, Feature extraction and finally recognition

   1. Pre-processing: Pre-processing consist of image getting converted to grayscale and further gets converted into graythreshold, then to black and white image by removing the unwanted impurities.

      Figure.1: Pre-processing

   2. Image segmentation: Next process is the image segmentation were, the three images are obtained in the cluster format. The images obtained are R, G and B image as shown Figure 2. The cluster 1 image highlights the defective part of the fruit. The cluster 2 shows the entire fruit image removing its background. The cluster 3 shows the fruit where the fruit is displayed in bright color.

      Figure.2: Image Segmentation

   3. Feature Extraction: In machine learning, pattern recognition and in image processing, feature extraction starts from an initial set of measured data and builds derived values (features) intended to be informative and non-redundant, facilitating the subsequent learning and generalization steps, and in some cases leading to better human interpretations. Here an initial set of raw variables is reduced to more manageable groups (features) for processing, while still accurately and completely describing the original data set. Next process is extraction of the features from the selected fruit image. Here various features are taken into consideration like color, shape and texture. The different methods used are HOG (Histogram of Oriented Gradients) for shape feature, color histogram, color moments and HSV are for color feature, GLCM for texture feature. The obtained value from the GLCM texture features like contrast, homogeneity, energy, correlation, entropy, RGB, color moments and HSV histogram are stored in dataset.

      Figure.3: Feature Extraction

   4. Recognition: In this step the SVM algorithm is used to recognize the fruit by comparing the values from test and train dataset. The value obtained is used to determine the fruit which is examined. After this process SVM algorithm is applied and name of the fruit is displayed.

   RealTime: Real time process begins with capturing of fruit in real time by using a mobile application IP Webcam. Initially the training process takes place where images of one fruit is taken in different dimensions. In the further process testing takes through pre-processing, image segmentation, Feature extraction and finally recognition.

   1. Pre-processing: Pre-processing consist of image getting converted to grayscale and further gets converted into graythreshold, then to black and white image by removing the unwanted impurities.

      Figure.4: Pre-processing

   2. Image segmentation: Next process is the image segmentation were, the three images are obtained in the cluster format. The images obtained are R, G and B image as shown Figure 5. The cluster 3 shows the fruit where the fruit is displayed in bright color. The cluster 1 image highlights the defective part of the fruit. The cluster 2 shows the entire fruit image removing its background.

      Figure.5: Image Segmentation

   3. Feature Extraction: Next process is extraction of the features from the selected fruit image. Here various features are taken into consideration like color, shape and texture. The different methods used are HOG (Histogram of Oriented Gradients) for shape feature, color histogram , color moments and HSV are for color feature, GLCM for texture feature. The obtained value from the GLCM texture features like contrast, homogeneity, energy, correlation, entropy, RGB, color moments and HSV histogram are stored in dataset.

      Figure.6: Feature Extraction

   4. Recognition: In this step the SVM algorithm is used to recognize the fruit by comparing the values from test and train dataset. The value obtained is used to determine the fruit which is examined.

3. RESULT ANALYSIS

   The project has been tested at Srinivas institute of technology. The above pictures taken show the working of the system where it is able to recognize the fruits using image processing in real time as well as non-real time. The system is thereby able to do all required task which as we stated above. The output of both Non Realtime and Real time after recognition is given below:

   Figure.7: Recognition of fruit image

   Figure 7 shows the output of the Non Realtime fruit image after SVM method is applied in recognition.

   Figure.8: Recognition of fruit image

   Figure 8 shows the output of the Realtime fruit image after SVM method is applied in recognition.

4. CONCLUSION

The proposed project is able to recognize the fruit based on the features like shape, colour, and texture. This increases the knowledge of common people about some rare and unknown fruits. The project is mainly concentrating on reducing human effort and making human life easier. Fruit recognition will be able to reduce the current ongoing problems. It reduces confusion among the particular fruit.

Future work that can be added to this project may be the development of a web app. Here the user can use this application anytime anywhere.

REFERENCES

1. Deepika Shukla Comp. Science and Engineering Department, Institute of Technology, Nirma University, Ahmedabad, Recognition of Fruits Using Hybrid Features and Machine Learning2016 International Conference on Computing, Analytics and Security Trends (CAST)College of Engineering Pune, India. Dec 19-21, 2016

2. Erwin, M. Fachrurrozi, Ahmad Fiqih, Bahardiansyah Rua Saputra, Rachmad Algani, Anggina Primanita Informatics Engineering, Universitas SriwijayaPalembang, Content Based Image Retrieval for Multi-Objects Fruits Recognition using k-Means and k-Nearest Neighbour,2017 International Conference on Data and Software Engineering (ICoDSE).

3. Yogesp, Ashwani Kumar Dubey2, AIIT, Amity University Uttar Pradesh, Noida, India, Fruit Defect Detection Based on Speeded Up Robust Feature Technique 2016 5th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO) (Trends and Future Directions), Sep. 7-9, 2016.

4. Mandeep Kaur, M tech Student Department of Electronics and Communication, Punjabi University, India, Reecha Sharma, Assistant Professor, Department of Electronics and communication Engineering, Punjabi university Patiala, India. Journal of Electronics and communication Engineering Quality Detection of fruits by using ANN technique, IOSR-JECE,Jul- Aug 2015.

5. Bengi OZTURK, Murvet KIRCI, Ece Olcay GUNES, Department of Electronics and communication Engineering, Istanbul, Detection of green and orange color fruits in outdoor conditions for robotic applications, 1911-1920

6. Inkyu Sa *, Zongyuan Ge, Feras Dayoub, Ben Upcroft, Tristan Perez and Chris McCool. DeepFruits: A Fruit Detection System Using Deep Neural Networks

7. Woo Chaw Seng, Faulty of Computer Science and Information Technology University of Malaya, A New Method for Fruits Recognition System,03 June 2014.

8. Yudong Zhang and Lenan Wu, School of Information Science and Engineering, South, www.mdpi.com/journal/sensors Classification of Fruits Using Computer Vision and a Multiclass Support Vecor Machine, 16 July 2012;in revised form 7 September 2012/