Development of Detection, Counting and Yield Estimation Algorithm for Agricultural Products

Development of Detection, Counting and Yield Estimation Algorithm for Agricultural Products

Neetu Mee1na Dr. Navneet Aggarwal2 Dr. Sunil Joshi 3 Dr.Suriti Gupta 4

1. ech Student Assistant Professor Professor & Head Assistant Professor Department of ECE Department of ECE Department of ECE Department of ECE

   CTAE, MPUAT CTAE, MPUAT CTAE, MPUAT CTAE, MPUAT

   Udaipur, India Udaipur, India Udaipur, India Udaipur, India

   Abstract Computer vision methods are helpful for automatic counting of fruits on trees. In this paper we present automatic detection, counting and yield estimation algorithm for fruits on the basis of color and shape analysis in the field. Firstly, preprocessing was applied on input fruit tree images and then it was converted from RGB to L*a*b color space to detect the fruit region from its background. Otsus method was used to create the image segmentation. Morphological operation was applied on binary images to remove noise. Then fruits were extracted by region labeling. Edge detection technique was applied on the labeled images to detect the edges of fruit region. On this edge detected image, circular fitting algorithm was applied for automatic counting of fruits. We used different types of fruits (apple, Tangerine/orange, pomegranate, lemon, peach, plum) for automatic counting. The simulation outputs show that new counting algorithm is found to be suitable and gives better accuracy than Newtons method [17] for yield estimation. It gives 88.73% accuracy for apple, 93.93% for Tangerine/Orange, 81.36% for pomegranate, 84.43% for Lemon, 83.14% for peach, 70.18% for plum fruits and gives average yield measurement error is 16.3%.

   Keywords – Counting algorithm, Edge detection, L*a*b color space.

   1. INTRODUCTION

      Yield prediction of fruit/vegetable through automatic counting in practical environment is one of the hardest and significant tasks to obtain better results in crop management system to achieve more productivity with regard to moderate cost. Yield estimates can provide valuable information for forecasting yields and generating prescription maps for tree- specific application. In the present work yield estimation of fruits is a process to find the total number of fruits in a tree automatically. For successful application of effective control methods, the yield information about individual tree is a prerequisite in real time.

   2. LITERATURE REVIEW

      Yield estimation of fruits/vegetable is a significant task to obtain better results in agriculture field. Many researchers have developed the algorithm for fruit detection and counting. The author in [3] conducted a preliminary step for

      developing real time spectral based system to identify green citrus fruits in an image. It was determined that reflectance ratio at wavelength at 815 nm and 1190 nm could correctly distinguish green citrus fruits from leaves.An automatic machine vision system is present with two charge coupled device (CCD)cameras, ultrasonic sensors, an encoder and differential Global positioning System(GPS)receiver to estimate citrus yeild.An alternative computer vision algorithm was proposed to recognize visible and partially occluded citrus fruit from trees.The average fruit size was determined from image using ulrasonic sensors measuring a distance between the cameras and the fruit laden tree.Finally,a citrus yeild map was created to show yeild variability for site-specific crop managment in [4]. Machine vision based citrus yield mapping and fruit quality inspection system was developed by the authors in [6]. The use of color and texture image processing together to detect the wheat ears, before to propose and compare different texture image segmentation techniques based on feature extraction by first and higher order statistical methods. The extracted features are used for unsupervised pixel classification to obtain the different classes in the image, before to use the k means algorithm. Three methods tested with very heterogeneous results, except the run length technique for which the results are close to the manual counting (66% error) in [8]. The author in [17] has worked on efficient locating the fruit on the tree is one of the major requirements for the fruit harvesting system. The average yield measurement error was found as 31.4%.The developed method gives better valid output for tangerine flower detection in natural outdoor lighting, with different lighting condition without any alternative lighting source to control the luminance. The simulation result reveals that the method is reliable, feasible and efficient compared to other existing methods in [19].The use of mathematical morphological in the CIE L*a*b color space. The weighting functions assign a lower weight to color vectors near the colors with maximum chroma and higher weight to color vectors near the lightness axis in [2]. A novel framework for combining all the three i.e. color, texture and shape information, and achieve higher retrieval efficiency using dominant color feature. The combination of the color, shape and texture features between image and its complement in conjunction with the shape features provide a robust feature

      set for image retrieval. The experimental results demonstrate the efficiency of the method in [20]. The study of segmentation image techniques by using five threshold methods as Mean method, P-tile method, (HDT), (EMT) and visual Technique and they are compared with one another so as to choose the best technique for threshold segmentation techniques image in [11].Edges characterize boundaries and are therefore a problem of fundamental importance in image processing. Image Edge detection significantly reduces the amount of data and filters out useless information, while preserving the important structural properties in an image. It has been observed that Cannys edge detection algorithm is computationally more expensive compared to LOG (Laplacian of Gaussian), Sobel, Prewitt and Roberts operator in [9]. Application of image processing in determination of apple quality and automatic algorithm is proposed in order to determine apples skin color defects. First, this image is converted from RGB to color space L*a*b*. Then fruit shape is extracted by ACM algorithm. Finally, the image has segmented using SHEM algorithm. Experimental results on the acquired images show that both EM and SHEM spend the same iterations to accomplish the segmentation process and same results are obtained. However, the proposed SHEM algorithm consumes less time than the standard EM algorithm in [16]. An automated method that uses computer vision to detect and count grape berries. The method could potentially be deployed across large vineyards taking measurements at every vine in a non-destructive manner. Berry detection uses both shape and visual texture and demonstrates detection of green berries against a green leaf background. Berry detections are counted results and comparison has also been made to show the efficiency of the numerical integration algorithms. It is found that the RK-Embedded Heronian Mean outperforms well in comparison with the RK- Embedded Centroidal Mean, Harmonic Mean and Contra- Harmonic Mean by author in [13]. In Agriculture the automatic methods for counting the number of fruits play a critical role in crop management. The various computer vision and optimization techniques are presented to automate the process of counting fruits by author in [21]. The author in

      [18] understands the fundamental concepts of various gradient filters apply in identifying a shark fish type which is taken as a case study. In this paper the edge detection tchniques are taken for consideration. The author in [8] was investigates the use of digital image analysis techniques for developing an automated kiwifruit counting system. Three simple counting methods followed by a minimum distance classifier based segmentation technique in L*a*b color space were studied. Tree canopy mapping with an automated ultrasonic system is inexpensive, fairly straightforward and could be used to estimate fruit yield within a grove to plan site-specific management practices. In specific, a detailed discussion on detecting edges for any kind of source images based on single layer / raster scheme and time-multiplexing under cellular neural network paradigm using new fourth order four stage numerical concepts, strategies, theory, and formulations in [5].

      After study of above papers we develop an idea of detection, counting and yield estimation algorithm which shown in a

      Fig 1.We had taken trees sectional images of different kinds of fruits from different camera distances then filtering these images by Gaussian filter[12]. Filtered image converted into L*a*b color space. Using new circle fitting method we developed automatic counting and yield estimation algorithm.

   3. MATERIALS AND METHODS

      The image processing and analysis is performed using MATLAB R2013a software with image processing toolbox. MATLAB is a high-level technical computing language and provide interactive environment for algorithm development, data visualization, data analysis, and numerical computation. MATLAB code can be integrated with other languages and applications, and distribute the MATLAB algorithms and applications.

      Start

      Image Capturing &Pre-Processing

      Convert image RGB to L*a*b space

      Generate binary image

      Remove noise

      Extract the fruit

      Label the pixels

      Edge detection

      Circle fitting

      Agriculture products counting

      Comparison between manually & algorithmically counting

      Fig 1: Flowchart of proposed Algorithm

      1. Image Capturing and Preprocessing

         We had taken some field images of different fruit tree from internet. Gaussian filter use for noise removes from input image. Then new counting algorithm is applied on these images and compare between manually and algorithmically counting. Test image of plum tree shown in Fig 2.

         Labeling is used for extract the fruit region and finds the connected components on Black and white images. Labeled image is shown in Fig7.

         Fig 2: Original image of plum tree Fig 3: Gaussian filtered image

      2. RGB to L*a*b Conversion

      1.5 Edge detection

      Fig7: Labeled image

      Filtered input image is converted in L*a*b color space for detect fruits from its background. By use of L*a*b color space segment the fruit regions with its perceptually uniform property. Fruits detected image is shown in Fig 4.

      Edge detection technique is detecting the edges of labeled fruit region. Sobel and canny operator [18] is best for the edge detection but canny is expensive then sobel, so we use sobel operator for the edge detection. This operator calculates the gradient of the image intensity at each point. Edge detected image is shown in Fig8.

      Fig4:a image

      1.3 Generate binary image

      Otsus method used for generate the binary image after l*a*b conversion, which chooses the threshold to minimize the intra

      1.6 Circle fitting

      Fig8: Edge detected image

      class variance of the black and white pixels where, nonzero pixels belong to an object and 0 pixels constitute the background. Binary image shown in Fig5.Then morphological operation is applied to remove noise from binary image. Noise removed binary image shown in Fig 6.

      Fig 5: Binary image Fig 6: noise remove binary image

      1. Region labeling

         Edge detected images are used for circle fitting which is implemented using the following step:

         1. Find the size of edge detected image and labeled each edge pixels.

         2. Detect the boundary points of the fruits. Each boundary point has two values (x co-ordinate and y co-ordinate).

         3. Make a vector XY which stores co-ordinates of the edge points.

         4. Find the number of data points (n).

         5. Calculate centroid of the data set by calculating mean of XY vector, which is a two column vector. First column stores the y-co-ordinates and second column stores the x co-ordinates of all the boundary points. XY=[y x].

         6. Fix the distance (a) of each edge points from the centroid.

            a = sqrt [(x1-x) 2+ (y1-y) 2)]

            x1, y1 are centroid and x, y are coordinates of each edge points.

         7. Find radius(R) by calculating the mean of distance a.

         8. Then by angle theta (0 to 360 degree) and radius R, get new points (xnew, ynew) and plot them. These points are superimposed on the fruit region and get the image with circle fitted on fruits of input image.

      1.7 Yield estimation and Quality parameters

      The proposed method is use the shape analysis for the yield measurement. Yield measurement is a process to estimate the total fruits in a tree in few seconds. Yield estimation is done by help of trees sectional images and new counting algorithm. The average yield measurement error is calculated by the quality parameters. Percentage Accuracy and Mean absolute percentage error (MAPE) are quality parameters of this method which are defined as:

      % Accuracy = Algorithmically counting × 100

      Manual counting The mean absolute percentage error

      MAPE = 1 |Mi-Ai| × 100 %

      N |Mi|

      Here, N=total no. of fruits, i=1to N M=Manual counted value A=algorithmically counted value

   4. RESULTS AND DISCUSSION

      We had taken six types of fruit tree images. These images are trees sectional images for yield estimation by new counting algorithm. After implementation of proposed method we get final resultant circle fit image which shown in Fig 9.

      Fig 9: circle fit image

      Calculated values of accuracy and mean absolute percentage error are individually mentioned for each kind of fruit tree in table1. Accuracy comparison between proposed method and Newtons method [17] has shown in Fig10.This chart shows

      that accuracy of proposed algorithm better than Newtons method [17].

      Table1: quality Parameters for different fruit trees

      S. N	Fruit Trees	Mean absolute %error (MAPE) (propose d method)	Mean absolute % error (MAPE) Newtons method [17]	Overall accurac y of propose d method (%)	Overall accuracy of Newtons method [17]
      1.	Apple	11.25%	56%	88.73%	43.79%
      2.	Tangerine/ orange	6.05%	13.5%	93.93%	86.42%
      3.	Pomegranate	18.6%	29.6%	81.36%	77%
      4.	Lemon	15.5%	60.5%	84.43%	39.43%
      5.	Peach	16.8%	18.7%	83.14%	81.23%
      6.	Plum	29.8%	40%	70.18%	60%

      Fig 10 : Comparison of proposed method and Newtons method[17]

      Table 2: Comparison between Manual and Algorithm counting

      Trees sectional images	Manual counting	Counting by Proposed Method	Counting by Newtons method[17]
      Apple 1	17	13	14
      Apple 2	16	14	(Newton-Pratt will not coverage)
      Apple3	10	10	(Newton-Pratt will not coverage)
      Apple4	7	7	6
      Apple5	19	13	18
      Apple6	13	13	(Newton-Pratt will not coverage)
      Tangerine1	7	7	7
      Tangerine2	10	9	8
      Tangerine3	10	10	8
      Tangerine4	7	6	6
      Pomegranate1	13	10	11
      Pomegranate2	8	8	7
      Pomegranate3	2	2	(Newton-Pratt will not coverage)
      Pomegranate4	10	7	9
      Pomegranate5	10	6	10
      Pomegranate6	16	13	16
      Lemon1	13	13	(Newton-Pratt will not coverage)
      Lemon2	5	3	4
      Lemon3	9	7	7
      Lemon4	23	23	(Newton-Pratt will not coverage)
      Peacp	6	6	5
      Peacp	14	9	10
      Peacp	5	4	4
      Peach4	7	5	5
      Peacp	6	6	6
      Plum1	5	2	3
      Plum2	10	9	9
      Plum3	11	10	(Newton-Pratt will not coverage)
      Plum4	10	8	10
      Plum5	6	3	3

      Fig 11: Comparison between Manual and proposed algorithm counting

      Table2 shows number of fruits counted by observer, Proposed algorithm and counted by Newtons method [17]. Graphical comparison between manually, proposed algorithm counting and counting by Newtons method [17] is shown in Fig11.

   5. CONCLUSION

In this paper, we have developed the new counting algorithm for yield estimation of fruits. Table1shows that the proposed method gives better accuracy than Newtons method [17]. It gives less mean absolute percentage error and high accuracy for all the mentioned fruits. The proposed method is accurate 88.73% for apple, 93.93% for Tangerine/orange, 81.36% for pomegranate, 84.43% for lemon, 83.14% for peach and 70.18% for plum fruit. From the proposed method we found average yield measurement error is 16.3%.

ACKNOWLEDGMENT

Authors would like to thank Department of Electronics and Communication Engineering of College of Technology and Engineering, MPUAT, Udaipur.

REFERENCES

1. D.C. Slaughter and L.F Tian, Environmentally adaptive segmentation algorithm for outdoor image segmentation, Computers and Electronics in Agriculture, 1998, vol. 21, pp 153- 168.

2. A. Hanbury and Serra, Mathematical morphology in CILAB space, 2002, vol. 21, pp 201-206.

3. P. Annamalai and W.S. Lee, Identification of green citrusfruits using spectral characteristics, SAE section meeting presentation, 2004, paper no.FL04-1001.

4. R.Chinchuluun, W.S. Lee and T.F.Burks, Machine Vision-Based

   citrus yield mapping system. Proc. Fla.

   State Hort. Soc, 2006, vol. 119, pp 142-147.

5. Q.U. Zaman and A.W., Schumann, Estimation of citrus fruit yield using ultrasonically sensed tree size, American Society of agricultural and Biological Engineers, 2006, vol. 22, pp 39-44.

6. R. Chinchuluun, Citrus yield mapping system on a canopy shake and catch harvester, ASABE Annual International Meeting Presentation, 2007, pp 78-79.

7. F. Cointault and P. Gouton, Texture or color analysis in agronomic images for wheat ear counting. Third International IEEE Conference on Signal Image Technologies and Internet- Based System, 2008, pp 696-701.

8. P.Wijethunga, S. Samarasinghe, D. Kulasiri and I. Woodhead, Digital Image Analysis Based AutomatedKiwifruitCounting Technique, 2008.

9. R. Maini and Dr. H.Aggarwal, Study and Comparison of Various Image Edge Detection Techniques, International Journal of Image Processing (IJIP), 2009, vol. 3, pp 43-48.

10. W.C Seng. and S., H Mirisaee, A New Method for Fruits Recognition System, 2009, pg no. 130 -134.

11. S.D Khamitkar, S.S. Al-amri, and N.V., Kalyankar, Image Segmentation by Using Threshold Techniques Journal of computing, 2010, vol.2, pp 2151-9617.

12. Salem Saleh Al-amri, Dr. N.V. Kalyankarand, Dr. S.D Khamitkar, A Comparative Study of Removal Noise from Remote Sensing Image, IJCSI International Journal of Computer Science Issues, 2010, Vol. 7, ISSN: 1694-0814.

13. R. Ponalagusamy and, S. Senthilkumar, Investigation on time- multiplexing cellular neural network simulation by

    RKAHeM (4, 4) technique, International Journal of Advanced Intelligence Paradigms, 2011, vol.3, pp 43-66.

14. S. Sengupta and W. S.Lee, Identification and determination of the number of green citrus fruit under different ambient light conditions, 2011, pp 137-145.

15. S. Nuske and S. Achar, Yield estimation in vineyards by visual grape detection, IEEE/RSJ International Conference on Intelligent Robots and Systems San Francisco CA. USA, September 25-30, 2011, pp 227-234.

16. G.Moradi, M. Shamsi, M.H. Sedaaghi and S. Moradi, Using statistical histogram based EM algorithm for apple defect detection American Journal of Signal Processing, 2012, vol. 2, pp 10-14.

17. H.N.Patel,R.K.Jain and M.V.Joshi, Automatic Segmentation and Yield Measurement of Fruit using Shape Analysis, International Journal of Computer Applications May 2012, Vol 7, and ISSN: 0975 8887.

18. G.T. Shrivakshan, Dr. C. Chandrasekhar, A Comparison of various Edge Detection Techniques used in Image Processing, IJCSI International Journal of Computer Science Issues, September 2012, Vol. 9, Issue 5, ISSN: 1694-0814.

19. U.-O. Dorj, M. Lee, and S. Han, A counting algorithm for tangerine yield estimation, 7th International conference on information security and assurance, 2013, vol. 21, pp 279- 282.

20. N.Jain, S. Sharma and R. Sairam, Content base image retrieval using Combination of Color, Shape and Texture Features. International Journal of Advanced Computer Research 2013, vol. 3, pp 70-77.

21. A. Syal Garg, and S. Sharma, A Survey of Computer visionmethods for Counting Fruits and Yield Prediction, 2013, vol.2, pp 2319-7323.