Recognition of Fruits in Fruits Salad Based on Color and Texture Features

Recognition of Fruits in Fruits Salad Based on Color and Texture Features

Abstract:

In this paper, we propose a methodology for recognition and classification of fruits in fruits salad image samples. The samples of different fruits like apple, chikku, banana, orange and pineapple are considered. Each sample of fruits are sliced into pieces and placed on the tray. The RGB color features extracted from the images from the knowledge base. A K mean classifier is proposed and has the classification efficiency of around 98%. The work finds application in restaurants, malls, motels and the like where the service robots automatically serve the food.

Keywords: color features; knowledge-based classifier; fruit juice.

1.0 INTRODUCTION

Object recognition is one of the challenging problems in the area of digital image processing. Particularly, the objects like fruits in fruits salad and their recognition and classification is a challenging task. Fruits in fruits salad are one of the food items consumed by humans on regular basis. The fruits salads are prepared by using the fruits like apple, chikku, banana, orange, etc., by cutting the fruits in small slices and placed on the tray. Presently, recognition of fruits in fruits salads is carried out manually in restaurants, malls, etc which is labour intensive job. The scarcity of labour encourages the development of an automated method for recognition of fruits in fruits salads in restaurants, cafeteria, malls, etc. We propose in this paper, a methodology for recognition and classification of fruits in fruits salads using a K mean classifier. In order to know the technology applications in this area, we have carried out literature survey. It revealed that certain connected works being carried in the area of computer technology in food processing.

(Valerie Davidson et al, 1999) have described quality assessment of chocolate chip cookies based solely on visual

features. Digital images were used to define physical characteristics of cookies produced on a commercial bakery line. Consumers were asked to rate typical cookies on a line scale. A number of fuzzy logic systems have been developed to assist quality control decisions based on features extracted from digital images. Results from two fuzzy logic systems are compared to consumer results for validation.

(Wang et al, 2002) investigated meltability and browning properties of cheddar and Mozzarella cheeses under different cooking condition and different sizes of samples using machine vision. Cheese shred dimensions were determined from skeletonised images using syntactic networks. This technique was successful in recognizing individual shred when two were touching or overlapping and results compared well with manual measurements.

(Margarita Osadchy,et al, 2004) have developed a model for detection of objects with a Lambertian surface under varying illumination and pose. They have applied a novel detection method that proceeds by modeling the different illuminations from a small number of images in a training set, which automatically voids the illumination effects, allowing fast illumination invariant detection, without having to create a large training set. In the experiments, an object is correctly detected under image plane rotations in a 45° range, and a wide variety of different illuminations, even when significant shadows were present.

(Cheng-Jin Du et.al 2005) have developed an automated classification system of pizza sauce spread using color vision and support vector machines (SVM). To characterize pizza sauce spread with low dimensional color features, a sequence of image processing algorithms was developed. The image was transformed from red, green, and blue (RGB) color space to hue, saturation

(Bin Zhu et al, 2007) have developed an automated inspection of apple quality based on geometric or statistical features. This

paper introduces a Gabor feature-based kernel principal component analysis (PCA) method by combining Gabor wavelet representation of apple images and the kernel PCA method for apple quality inspection using near-infrared (NIR) imaging. First, Gabor wavelet decomposition of whole apple NIR images was employed to extract appropriate

Gabor features. Then, the kernel PCA method with polynomial kernels was applied in the Gabor feature space to handle non-linear separable features.

(Anam B.S et.al 2009) have proposed a method for identification and classification of boiled food grains with their level of boiling. Different types of color features are used to develop the knowledge based classifier.

From the published research work, it is observed that color and texture features are being widely used in recognition and classification task. Work connected with fruits in fruit salad recognition and classification is not much observed and hence the motivation for taking up the work on recognition and classification of fruits in fruits salad.

1. Proposed methodology

   The proposed methodology consists of four phases, namely, image acquisition, feature extraction, knowledge base creation and the development of a classifier. The block diagram of the proposed methodology is shown in Figure 1. The details of each phase are given as under.

   Figure 1. Block diagram of proposed methodology.

   Knowledge base system

   Acquisition of fruits salad images

   Image segmentation

   Classification using Knowledge base system

   Results

   Figure 1. Block diagram of proposed methodology.

   1. Image acquisition

      The fruits salads are prepared by cutting the different fruits into small slices and placed on the tray. A Sony Alpha digital color camera with an image resolution of 10 mega pixels is used for capturing images of fruits salad. The camera is vertically oriented and the distance of 0.5 m from the surface of the fruit salad is maintained. The different fruits and their scientific names are apple (Malus pumila), chikku (Manilkara zapode), mango (Mangifera indica), orange (Sitrus sinesis) and pineapple (Aninas comosus). The images of the of different fruits salad are as shown in Figure 2. We have considered maximum of 500 image samples by drawing 200 images of each type contain different type of fruit slices. The color features are extracted from the images to create a knowledge base and further a k mean classifier.

      1 2 3 4 5

      NOTES: 1.(chikku,tomoto,capsicum,mango) 2.(tomoto,capsicum,mango) 3.(chikku,capsicum,mango,banana) 4.(chikku,capsicum,mango) 5.(chikku,capsicum,banana)

      Figure 2. Fruits salad image samples

      The fruits have certain colors by which human beings identify them in the real world as the same color is retained upon dilution. Hence, we have used color features based on RGB Model. The RGB components are separated from the original images.

   2. Image Segmentation

The color information helps obtain the texture information of the target image while the edge extraction detects the boundary of the target image. By combining these, the target image can be correctly segmented and represent.

Divisive methods are usually modified by using some form of summaryof a cluster to suggest a good split. A natural summary to use is a histogram of pixel colors (or grey levels). In one of the earliest segmentation algorithms, regions are split by identifying a peak in one of nine feature histograms (these are color coordinates of the pixel in each of three different color spaces eg:R G B) and attempting to separate that peak from the histogram. Texture regions need to be masked to avoid splitting texture components apart.

Natural objective function can be obtained by assuming that we know there are k clusters, where k is known. Each cluster is assumed to have a center; we write the center of the ith cluster as ci. The jth element to be clustered is described by a feature vector xj as in equation 2.1.

.2.1

In demosaicking technique a luminance image is formed and the red, green and blue component images are interpolated in a conventional way by equation 2.2

.2.2

The merging procedure works at each scale and at each position, the wavelet coefficients of each band of the color image are modified according to the corresponding wavelet coefficients of the luminance image by equation 2.3

2.3

The max rule replaces the coefficients from the color band by the maximum of the coefficients from the luminance and the color band by equation 2.4

2.4

Finally obtained color wavelet coefficients are inversely wavelet transformed to obtain the demosaicked RGB image as shown in the figure.

a b c1 c2 d

(a) Original Image. (b) De-noised Gray scale.

(c1) Under Segmentation. (c2) Final Segmentation.

(d) Final extracted.

3.1Feature Extraction

The important component of the present work is to extract the meaningful features, which characterize the texture aspect of the fruits and vegetables. We have used the following two types of features. 1) Texture features based on Co-Occurrence Matrix. 2)Color features based on R.G.B Model.

The texture measures based on Gray Level Co-occurrence Matrices (GLCM) are used in this method. Let a two- dimensional image I(x, y), (x =1… M and y = 1… N), have Ng gray levels. A co-occurrence matrix depicts the joint gray- level histogram of the image (or a region of the image) in the form of a matrix with the dimensions of NgxNg.

Suppose Pd(i, j) denotes the cardinality of the set of pairs of points that have gray level values of i and j, for a displacement vector d = (dx, dy).

Pd(i, j) = |{((r, s), (r + dx, s + dy)) : I(r, s) = i, I(r + dx, s + dy) = j}|

Where (r, s) M x N, and |.| is the cardinality of a set

matrix.

Figure 3.1 Region for texture feature extraction similar lines the matrices in the other six directions namely, 900, 1350, 1800, 2250, 2700, 3150 and 3600 are also computed. Symmetrical GLCM is generated by pooling frequencies of

pairs of gray levels occurrences at separation positive or The corresponding sample statistic is the third k-statistic,

negative distance (d).

The following are the equations used for computing the average of R, G, B components of an image( I) and w and h

k3

n

n 1n 2

x x 3 .(3.7)

represent width and height of an image respectively,

The corresponding computational formulas are

I(x, y) – the pixel of image I at row y, column x,R(p), G(p), B(p) – the red, green and blue color component of pixel p, ra, ga, ba – the average red, green and blue component of image

Ia respectively,d(Ia,Ib) – the distance measure between image

1 x

3

N

3 3 x

2 2N 3

and

(3.4)

Ia and Ib

The equations are

k n

3 n 1n 2

x3 3x

x 2 2nx 3

….. (3.9)

x w, y h R(I (x, y))

If we want to compare shapes, we need measurements that

R=

x 1, y 1

w h

(3.2)

will not change if we multiply all values by a constant. Such a measure would be called the coefficient of relative skewness,

x w, y hG(I (x, y))

with the formulas

G=

x 1, y 1

w h

.. (3.3)

3 and g k3

x w, y h B(I (x, y))

1 3

1 s3 … (3.10)

B=

x 1, y 1

w h

(3.4)

Note that for the Normal distribution 1 0 . Other measures of skewness are Pearson's measures of skewness,

In color based classification, we consider the average A and B component present in an image.

SK1

mean mod e

std.deviation &

Average color = (intensity of all pixels in the current block)

(total pixels in the block)

.(3.5)

The output of this procedure would be a region matrix, of 30×30 (for 10×10 block or 37×37 for 8×8) size, with 1 in the areas corresponding to the presence of color match and 0 in the areas without color match.

The usual measurement of skewness is often called the third moment about the mean (The population variance is the second). The formula for population skewness is:

x 3

3mean median

SK 2

std.deviation

…. (3.11)

Table3.1: knowledge base used for classification of different fruits slices

3 N . (3.6)

4.0 K-Mean Classification

The nearest mean classifier considers the linear set of values as the training sets. These can be directly the image matrix or the frequency transformed data or some other information. Training process groups these values in a set and labels them as the classes. The input image features would be extracted and their closeness with all the training classes would be compared. Whichever class values are closer to the features (maximum match of minimum deviation from the respective class values) is detected as the matching class for the image.

Template matching can easily be expressed mathematically. Let x be the feature vector for the unknown input, and let m1, m2, …, mc be templates (i.e., perfect, noise- free feature vectors) for the c classes. Then the error in matching x against mk is given by || x – mk || . Here || u || is called the norm of the vector u. A minimum-error classifier computes || x – mk || for k = 1 to c and chooses the class for which this error is minimum. Since || x – mk || is also the distance from x to mk, we call this a minimum-distance classifier. Clearly, a template matching system is a minimum- distance classifier.

The kth class Wkis represented by its mean vector Mkand covariancematrix which can be estimated from the training samples.

.. (4.1)

.. (4.2)

The Nearest Mean Classifier is used to classify unknown image data to classes which minimize the distance between the image data and the class in multi-feature space. The distance is defined as an index of similarity so that the minimum distance is identical to the maximum similarity. Figure shows the

concept of a minimum distance classifier. The following distances are often used in this procedure.

The Normalized Euclidian distance is proportional to the similarity in dex, as in the case of difference variance.

.. (4.3)

A given pattern X of unknown class is classified to Wk if its Normalized Euclidian distance to Wk is smaller than those to all other classes:

… (4.4)

For simplicity, the distance DL(X,Mi) can be used to replace DM(X,Wi) above. As now only the mean vector of each class is used, the classification does not take into account how the classes are distributed in the feature space.

5.0 Results and discussion

The different types of fruits slices are used to test the developed K-Mean classifier. The overall efficiency of recognition & classification of fruits in fruit salad is found to be 80%. The color feature values of some fruits overlap and attribute to the reduction in recognition and classification efficiency. The efficiency for certain fruit salad images is found to be more than 90%. This has shown that reduction in efficiency is also due to quality of input images. This developed methodology is implemented using MATLAB 7.2 running on 3.0 GHz core 2 deo processor. The efficiencies of classification of different types of fruit salads are given Fig 22.

100

0

1 2 3 4 5 6 7 8 9 10

Figure 5.0 .Feature variations

6.0 Conclusion

This paper has reviewed different computer vision approaches for the detection of fruits and vegetables by using CFA. A description of approach is presented, emphasizing the kind of methodology used. An complex level analysis and detection strategy applied to detect the food, and the performance in terms of correct and false detections as well as its sensibility to changes in texture, color, brightness, contrast, smoothness and other image properties. The performance reported depends on the global strategy chosen and the particular features of each considered application.

The image manipulations refer to bidirectional reflectance factor values (BRF) for solar channels and equivalent brightness temperature values (EBT) for infrared channels. The enhancement operations expand to the full range of display values (0-255, BYTE) a limited range (MIN, MAX) of BRF1) or EBT values, applying subsequently a gamma correction2) (GAMMA) for possible non linear expansion.

References

Anami, B.S. and Burkpalli, V.C. (2009) Color based

identification and classification of boiled food grain images,

Journal of Food Engineering, Vol. 5, No. 5, Article 7.

De Silva, L.C., Pereira, A. and Punchihewa, A. (2005) Food classifications using color imaging,Conference on Image and

Vision Computing New Zealand (ivcnz), University of Otago, Dunedin, pp.16.

Dettmar, H.P., Barbour, G.S., Blackwell, K.T., Vogl, T.P.,

Alkon, D.L., Fry, F.S., Jr., Totah, J.E.

and Chambers, T.L. (1999) Orange juice classification with a biologically based neural network, Journal of Computers & Chemistry, Vol. 20, No. 2, pp.261266.

Du, C-J. and Sun, D-W. (2005) Pizza sauce spread classification using color vision and support vector machines, Journal of Food Engineering, Vol. 66, No. 2, pp.137145.

Fernandez, L., Castillero, C. and Aguilera, J.M. (2005) An

application of image analysis to dehydratation of apple discs, Journal of Food Engineering, Vol. 7, Nos. 12, pp.187193. Goodner, K.L., Rouseff, R.L. and Hofsommer, H.J. (2001) Orange, mandarin, and hybrid classification using

multivariate statistics based on carotenoid profiles, J. Agric. Food Chem.,

Vol. 49, No. 3, pp.11461150.

Jezek, J. and Suhaj, M. (2001) Application of capillary

isotachophoresis for fruit juice authentication, Journal of Chromatography A, 4 May, Vol. 916, Nos. 12, pp.185189. Kelly, J.F.D. and Downey, G. (2005) Detection of sugar adulterants in apple juice using Fourier transform infrared spectroscopy and chemometrics, J. Agric. Food Chem., Vol. 53, No. 9,

pp.32813286.

Lana, M.M., Tijskens, L.M.M. and van Kooten, O. (2006) Modeling RGB color aspects and translucency of fresh-cut

tomatoes, Postharvest Biology and Technology, Vol. 40, No. 1,

pp.1525.

Lana, M.M., Tijskens, L.M.M., de Theije, A., Hogenkamp, M. and van Kooten, O. (2006) Assessment of changes in optical properties of fresh-cut tomato using video image analysis, Postharvest Biology and Technology, Vol. 41, No. 3, pp.296 306.