Integration of Color and Texture Features for Content Based Image Retrieval

Integration of Color and Texture Features for Content Based Image Retrieval

K . Guru Shravani R . Siva

1. ech II Year Assistant Professor

   Dept.Of CSE Dept.CSE

   Annamacharya Institute of Tech,and Science Annamacharya Institute of Tech., and science., (AITS), Tirupati, A.P., India (AITS), Tirupati, A.P., Indi

   Abstract This paper presents a new image indexing and retrieval algorithm by combining the color (RGB histogram) and texture feature (local derivative patterns (LDPs). Texture feature, LDP extracts the high-order local information by encoding various distinctive spatial relationships contained in a given local region. Color features, histogram extracts the distribution of various colors in an image. The experimentation has been carried out for proving the worth of our algorithm. It is further mentioned that the database considered for experiment is Corel 1000 databased. The results after being investigated show a significant improvement in terms of their evaluation measures as compared to LDP, RGB histogram.

   Keywords-Local Derivative Patterns; Feature Extraction; Local Binary Patterns; Image Retrieval; Histogram.

   1. INTRODUCTION

      1. Motivation

         Digital image libraries and other multimedia databases have been dramatically expanded in recent years. Storage and retrieval of images in such libraries has become a real demand in industrial, medical, and other applications. Content-based image indexing and retrieval (CBIR) is considered as a solution. In such systems, in the indexing algorithm, some features are extracted from every picture and stored as an index vector. The CBIR utilizes the visual contents of an image such as color, texture, shape, faces, spatial layout etc. in order to represent and index the image. The visual features can further be classified into general features which include color, texture and shape and domain specific features as human faces and finger prints. There is no single best representation of an image for all perceptual subjectivity, because the user may take the photographs in different conditions (view angle, illumination changes etc.). Learning of high level semantic concepts is a challenging task for CBIR systems. Comprehensive and extensive literature survey on CBIR is presented in [1][4].

         Swain et al. proposed the concept of color histogram in 1991 and also introduced the histogram intersection distance metric to measure the distance between the histograms of images [5]. Stricker et al. (1995) used the first three central moments called mean, standard deviation and skewness of each color for image retrieval [6]. Pass et al. (1997) split the each histogram bin into two parts called a color coherence vector (CCV) [7]. CCV partitions the each bin into two types, i.e., coherent, if it belongs to a large uniformly colored region

         or in coherent, if it does not. Huang et al. (1997) used a new color feature called color correlogram [8]. Color correlogram characterizes not only the color distributions of pixels, but also spatial correlation of pair of colors. Lu et al. (2005) proposed color feature based on vector quantized (VQ) index histograms in the DCT domain. They computed 12 histograms, four for each color component from 12 DCT-VQ index sequences [9].

         Texture is another salient and indispensable feature for CBIR. Smith et al. used the mean and variance of the wavelet coefficients as texture features for CBIR [10]. Moghaddam et al. proposed the Gabor wavelet correlogram (GWC) for CBIR [11, 12]. Ahmadian et al. used the wavelet transform for texture classification [13]. Moghaddam et al. introduced new algorithm called wavelet correlogram (WC) [14]. Saadatmand et al. [15, 16] improved the performance of WC algorithm by optimizing the quantization thresholds using genetic algorithm (GA). Birgale et al. [17] and Subrahmanyam et al. [18] combined the color (color histogram) and texture (wavelet transform) features for CBIR. Subrahmanyam et al. proposed correlogram algorithm for image retrieval using wavelets and rotated wavelets (WC+RWC) [19].

      2. Related Work

         The recently proposed local binary pattern (LBP) features are designed for texture description. Ojala et al. proposed the LBP [20] and these LBPs are converted to rotational invariant for texture classification [21]. pietikainen et al. proposed the rotational invariant texture classification using feature distributions [22]. Ahonen et al. [23] and Zhao et al [24] used the LBP operator facial expression analysis and recognition. Heikkila et al. proposed the background modeling and detection by using LBP [25]. Huang et al. proposed the extended LBP for shape localization [26]. Heikkila et al. used the LBP for interest region description [27]. Li et al. used the combination of Gabor filter and LBP for texture segmentation [28]. Zhang et al. proposed the local derivative pattern for face recognition [29]. They have considered LBP as a nondirectional first order local pattern, which are the binary results of the first-order derivative in images.

      3. Main Contribution

      To improve the retrieval performance in terms of retrieval accuracy, in this paper, we color (RGB histograms) and texture features (LDP histograms). The experimentation has been carried out on Corel database for proving the worth of our algorithm. The results after being investigated shows a significant improvement in terms of their evaluation measures as compared to LDP and RGB histogram techniques.

      The organization of the paper as follows: In section I, a brief review of image retrieval and related work is given. Section II, describes the collection of color feature, Section III, presents a concise review of Local Binary Patterns. Section IV, presents the local derivative patterns and proposed system framework. Experimental results and discussions are given in section V. Based on above work conclusions are derived in section V.

   2. RGB COLOR HISTOGRAM

The color histogram [5] is obtained by counting the number of times each color occurs in the image array. Histogram is invariant to translation and rotation of the image plane, and change only slowly under change of angle of view.

1. LOCAL DERIVATIVE PATTERNS (LDP)

   1. Local Derivative Patterns (LDP)

      Baochang Zhang et al. proposed the LDP operator for face recognition [29]. In this scheme, LBP is conceptually regarded as the nondirectional first-order local pattern operator; because LBP encodes all-direction first-order derivative binary result while LDP encodes the higher-order derivative information which contains more detailed discriminative features that the first-order local pattern (LBP) cannot obtain from an image.

      Given an image I, the first-order derivatives along 00, 450, 900 and 1350 directions are denoted as where I ' , where =00,

      c

      p

      450, 900 and 1350. Let g be a center point in I, and g ,

      A color histogram H for a given image is defined as a vector

      p=1,2,,8 be the neighboring point around

      gc . The four

      H H[0], H[1],….., H[i],….H[N]

      (1)

      first-order derivatives at

      gc can be written as:

      where i represent the color in color histogram and H[i] represent the number of pixels of HSV color i in the image,

      and N is the number of bins used in color histogram. For

      I ' (g ) I (g ) I (g );

      00 c c 1

      I ' (g ) I (g ) I (g );

      0

      (5)

      (6)

      45 c c 2

      comparing the histogram of different sizes, color histogram should be normalized. The normlized color histogram is

      given as

      I ' (g ) I (g ) I (g );

      900 c c 3

      I ' (g ) I (g ) I (g );

      (7)

      (8)

      1350 c c 4

      H The second-order directional LDP,

      LDP2 (g )

      , in

      H ' (2)

      p

      c

      direction at gc is defined as

      where p is the total number of pixels in the image.

      LDP2 (g ) f (I ' (g ), I ' (g )), f (I ' (g ), I ' (g )),……

      In this paper, RGB color space is used i.e. histogram for each color channel is used as feature for image retrieval.

      c c 1

      c 2

      c

      8

      ……., f (I ' (g ), I ' (g ))

      (9)

      III. LOCAL BINARY PATTERNS (LBP)

      The LBP operator introduced by Ojala et al. [20] as shown in Fig. 1. For given a center pixel in the image, a LBP value

      where f(.,.) is a binary coding function determining the types of local pattern transitions. It encodes the co-occurrence of two derivative directions at different neighboring pixels as

      is computed by comparing it with those of its neighborhoods:

      0,

      if I ' (g ) * I ' (g ) 0

      P 1

      f (I ' (g ), I ' (g ))

      c 1

      LBP

      2i f (g

      • g )

      (3)

      c p 1

      if I ' (g ) * I ' (g ) 0

      (10)

      P, R p c

      i 0

      c 1

      0

      f (x) 1

      x 0

      x 0

      (4)

      p 1, 2, 8

      The more details of the LDP is available in [29].

      The uniform LBP/LDP pattern refers to the uniform

      where gc is the gray value of the center pixel, gi is the gray value of its neighbors, P is the number of neighbors and R is the radius of the neighborhood. Fig. 2 shows the examples of circular neighbor sets for different configurations of (P, R) .

      appearance pattern which has limited discontinuities in the circular binary presentation. In this paper, the pattern which has less than or equal to two discontinuities in the circular binary presentation is considered as the uniform pattern and remaining patterns considered as non-uniform patterns.

      Fig. 3 shows all uniform patters for P=8. The distinct values

      for given query image is patterns.

      P(P 1) 3 by using uniform

      Fig. 1: LBP calculation for 3×3 pattern

      After identifying the LP (LBP/LDP) pattern of each pixel

      (j, k), the whole image is represented by building a histogram:

      N1 N2

      S P, R

      H (l) f (LPu 2 ( j, k),l); l [0, P(P 1) 3]

      j 1 k 1

      (11)

      f (x, y) 1

      0

      x y othrwise

      (12)

      where the size of input image is N1 N2 .

      Fig. 2: Circular neighborhood sets for different (P,R)

   2. Proposed System Framework

      In this paper, we proposed the new technique by combining color and texture features for image retrieval. The

      algorithm for the proposed image retrieval system is given below:

      Algorithm:

      Input: Image; Output: Retrieval results.

      1. Load the input image.

      2. Separate the RGB spaces.

      3. Construct the histogram on R, G and B spaces respectively.

      4. Convert RGB image into gray scale.

      5. Perform the first order derivatives along 00, 450, 900 and 1350 directions.

      6. Calculated the second order LDPs in 00, 450, 900 and 1350 directions using Eq. (9).

      7. Calculate the LDP histograms in 00, 450, 900 and 1350 directions using Eq. (11).

      8. Form the feature vector by concatenating the both LDP and color histograms.

      9. Calculate the best matches using Eq. (15).

      10. Retrieve the number of top matches.

   3. Similarity Measurement

   In the presented work four types of similarity distance metric ares used as shown below:

   Manhattan or L1 or city-block Distance

   This distance function is computationally less expensive than Euclidean distance because only the absolute differences in each feature are considered. This distance is sometimes called the city block distance or L1 distance and defined as

   Fig. 3: Uniform patters when P=8. The black and white dots represent the bit values of 1 and 0 in the S_LP operator.

   Fig. 4: Sample images from Corel 1000 (one image per category)

   D(Q,T ) i

   fi (Q) f j (T )

   (13)

   Euclidean or L2 Distance

   For p=2 in the equation (1.1) give the Euclidean distance and defined as:

   D(Q,T ) i

   fi (Q) f j (T ) 2 1 2

   (14)

   The most expensive operation is the computation of square root.

   D1 Distance

   Lg fT ,i fQ,i

   1 f

   D(Q,T )

   i 1

   Canberra Distance

   T ,i fQ,i

   (15)

   Lg fT ,i fQ,i

   D(Q,T ) (16)

   i 1 fT ,i fQ,i

   where Q is query image, Lg is feature vector length, T is

   image in database;

   fI ,i is

   ith

   feature of image I in the

   Q,i

   database,

   f is ith feature of query image Q.

   Fig. 5: Comparison of proposed method with LBP in terms of: (a) & (c) Average retrieval precision, (b) average retrieval rate according to no. of top matches considered

2. EXPERIMENTAL RESULTS AND DISCUSSIONS For the work reported in this paper, retrieval tests are

   conducted on Corel 1000 and results are presented in the following sections.

   A. Corel 1000 Database

   Corel database [30] contains large amount of images of various contents ranging from animals and outdoor sports to natural images. These images are pre-classified into different categories of size 100 by domain professionals. Some researchers think that Corel database meets all the requirements to evaluate an image retrieval system, because of its large size and heterogeneous content. In this paper, we collected the database DB1 contains 1000 images of 10 different categories (groups G). Ten categories are provided in the database namely Africans, beaches, buildings, buses, dinosaurs, elephants, flowers, horses, mountains and food. Each category has 100 images (NG 100) and these have either 256384 or 384 256 sizes. Fig. 4 depicts the sample images of Corel 1000 image database (one image from each category).

   The performance of the proposed method is measured in terms of average precision and average recall by Eq. (17) and

   (18) respectively.

   Hist. in terms of average retrieval precision and recall respectively. From Table I, it is clear that the proposed method shows better performance as compared to LDP and RGB Hist. in terms of average retrieval precision and recall. Table II and Fig. 5 (a) provides the comparison between proposed method and other methods (LDP and RGB Hist.) in terms of average retrieval precision. From Table II and Fig. 5 (a), it is clear that the proposed method outperforms the other methods. Table III and Fig. 5 (b) illustrate the comparison between various methods in terms of average retrieval rate. From Table III and Fig. 5 (b), it is clear that the proposed method outperforms the other methods. Table IV and Fig. 5

   (c) provides the performance of proposed method using various distance measures. From Tables IV and Fig. 5 (c), it is found that the d1 distance is outperforming the other distance measures. Fig. 6 illustrates the retrieval results of proposed method on Corel 1000 database.

   TABLE I RESULTS OF ALL TECHNIQUES IN TERMS OF PRECISION AND

   	Precision (%)			Recall (%)
   Category	LDP	RGB Hist.	PM	LDP	RGB Hist	PM
   Africans	62.4		87.4	38.12	47.24	57.63
   Beaches	58.8	49.5	64.7	36.18	22.45	38.2
   Buildings	74	53.9	74.6	36.46	26.24	37.78
   Buses	97.5	57.7	95.2	74.21	39.39	73.18
   Dinosaurs	96.1	99.9	99.3	77.16	96.51	92.1
   Elephants	53.5	68.3	75.8	28.5	36.14	41.2
   Flowers	90.1	83.4	93.1	62.23	47.72	67.41
   Horses	78.7	89.9	89.9	44.26	40.88	47.74
   Mountains	39.6	34.2	48.7	24.59	17.12	26.77
   Food	83.6	76	88.3	47.97	35.57	48.7
   Total	73.43	65.91	81.7	46.96	40.92	53.07

   RECALL ON COREL 1000 DATABASE

   Precision P(I , n) No.of Relevant Images Retrieved

   (17)

   q

   Total No.of Images Retrieved

   Recall R(Iq , n)

   No.of Relevant Images Retrieved Total No.of Relevant Images in Database

   (18)

   where Iq is the query image and n is number of top matches considered.

   Table I summarize the retrieval results of the proposed method (RGB Hist.+LDP Hist.), LDP Hist. (LDP) and RGB

   TABLE II RESULTS OF VARIOUS TECHNIQUES IN TERMS OF AVERAGE RETRIEVAL PRECISION ON COREL 1000 DATABASE

   	10	20	30	40	50	60	70	80	90	100
   LDP	73.43	66.76	63	60.1625	57.87	55.5983	53.4343	51.2613	49.08	46.968
   RGB Hist.	69.51	62.09	57.49333	53.9425	50.954	48.4467	46.3029	44.2638	42.56889	40.926
   PM	81.7	76.21	72.13667	68.565	65.642	62.7767	60.2443	57.8238	55.45333	53.071

   TABLE III RESULTS OF VARIOUS TECHNIQUES IN TERMS OF AVERAGE RETRIEVAL RATE ON COREL 1000 DATABASE

   	10	20	30	40	50	60	70	80	90	100
   LDP	7.343	13.352	18.9	24.065	28.935	33.359	37.404	41.009	44.172	46.968
   RGB Hist.	6.951	12.418	17.248	21.577	25.477	29.068	32.412	35.411	38.312	40.926
   PM	8.17	15.242	21.641	27.426	32.821	37.666	42.171	46.259	49.908	53.071

   TABLE IV RESULTS OF PROPOSED METHOD WITH DIFFERENT DISTANCE MEASURES IN TERMS OF AVERAGE RETRIEVAL PRECISION ON COREL DATABASE

   	10	20	30	40	50	60	70	80	90	100
   Manhattan	74.63	68.205	63.77	60.34	57.42	54.96	52.81	51	49.23	47.45
   Canberra	81.59	76.055	71.99	68.43	65.49	62.62	60.08	57.64	55.31	52.92
   Euclidean	67.38	60.025	55.77	52.55	50.09	48.12	46.29	44.59	42.72	40.99
   d1	81.7	76.21	72.13	68.56	65.64	62.77	60.24	57.82	55.45	53.07

3. CONCLUSIONS

A new image indexing and retrieval algorithm is proposed in this paper by combining color (RGB histogram) and texture (LDP). The experimentation has been carried out on Corel database for proving the worth of our algorithm. The results after being investigated show a significant improvement in terms of their evaluation measures as compared to LDP and RGB histogram techniques.

Fig. 7: Retrieval results of proposed method for a given query image.

REFERENCES

1. Y. Rui and T. S. Huang, Image retrieval: Current techniques, promising directions and open issues, J.. Vis. Commun. Image Represent., 10 (1999) 3962.

2. A. W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, Content-based image retrieval at the end of the early years, IEEE Trans. Pattern Anal. Mach. Intell., 22 (12) 13491380, 2000.

3. M. Kokare, B. N. Chatterji, P. K. Biswas, A survey on current content based image retrieval methods, IETE J. Res., 48 (3&4) 261271, 2002.

4. Ying Liu, Dengsheng Zhang, Guojun Lu, Wei-Ying Ma, Asurvey of content-based image retrieval with high-level semantics, Elsevier J. Pattern Recognition, 40, 262-282, 2007.

5. M. J. Swain and D. H. Ballar, Indexing via color histograms, Proc. 3rd Int. Conf. Computer Vision, Rochester Univ., NY, (1991) 1132.

6. M. Stricker and M. Oreng, Similarity of color images, Proc. SPIE, Storage and Retrieval for Image and Video Databaes, (1995) 381392.

7. G. Pass, R. Zabih, and J. Miller, Comparing images using color coherence vectors, Proc. 4th ACM Multimedia Conf., Boston, Massachusetts, US, (1997) 6573.

8. J. Huang, S. R. Kumar, and M. Mitra, Combining supervised learning with color correlograms for content-based image retrieval, Proc. 5th ACM Multimedia Conf., (1997) 325334.

9. Z. M. Lu and H. Burkhardt, Colour image retrieval based on DCT domain vector quantization index histograms, J. Electron. Lett., 41 (17) (2005) 2930.

10. J. R. Smith and S. F. Chang, Automated binary texture feature sets for image retrieval, Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Columbia Univ., New York, (1996) 22392242.

11. H. A. Moghaddam, T. T. Khajoie, A. H Rouhi and M. Saadatmand T., Wavelet Correlogram: A new approach for image indexing and retrieval, Elsevier J. Pattern Recognition, 38 (2005) 2506-2518.

12. H. A. Moghaddam and M. Saadatmand T., Gabor wavelet Correlogram Algorithm for Image Indexing and Retrieval, 18th Int. Conf. Pattern Recognition, K.N. Toosi Univ. of Technol., Tehran, Iran, (2006) 925- 928.

13. A. Ahmadian, A. Mostafa, An Efficient Texture Classification Algorithm using Gabor wavelet, 25th Annual international conf. of the IEEE EMBS, Cancun, Mexico, (2003) 930-933.

14. H. A. Moghaddam, T. T. Khajoie and A. H. Rouhi, A New Algorithm for Image Indexing and Retrieval Using Wavelet Correlogram, Int. Conf. Image Processing, K.N. Toosi Univ. of Technol., Tehran, Iran, 2 (2003) 497-500.

15. M. Saadatmand T. and H. A. Moghaddam, Enhanced Wavelet Correlogram Methods for Image Indexing and Retrieval, IEEE Int. Conf. Image Processing, K.N. Toosi Univ. of Technol., Tehran, Iran, (2005) 541-544.

16. M. Saadatmand T. and H. A. Moghaddam, A Novel Evolutionary Approach for Optimizing Content Based Image Retrieval, IEEE Trans. Systems, Man, and Cybernetics, 37 (1) (2007) 139-153.

17. L. Birgale, M. Kokare, D. Doye, Color and Texture Features for Content Based Image Retrieval, International Conf. Computer Grafics, Image and Visualisation, Washington, DC, USA, (2006) 146 149.

18. M. Subrahmanyam, A. B. Gonde and R. P. Maheshwari, Color and Texture Features for Image Indexing and Retrieval, IEEE Int. Advance Computing Conf., Patial, India, (2009) 1411-1416.

19. Subrahmanyam Murala, R. P. Maheshwari, R. Balasubramanian, A Correlogram Algorithm for Image Indexing and Retrieval Using Wavelet and Rotated Wavelet Filters, Int. J. Signal and Imaging Systems Engineering.

20. T. Ojala, M. Pietikainen, D. Harwood, A comparative sudy of texture measures with classification based on feature distributions, Elsevier J. Pattern Recognition, 29 (1): 51-59, 1996.

21. T. Ojala, M. Pietikainen, T. Maenpaa, Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell., 24 (7): 971-987, 2002.

22. M. Pietikainen, T. Ojala, T. Scruggs, K. W. Bowyer, C. Jin, K. Hoffman, J. Marques, M. Jacsik, W. Worek, Overview of the face recognition using feature distributions, Elsevier J. Pattern Recognition, 33 (1): 43-52, 2000.

23. T. Ahonen, A. Hadid, M. Pietikainen, Face description with local binary patterns: Applications to face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 28 (12): 2037-2041, 2006.

24. G. Zhao, M. Pietikainen, Dynamic texture recognition using local binary patterns with an application to facial expressions, IEEE Trans. Pattern Anal. Mach. Intell., 29 (6): 915-928, 2007.

25. M. Heikkil;a, M. Pietikainen, A texture based method for modeling the background and detecting moving objects, IEEE Trans. Pattern Anal. Mach. Intell., 28 (4): 657-662, 2006.

26. X. Huang, S. Z. Li, Y. Wang, Shape localization based on statistical method using extended local binary patterns, Proc. Inter. Conf. Image and Graphics, 184-187, 2004.

27. M. Heikkila, M. Pietikainen, C. Schmid, Description of interest regions with local binary patterns, Elsevie J. Pattern recognition, 42: 425-436, 2009.

28. M. Li, R. C. Staunton, Optimum Gabor filter design and local binary patterns for texture segmentation, Elsevie J. Pattern recognition, 29: 664-672, 2008.

29. B. Zhang, Y. Gao, S. Zhao, J. Liu, Local derivative pattern versus local binary pattern: Face recognition with higher-order local pattern descriptor, IEEE Trans. Image Proc., 19 (2): 533-544, 2010.

30. Corel 1000 and Corel 10000 image database. [Online]. Available: .

1.Registration Form(Given in brochure) 2.Copy Right Form

1. DD

2. Original Paper in format given in attachment NCDMA-032