Complex Wavelet based Color Image Retrieval with Relevance Feedback

Complex Wavelet based Color Image Retrieval with Relevance Feedback

Jayashree Khanapuri

K J Somaiya Institute Of Engineering and Information Technology

Abstract

Color and texture description is important aspects in image retrieval from large image databases. It is necessary to build an efficient search system which provides fast and accurate retrieval. In this paper, an approach to improve retrieval accuracy using relevance feedback is discussed with proposed distance measure. The information in images is represented utilizing dual tree complex wavelet transform (DT- CWT) and dual tree rotated complex wavelet filter jointly. The properties of DT CWT like shift invariance, good directional selectivity, computational efficiency and perfect reconstruction are effective in describing texture patterns. The DT-RCWF improves the retrieval by adding the information of oriented textures. The retrieval is carried out by decomposing the image using complex wavelet transform and rotated complex wavelet filter & computing the energy, standard deviation of the sub bands as feature vectors. The average retrieval accuracy of the class is improved by relevance feedback.

1. Introduction

   Large number of users is able to access huge amount of data as there is vast growth of internet and world – wide web. Internet has created super highway of information for the end user. Exponential development of image database has led to the creation of digital libraries and multimedia databases. This rapid growth of information has created the need to develop effective and efficient search algorithm for image retrieval. It is difficult to annotate the images using keywords as they contain rich content. This limits the search operation to limited number of keywords and also it is time consuming. This led to the development search system based on content. The color, texture and shape of the image play important role as features in content based image retrieval system. A majority of indexing techniques are based on spatial domain features like color [2], texture [3], and shape [4]. The aim of efficient content based image retrieval system is to capture the important

   characteristics of an image which makes it different from the other similar images. The Content-Based Image Retrieval [1] is aimed at efficient retrieval of relevant images from large database. The visual contents of the image are extracted in the form of feature vector and stored in the database. This process is known as indexing. The features of the query image are extracted. The query features are compared with database features by using similarity metric to retrieve similar images [15].

   Many researches are based on indexing in frequency domain. The compressed domain indexing techniques have become more popular because of less complexity. Some frequency domain techniques include wavelet domain features, Gabor transform and Fourier domain features etc. for efficient extraction of color and texture features together. Color is one of the most widely used features in image retrieval due to cheap availability of color image acquisition. Methods have been proposed with complex wavelet transform which provide fast way of computing texture image retrieval. Much of the research work has been done in this direction in terms of texture analysis, classification and segmentation extending the study to texture in color images. The texture characteristics are extracted taking into account the color bands giving the information about color texture. [7, 8, 9, 10, 11]. Methods have been proposed how we can enhance the texture extraction capabilities of CWT for color image retrieval [6]. The wavelet approach has emerged as efficient over other methods because of the drawbacks of other frequency domain techniques. The standard DWT lacks sensitivity and directionality and not able to analyze the high frequency signal with narrow bandwidth. The gabor wavelet introduces redundancy and memory requirement as its basis functions are not orthogonal.

   Many researches are conducted to improve the retrieval accuracy by relevance feedback. Relevance feedback based on Nearest neighbor approach is proposed in which each image is ranked with reference to relevance score based on nearest neighbor distance [12]. Another method proposed using Markov model mediators frameworks to cluster the database in order to improve the performance of query processing and matching

   [13]. Region based image retrieval with relevance feedback is proposed in which the system learns the importance of region based on the feedback from the user to improve the performance [14]. Addressing small sample size for training will reduce the retrieval time[19].

   Since, the objective of this paper is to present an efficient method which provides better average retrieval accuracy. An approach of relevance feedback is introduced to improve the performance of the system. An investigation is presented in this direction in the paper using color and texture feature retrieval by complex wavelet transform and rotated complex wavelet filter which provide texture information in twelve different directions. The relevance feedback further enhances the performance of the system in terms of retrieval accuracy.

   The paper is organized as follows. The theory related to dual tree complex wavelets & rotated complex wavelet filter are discussed in section 2. The design and implementation of proposed algorithm is discussed in section 3. The experiment and its results are discussed in section 4 followed by conclusion in section 5.

2. Wavelet decomposition

   1. Dual tree complex wavelet transform

      In multi resolution analysis of wavelets, the low pass information consists of approximated version of high resolution image. The high pass information gives sharper variation details. In dual tree complex wavelets, two trees are used in parallel to generate the output interpreting them as real and imaginary part of complex coefficient. Two sets of QMF filter pairs are used in the generation of real and imaginary coefficients in the analysis branch. The transform decomposes the image into sub bands providing the two smoothed versions of image and the information in six different directions at each stage. All the filters used in the analysis are real & orthogonal [16, 18]. The outputs of the QMF filters is given as,

      The resultant six wavelets generated using equations (2), (3) and (4) are as follows.

      These are directional and two times redundant. These six wavelets are strongly oriented in {+150, +450, +750,

      -150,-450, -750} to capture information in respective direction and are shift invariant.

   2. Rotated complex wavelet filter

      In image retrieval, it is necessary to capture the information in many directions in order to improve the retrieval performance of the system. The texture properties are very well characterized by edge information. The 2-D rotated complex wavelet filters are non separable and provide orientation selectivity[5]. They characterize texture properties by gathering edge information in 6 different directions. i.e. {-300, 00,

      +300, 600, 900, 1200} . The filters are generated by rotating the 2-D complex wavelet filters by 450. This filter combined with dual tree complex wavelet filter will provide edge information in 12 different directions.

      2.3. Relevance feedback

      The efficiency of the image retrieval system can be improved by relevance feedback by training the images in the database by end users feedback. The database consists of images belonging to different classes. Eac query image under consideration belongs to one of the classes. During training using relevance feedback, the top 100 retrievals (assuming there are 10 different classes each consisting of 100 images) for each query image are extracted. These are classified as relevant and non relevant image groups. The feature vector is constructed for relevant and non relevant groups. The feature vectors corresponding to the relevant and non relevant group of query image is calculated as,

      The vectors Frelevant and Fnon_rrelevant are calculated for query image under training. New feature vector is generated for query image under consideration by modifying the original feature vector.

      The modified feature vector of the query image is given as,

      where, Foriginal is the original feature vector of query image. The image retrieval is carried out with modified feature vector using proposed similarity measure to obtain improved retrieval accuracy.

3. Feature extraction

   The DT- CWT and DT RCWF are jointly used for the feature extraction. The decomposition is carried out up to eighth level using selsenicks first and higher stage low pass & high pass filters. As DT-CWT and DT- RCWF provide edge information in twelve different directions, the filter coefficients are obtained by passing the image (2-D signal) through real & imaginary trees. The energy and standard deviation are calculated for all the sub bands[17]. The energy and standard deviation is calculated for all sub bands for

   DT-CWT and DT -RCWF. The calculations for the energy and standard deviation of kth sub band are given as,

   Where Wk( i, j) is the kth wavelet decomposed sub band , M x N is the size of the kth sub band, µK is the mean of kth sub band. The feature vector is constructed

   using the energy & standard deviation of the sub bands. The resulting feature vector is given by,

   Feature Vector at each stage={E1,E2E6, 1,2 6)

   Final Feature Vector = 8 x size of Feature Vector at each stage (15)

   The final feature vector is constructed for all the images in the database. The similarity measure between the query and the database images is carried out using proposed distance. If D & Q are the feature vectors of the database and query images respectively, hen the proposed distance is given by,

   The retrieval accuracy of each class is further improved by relevance feedback. The vectors Frelevant and Fnon_rrelevant are calculated for query images under

   training. The modified feature vector for the respective query image is generated using these vectors. The image retrieval is carried out with modified feature vector using proposed distance to obtain improved retrieval accuracy.

4. Experimental results

   The experiment was conducted on Wang database consisting of 1000 images belonging to 10 different classes. Namely, People, beach, elephant, horse, food, monument, rose, dinosaur, bus and mountain. The pre processing stage involves the resizing images to [256,256] and then converting into Y, Cb, Cr domain. The energy and standard deviation values are calculated for the channels Y, Cb and Cr separately. The features are extracted by decomposing the 2-D signal (image) up to eighth level and constructing feature vector consisting of energy, standard deviation values. Thus, each image in the database consists of 288 features corresponding to DT CWT and DT RCWF each in its feature vector. The similarity measure between query image and the images in the database executed using proposed distance given in equation (20) by considering each image in the database as query image. The retrieval accuracy of query image and the average retrieval accuracy of all the classes are calculated. The Retrieval Accuracy of the query image is calculated as,

   The Average Retrieval Accuracy of each class is given by,

   Where n=No. of images in the class

   The performance of the system is evaluated considering only DT-CWT, DT-RCWF and combination of DT- CWT and DT-RCWF. The average retrieval accuracy of the class is calculated. The training of query images is carried out for all images of the class with relevance feedback in order to improve retrieval accuracy. However, in the experiment, only five iterations are used to improve the retrieval accuracy of query images. The number of images considered for training with relevance feedback and the average retrieval accuracy achieved in the iterations is shown in Table 1. The

   average retrieval accuracy obtained using proposed distance as similarity measure with and without Selective Relevance Feedback is tabulated in Table 2. Table 3 indicates the percentage of improvement achieved by implementing proposed method.

   Table1. Average Retrieval Accuracy of Images with CWT, RCWF and combination of CWT & RCWF

   No	Class	Average Retrieval Accuracy of each class
   		CWT	RCWF	CWT +RCWF
   1	People	29.16	33.79	32.76
   2	Beach	25.69	34.57	31.94
   3	Building	28.68	24.77	30.25
   4	Bus	47.55	48.69	52.30
   5	Dinosaur	73.06	98.24	89.22
   6	Elephant	40.84	36.81	46.41
   7	Rose	51.12	72.07	63.71
   8	Horse	41.48	52.91	50.90
   9	Mountain	24.35	33.48	30.63
   10	Food	42.02	33.36	42.21
   	ARA of class	40.395	46.869	47.033

   ARA Average Retrieval Accuracy

   Table 2. Average Retrieval Accuracy of all the Classes without & with Relevance Feedback using CWT and RCWF

   Class	Average Retrieval Accuracy
   		With Relevance Feedback
   	Iter 0	Iter 1	Iter 2	Iter 3	Iter 4	Iter 5
   People	32.76	45.51	54.03	59.64	62.97	64.93
   Beach	31.94	44.10	51.24	55.38	58.03	59.96
   Building	30.25	40.60	44.85	46.60	47.29	47.80
   Bus	52.30	72.63	82.68	86.39	88.34	89.22
   Dinosaur	89.22	97.39	98.28	98.53	98.71	98.84
   Elephant	46.41	66.89	76.40	79.97	81.00	81.37
   Rose	63.71	73.80	75.62	78.16	80.44	80.44
   Horse	50.90	70.80	80.11	85.06	88.49	90.90
   Mountain	30.6	41.15	47.29	52.08	55.16	56.91
   Food	42.21	57.23	63.55	66.38	68.03	69.20
   ARA	47.03	61.01	67.40	70.80	72.84	73.95

   Table 3. Comparison of outputs without and with Relevance Feedback

   Class	Average Retrieval Accuracy without Relevance feedback	Average retrieval accuracy With Relevance Feedback (iteration 5)	Improvem ent in Average Retrieval Accuracy With Relevance Feedback
   People	32.76	64.93	32.17
   Beach	31.94	59.96	28.02
   Building	30.25	47.80	17.55
   Bus	52.30	89.22	36.92
   Dinosaur	89.22	98.84	9.62
   Elephant	46.41	81.37	34.96
   Rose	63.71	80.44	16.73
   Horse	50.90	90.90	40.00
   Mountain	30.63	56.91	26.28
   Food	42.21	69.20	26.99
   ARA	47.03	73.95	26.92

5. Conclusion

In this paper, the implementation of proposed method and its comparison with retrieval without relevance feedback is carried out. The result indicates that the average retrieval accuracy is improved with Relevance Feedback. It is observed that dinosaur and building Class have maximum and minimum average retrieval accuracy respectively. The retrieval accuracy increases with every iterations. The retrieval accuracy of most of the classes is increased by more than 25%.

References

1. Kato, T .,Database architecture for content based image retrieval in Image storage and Retrieval Systems (Jambardino A and Niblack Weds), Proc SPIE 2185, pp 112-123,1992.

2. Kokare, M., Chatterji, B. N., Biswas P. K., (2002) A Survey on Current Content-based Image Retrieval Methods, IETE J. Res. 48(3&4), pp. 261-271.

3. Rui, Y. Huang, T. Ortega, M. Mehrotra, S. Relevance Feedback: A Power Tool in Interactive Content-Based Image Retrieval, IEEE Transactions on Circuits and Systems for Video Technology, 1998, Vol. 8(5), pp. 644-655.

4. Rui, Y., Huang, T.S., and Mehrotra,S. Content-based Image Retrieval with Relevance Feedback in MARS, 1997, in Proc. IEEE Int. Conf. on Image proc.

5. M. Kokare, P. K. Biswas, B. N. Cheterji, Texture Image Retrieval Using New Rotated Complex Wavelet Filters. SMC-B, 35(6) 2005, pp. 11681178.

6. P. Janney, G. Sridhar, V. Sridhar, Enhancing capabilities of Texture Extraction for Color Image Retrieval. WEC, 5 2005, pp 282285.

7. G. Paschos, Chromatic Correlation Features for Texture Recognition, Pattern Recognition Letters(1998), 19:643-50.

8. G. Paschos, Fast Color Texture Recognition using Chromaticity Moments, Pattern Recognition Letters(2000), 21:837-41.

9. G. Van de Wouver , P. Scheunders, S. Livens,

   D.V. Dyck, Wavelet correlation signatures for color texture characterization, Pattern recognition Lettesr(1999), 32:443-51.

10. M.P. Dubuisson-Jolly ,A. Gupta, Color and Texture Fusion: application to aerial image segmentation and GIS updating , Image and Vision computing ,2000, 18: 823-32.

11. A. Drimbean, P F Whelan, Experiments in Color Texture Analysis, Pattern Recognition Letters,2001, 22:1161-70.

12. Giorgio Giacinto, A Nearest Neighbor Approach to Relevance Feedback in Content based Image Retrieval, CIVR-2007, July 9-11, 2007, pp 456-463.

13. Mei-Ling Shyu, Shu-Ching Chen, Chengcui Zhang, A unified framework for image database clustering and content based retrieval, MMDB-04, 2004, 19-27

14. Feng Jing, Bo Zhang, Fuzong Lin, Wei-Ying ma, A novel region based image retrieval method using relevance feedback, International conference on Multimedia information retrieval, 2001, 28-31

15. A.W.M Smeulders, S. Santini, M. Worring, Content Based Image Retrieval at the End of the Early Years, IEEE. Trans. PAMI, vol.22, December 2000, 1349-1380.

16. N G Kingsbury, The dual tree complex wavelet transform: A new technique for shift invariance and directional filters, in Proc. 8th IEEE DSP workshop, Utah, Aug 9-12, 1998.

[17]. An Zhiyong, Zhao Feng, Image Retrieval based on Energy and Entropy of Multi wavelet Transform , 2009 International conference on Information Technology and Computer Science, 2009, 544-577.

1. K N Prakash, K Satyaprasad, M Band Dual tree complex wavelet transform for texture Image Indexing and Retrieval, International Journal of Advance in Engineering & Technology, Vol 4, Issue1, July 2012, 661-671.

2. Pushpa Patil, Manesh Kokare, Semantic Image Retrieval Using Relevance Feedback, International Journal of Web & Semantic Technology, Vol 2, No.4, October 2011, 139-148.