Relevance Feedback in Content Based Image Retrieval

Relevance Feedback in Content Based Image Retrieval

Ms. Shital S. Jadhav

1. (CSE)student ,Dept of CSE G.H.Raisoni Institute of Engg and Management Jalgaon ,

   Maharashtra, India

   Prof. Mrs. Swati Patil

   Assistant Professor, Dept of Computer Science and Engg, G.H.Raisoni Institute of Engg and Management, Jalgaon, Maharashtra, India

   AbstractCBIR has been a very active research area .CBIR is the mainstay of current image retrieval system. The purpose of CBIR is to present an image conceptually, with a set of low-level visual features such as color, texture, and shape. The computational complexity and retrieval accuracy are main problems in CBIR. To avoid this problem, this paper provides a overview on a new content-based image retrieval method using color and texture feature with relevance feedback. Relevance feedback techniques were incorporated into CBIR such that more precise results can be obtained by taking users feedbacks into account. Relevance feedback is a powerful technique in CBIR systems. However, existing relevance feedback-based CBIR methods usually request a number of iterative feedbacks. So our system will try to reduce the number of feedback by mining the navigation behavior of user. That navigation behavior will be stored in the log database.

   Keywords: Relevance feedback, image retrieval, content-based image retrieval

   1. INTRODUCTION

      Image databases and collections can be enormous in size, containing hundreds, thousands or even millions of images. Currently under development, even though several systems exist, is the retrieval of images based on their content, called Content Based Image Retrieval, CBIR.[1]

      A huge amount of Image databases are added every minute and that is why it is the need for effective and efficient image retrieval systems. There are many features of content-based image retrieval but four of them are considered to be the main features. They are color, texture, shape, edge direction and spatial properties[2]. Spatial properties, however, are implicitly taken into account so the main features to investigate are color, texture and shape .

      A typical CBIR uses the contents of an image to represent and access. CBIR systems extract features like color, texture, and shape from images in the database based on the value of the image pixels. These features are smaller than the image size. These features are stored in a database called feature database. Thus the feature database contains an abstraction of the images in the image database; each image is represented by a compact representation of its contents that is color, texture, shape, and spatial information. This is called offline feature extraction. The main advantage of using CBIR system

      is that the system uses image features instead of using the image itself. So, CBIR is cheap, fast, and efficient over image search methods.

      All CBIR systems view the query image and the retrieved images as a collection of features. These features, or image signatures, characterize the content of the image. The advantages of using image features instead of the original image pixels appear in image representation and comparison for retrieving. When the system uses image features for matching, it almost do compression for the image and use the most important content of the image[3]. This also bridges the gaps between the high level concept and low level feature.

      Several methods for retrieving images on the basis of color similarity are being used. Each image added to the database is analyzed and a color feature is computed which shows the proportion of pixels of each color within the image. Then this color feature for each image is stored in the database. During the search time, the user can either specify the desired proportion of each color or submit a reference image from which a color feature is calculated. The matching process then retrieves those images whose color matches those of the query most closely. The ability to match on texture similarity can often be useful in distinguishing between areas of images with similar color. A variety of techniques has been used for texture similarity. Texture similarity means calculate the relative brightness of selected pairs of pixels from each image[4]. From these it is possible to calculate image texture measures such as the degree of contrast, coarseness, periodicity, directionality and randomness ,directionality and regularity.

      In general, the purpose of CBIR is to present an image conceptually, with a set of low-level visual features such as color, texture, edge direction and shape. These conventional approaches for image retrieval are based on the computation of the similarity between the users query and images via a query by example system. Despite the power of the search strategies, it is very difficult to obtain the accurate retrieval result of CBIR within only one query process. The problem is that the extracted visual features are too diverse to capture the concept of the users query. To solve such problems, in the QBE system, the users can pick up some preferred images to refine the image explorations iteratively. The feedback procedure, called Relevance Feedback (RF), repeats until the user is satisfied with the retrieval results.

      In relevance feedback-based approaches, a CBIR system learns from feedback provided by the user. The learning is

      categorized into short-term learning, and long term learning. Relevance feedback has gained much attention in the research and development of content-based image retrieval systems[9].

      Although a number of RF studies have been made on CBIR, they still having some common problems such that existing RF methods focus on how to earn the users satisfaction in one query process. That is, existing methods refine the query again and again by analyzing the specific relevant images picked up by the users. Especially for the compound and complex images, the users might go through a long series of feedbacks to obtain the desired images using current RF approaches. But it is not efficient in real applications. Existing relevance feedback-based CBIR methods usually request a number of iterative feedbacks to produce refined search results, especially in a large-scale image database. This is impractical and inefficient in real applications. So it is necessary to reduce the number of feedback .

   2. RELATED WORK

      For color feature Ahmed J. Afifi, Wesam M. Ashour used color moment method because it has the lowest feature vector dimension and lower computational complexity. To extract the color features from the content of an image, Ahmed J. Afifi, Wesam M. Ashour need to select a color space and use its properties in the extraction.For texture feature Ahmed J. Afifi, Wesam M. Ashour used Ranklet transform. Before extract the texture feature from the image, perform a preprocessing step using Ranklet Transform. The result of applying Ranklet Transform on the image is 3 ranklet images in different orientation (vertical, horizontal, and diagonal)[3]. The concept of relevance feedback was introduced into CBIR from the concept of text-based information retrieval in the 1998s [5] and then has become a popular technique in CBIR. By using relevance feedback, Content-Based Image Retrieval (CBIR) allows the user to retrieve images interactively. Begin with a coarse query, the user can select the most relevant images and provide a weight of preference for each relevant image to refine the query. The high level concept born by the user and perception subjectivity of the user can be automatically captured by the system to some degree. Pengyu Hng, Qi Tian, Thomas S . Huang proposes an approach to utilize both positive and negative feedbacks for image retrieval. Support Vector Machines (SVM) is applied to classifying the positive and negative images. The SVM learning results are used to update the preference weights for the relevant images[6]. For using relevance feedback to improve the retrieval accuracy, Dewen Zhuang, Shoujue Wang refined the visual features extracted based on linear discriminant analysis algorithm Dewen Zhuang, Shoujue Wang segment image into main region and margin region for cognition the whole image characteristic. . To extract image features, Dewen Zhuang, Shoujue Wang select the width of margin as 0.1 of image size according to the experiment results. Color histogram is one commonly used visual features and has a computation simple but efficient characteristics. Owing some color spaces(e.g. LUV, HSV) seem to coincide better with human perception than the basic RGB color space, so Dewen Zhuang, Shoujue Wang use HSV color space for histogram-based descriptors. [7]. Yuki Sugiyama, Makoto P. Kato et.al. propose relative relevance feedback for image retrieval that aims to capture the relativity of positive and negative examples in search result

      pages. Given positive examples, the only thing is that selected items are more similar to an ideal query than items that are not selected as relevant [10].

   3. PROBLEM DEFINITION

      Early studies on CBIR used a single visual content such as color, texture, or shape to describe the image. The drawback of this method is that using one feature is not enough to describe the image since the image contains various visual characteristics. The computational complexity and the retrieval accuracy are the main problems that CBIR systems have to avoid. Early studies on CBIR used a single visual content such as color, texture, or shape to describe the image. The drawback of this method is that using one feature is not enough to describe the image since the image contains various visual Characteristics. The result of using multiple feature are more accurate than using only one feature[3]. The conventional approaches for image retrieval are based on the computation of the similarity between the users query and images. Despite the power of the search strategies, it is very difficult to optimize the retrieval quality of CBIR within only one query process. The hidden problem is that the extracted visual features are too diverse to capture the concept of the users query[5]. It is necessary to retrieve the image by considering the user feedback. But the problem is that this method requires the more number of feedback. So it is necessary to reduce the number of feedback. Extracting multimedia data from the large multimedia repository suffer from problem such as redundant browsing and exploration convergence. Whenever the user query the database the resultant data is irrelevant with the users query and it takes long iterations of feedback to produce the result. The goal is to assist the search strategy in efficiently hunting the desired images.

   4. PROPOSED SYSTEM

      The proposed system will implement the CBIR using color and texture feature with relevance feedback. But existing relevance feedback request a number of feedback. So the proposed system will be trying to reduce the frequency of relevance feedback.

      Color Feature

      Color is the most fundamental characteristic of image. Color provides better stability. It is insensitive to rotation and zoom of image. Human eyes are sensitive to colors, and color features enable human to distinguish between objects in the images. Colors are used in image processing because they provide powerful descriptors. Color features provide sometimes powerful information about images, and they are very useful for image retrieval. Many methods can be used to describe color feature. There are color histogram, color correlation , color moments, color structure descriptor (CSD), and scalable color descriptor (SCD) . The proposed system will use color moment[3] method because it has the lowest feature vector dimension and lower computational complexity.

      To extract the color features from the content of an image, we need to select a color space and use its properties in the extraction. Colors are defined in three-dimensional color space. The most widely used color space is RGB color space. The main drawback of the RGB color space is that it is device dependent and non-uniform.So the RGB color space is

      converted into HSV color space. The HSV color space describes a specific color by its hue, saturation, and brightness values. Hue represents the dominant wavelength in light. Saturation is intensity of color or degree of purity of color. Value is brightness of color or relative lightness and darkness of color.

      1. Texture Feature

         Texture is another important property of images it provides the measures of properties such as smoothness, coarseness, and regularity. Furthermore, texture can be thought as repeated patterns of pixels[3].Texture is an innate property of all surfaces that describes visual patterns, each having homogeneity. It contains important information about the structural arrangement of a surface, such as; clouds, leaves, bricks, fabric, etc. It also describes the relationship between the surfaces to the surrounding environment. In short it is a feature that describes the distinctive physical composition of a surface. Basically, texture representation methods can be classified into two categories: structural and statistical. Structural methods, including morphological operator and adjacency graph, describe texture by identifying structural primitives and their placement rules. They tend to be most effective when applied to textures that are very regular[8]. Statistical methods, including Fourier power spectra, co- occurrence matrices, Tamura feature, Wold decomposition, Markov random field, fractal model, and multi-resolution filtering techniques such as Gabor and wavelet transform, characterize texture by the statistical distribution of the image intensity. To extract the texture feature, Local Binary Pattern (LBP) is found to be a powerful feature. The local binary pattern operator is an image operator which transforms an image into an array or image of integer labels describing small-scale appearance of the image. These labels or their statistics, most commonly the histogram, are then used for further image analysis. It is an operator for image description that is based on the signs of differences of neighboring pixels. The LBP histogram is widely applicable image feature for,

         e.g. texture classification, face analysis, video background subtraction, etc[11].

      2. Relevance Feedback

         A CBIR system learns from feedback provided by the user in relevance feedback-based approaches,. CBIR systems categorized the learning process as short-term learning, and long term learning [9].Query refining using relevance feedback has gained much attention in the research and development of content-based image retrieval(CBIR) systems. Most of the researches have focused on query tuning in a single retrieval session. This is commonly known as intra- query learning or short term learning. In contrast, inter-query, also known as long term learning is strategy that attempts to analyze the relationship between the current and past retrieval sessions.

         By using relevance feedback, Content-Based Image Retrieval (CBIR) allows the user to retrieve images efficiently and effectively. Begin with a initial query, the user can select the most relevant images and provide a weight for features of each relevant image to refine the query. Relevance Feedback constitutes the process of refining the results returned by the CBIR system in a given iteration of an interaction session. The user performs some sort of evaluation over the results returned in the last iteration and this evaluation is fed back to the

         system . However, existing relevance feedback-based CBIR methods usually request a number f iterative feedbacks to produce refined search results, especially in a large-scale image database. This is impractical and inefficient in real applications. So our system will try to reduce the number of feedback that is frequency of feedback by mining the navigation behavior of user. That navigation behavior wiil be stored in the log database.

      3. Proposed Architecture

      To resolve the problem of number of feedback, the proposed system will use the navigation behavior of user to achieve the high retrieval quality of CBIR with RF. The navigation patterns mined from the user query log can be viewed as the shortest paths to the users interested space. The navigation pattern will be stored in log database. A log-based relevance feedback is a technique that integrates the log of feedback data into the traditional relevance feedback schemes to learn effectively the correlation between low-level image features and high-level concepts.

      Fig 1.Architecture of Log Based Relevance feedback [12]

      At first a query image will be submitted to this system, the system first finds the most similar images without considering any search strategy, and then returns a set of the most similar images. The first query process is called initial feedback. Next, the good examples picked up by the user deliver the valuable information to the image search phase, including the users intention. Then, by using the navigation patterns find the desired images. Overall, at each feedback, the results are presented to the user and the related browsing information will be stored in the log database.

   5. CONCLUSION AND FUTURE SCOPE

      CBIR has been a very active research area. CBIR gives more accurate result with multiple feature than with a single feature The purpose of CBIR is to present an image conceptually, with a set of low-level visual features such as color, texture, and shape. The computational complexity and the retrieval accuracy are the main problems in CBIR. To avoid these problems, the proposed system proposes a new

      content-based image retrieval method that will uses both color and texture feature.

      Relevance feedback techniques were incorporated into CBIR such that more precise results can be obtained by taking users feedbacks into account. Existing relevance feedback- based CBIR methods usually request a number of iterative feedbacks to produce refined search results. So the proposed system uses Relevance Feedback that reduces the number of feedback to achieve the high retrieval quality of CBIR by using the navigation patterns of user .

   6. REFERENCES

1. Ruba A. A. Salamah, Efficient Content Based Image Retrieval, Dept.

   Comp. Sci., Islamic University Gaza, Tech. Rep, 2010

2. P. V. N. Reddy, K. Satya Prasad, Color and Texture Features for Content Based Image Retrieval, P V N Reddy et al, Int. J. Comp. Tech.

   Appl., Vol 2 (4), 1016-1020, 2011

3. Ahmed J. Afifi, Wesam M. Ashour, Content-Based Image Retrieval Using Invariant Color and Texture Features, IEEE 978-1-4673-2181- 5/12, 2012

4. Ms. K. Arthi , Mr. J. Vijayaraghavan , Content Based Image Retrieval Algorithm Using Colour Models, International Journal of Advanced Research in Computer and Communication Engineering Vol. 2, Issue 3,

   March 2013

5. Ja-Hwung Su, Wei-Jyun Huang, Philip S. Yu, Fellow, Efficient Relevance Feedback for Content-Based Image Retrieval by Mining User Navigation Patterns, IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 23, NO. 3 ,March 2011

6. Pengyu Hong, Qi Tian, Thomas S . Huang, Incorporate Support Vector Machines To Content-Based Image Retrieval With Relevant Feedback,

   IEEE, 0-7803-6297-7/00, 2000

7. Dewen Zhuang, Shoujue Wang, Content-based image retrieval based on integrating region segmentation and relevance feedback, 978-1- 4244-7874-3/10, 2010 IEEE

8. Dr. Fuhui Long, Dr. Hongjiang Zhang and Prof. David Dagan Feng, Fundamentals Of Content-Based Image Retrieval

9. Pushpa B. Patil, Manesh B. Kokare, Relevance Feedback in Content Based Image Retrieval: A Review, Journal of Applied Computer Science & Mathematics, no. 10 (5) /2011, Suceava,2011

10. Yuki Sugiyama, Makoto P. Kato, Hiroaki Ohshima, Katsumi Tanaka, Relative Relevance Feedback In Image Retrieval, 2012 IEEE International Conference On Multimedia And Expo, 978-0-7695-4711- 4/12, 2012 IEEE

11. Timo Ahonen, Matti Pietikäinen, Soft Histograms For Local Binary Patterns, Machine Vision Group, Infotech Oulu

12. Shruti Patil, A Comprehensive Review of Recent Relevance Feedback Techniques in CBIR, International Journal of Engineering Research & Technology (IJERT) Vol. 1 Issue 6, August – 2012