Image Retrieval With Relevance Feedback By Using Semisupervised Biased Maximum Margin Analysis

Image Retrieval With Relevance Feedback By Using Semisupervised Biased Maximum Margin Analysis

S. Himaja (IInd M.tech,MITS N. Sudhakar Yadav (Assistant Professor

of CSE,MITS

Abstract : In past few years CBIR (content based image retrieval) system is most popular in retrieving the images. But it will not fill the gap between low-level visual features and high-level semantic concepts. To fill this gap relevance feedback was introduced as a powerful tool to improve the performance of the CBIR system. Various relevance feedback techniques are used to improve the performance likequery movement and reweighting algorithm, classification based algorithm, subspace learning algorithme.t.c.By these techniques it will need to perform more number of iterations. For this purpose Support Vector Machine (SVM) active learning algorithm is introduced in relevance feedback. But there are two drawbacks are there by using these SVM based RF techniques. First it will treat positive and negative feedbacks equally. Second it will not consider unlabeled samples. To overcome these two drawbacks BMMA and SemiBMMA are introduced in SVM based RF.BMMA is used to analyze the difference between positive and negative samples

unequally. SemiBMMA is used for considering the unlabeled samples by introducing laplacian regularizer to the BMMA.In the CBIR system images are retrieved based on the computation of the similarity between the users query and images via a query by example (QBE) system. But the hidden problem is that the extracted visual features are too diverse to capture the concept of the users query. To solve these type of problems, in the QBE system an automatic method is proposed to estimate the semantic similarity between words or entities using web search engines. 1.INTRODUCTION

During the past few years,Keyword-based image retrieval is used for retrieving the

images.There are some drawbacks are there with these system,but one of the main drawback isa real world image contain different concepts it is difficult to annotate such an image by small number of keywords.Consider a case if user wants to retrieve an image world mapwhich contains country names and state names, in this case it is very difficult for user to

annotate names of countries which are present in the world map.To overcome those drawbacks content based image retrieval(CBIR) system was introduced.In the CBIR system an image is retrieved based based on low-level visual features(color,shape,texturee.t.c),but it will not characterize the high-level semantic concepts.That means it will not provide what the human expect.To fill this semantic gap between low-level visual features and high-level semantic concepts relevance feedback(RF) was introduced as a powerful tool to improve the performance of the CBIR system.Relevance feedback provides interaction between human and computer by using feedback concept.There are two types of feedbacks are there.They are (1)Positive Feedback(2)Negative Feedback.If the retrieved image is similar to the target image then the user will labeled it as a relevant (or) positive.If the retrieved image is not similar to the target image then the user will labeled it as irrelevant(or)negative.

Several relevance feedback techniques are used for retrieving relevant images.some of the techniques are Query movement and reweighting techniques,density estimation method,subspace learning technique,classification based algorithme.t.c.Among these techniques classification based RF is one of the most popular technique in CBIR system.But by using RF in CBIR system it will need to perform more iterations for retrieving more relevant images.To overcome this problem active learning algorithm is used.Several active learning algorithms are there but among these Support vector machine(SVM) active learning algorithm is used in relevance feedback.In this SVM based RF hyperplane is used to differentiate positive and negative feedbacks.

But by using SVM based RF in Content based image retrieval(CBIR) system there are two main drawbacks are there.(1)It will consider both positive and negative feedback classes while retrieving images (2)It will not consider unlabeled samples.To overcome these two drawbacks Biased maximum margin analysis(BMMA) and Semi BMMA are introduced for SVM based RF based on the Graph-embedding framework.BMMA is used for differentiating the positive and negative feedback samples.SemiBMMA is used for considering the unlabeled samples.To utilize the information of unlabeled samples in the databased Laplacian regularizer is introduced in to the BMMA which will lead to SemiBMMA. In general, the purpose of CBIR is to retrieve relevant images based on low-level visual features such as color, texture, and shape. In These conventional

approaches images are retrieved based on the computation of the similarity between the users query and images via a query by example (QBE) system. In spite of the power of search strategies, it is very difficult to improve the retrieval quality of CBIR within only one query process. The main hidden problem is that the extracted visual features are widely varied to capture the concept of the users query. To solve these problems, in the QBE system an automatic method is proposed to estimate the semantic similarity between words or entities using web search engines. In the existing system SVM (Support Vector Machine) Technique was used. Where the images are tagged and trained in the training module, and then the results can be retrieved

.we propose an enhanced method of CBIR with considering the ambiguity between words and solves the ambiguity problem by using Web Search Engines.

1. In CBIR system it is possible to capture only one aspect of image property. To overcome this problem Relevance Feedback was introduced. It is used to establish the link between high-level concepts and low-level features. By using the following three steps

   it will establish the link between humans and computers.

   Step1:For a given query first retrieve a list of ranked images according to predefined similarity metrics.

   Step2:The user marks the retrieved images as relevant to the query or not relevant.

   Step3:The system will refine the retrieval results based on the feedback and present new list of images to the user.

   Some relevance feedback techniques are used for performing these tasks.But by using these techniques it will need to perform lot of iterations.To overcome this problem support vector machine is used.support vector machine is an active learning algorithm.By using this algorithm images can be retrieved very effectively.

   Support vector Active Learning Algorithm

   Step1:Start

   Step2: In relevance feedback user can mark as relevant or irrelevant.

   Step3:Consider the top N images in the result of training data.

   Step4: Calculate the Euclidian distance between the target image to the database image.

   n

   n

   r j j

   r j j

   D Fq , Fd Fq Fd

   j 1

   Where D=Euclidian distance

   Fq = Target Image features

   Fd = Database Image features

   Step5: According to the user query construct an hyper plane which will separate relevant and irrelevant images.

   2 1 2 1

   2 1 2 1

   D x x 2 y y 2

   Step6: Sort images based on the distance. Step 7:Stop.

   2.2 Biased Maximum Margin Analysis(BMMA)

   2

   2

   BMMA is used for differentiating the

   And then calculate the objective function of the intrinsic graph by using the following formulas

   positive feedbacks from negative ones baed

   S T x

   T x W

   on local analysis.SemiBMMA is used for

   I

   i j

   i j

   i j: jNsoriNs

   i j ij

   integrating the information of unlabeled samples by introducing laplacian regularizer to the BMMA.In the proposed system graph embedded framework is used.BMMA is used for overcome the first drawback.To overcome that problem let us assume All positive feedbacks are alike and each negative feedback is in its own way and then construct the two graphs Intrinsic and penalty graphs to describe the similarity and dissimilarity between positive and negative feedbacks.These two graphs are constructed

   by using Graph embedded framework tool.

   =2tr[T x(D W )xt ]

   Where = similarity objective for intrinsic graph

   i=set of positive sample set J=set of negative samples = standard deviation

   =Weighting matrix

   Weighting Matrix is calculated by using the following formula

   W 1 N s if l(i)=1 and l(j)=1,i Nsor j N s , 0 else

   Intrinsic graph is used to describe the similarity between positive samples.Consider the case of All positive feedbacks are alike and calculate the pairwise distance between the images.In this case the distance will be decreased.

   ij j i

   Penalty graph is used to describe the dissimilarity between positive feedbacks and negative feedbacks.Consider the case of Each negative sample is in its own way and calculate the pairwise distance between the images.In this case the distance will be increased.

   And then calculate the objective function of the penalty graph by using following formula

   2

   2

   S T x T x W p

   SmiBMMA is used.Introducing the unlabeled samples into BMMA we can have Semisupervised BMMA .construct intrinsic graph for unlabeled samples in

   SemiBMMA.

   p i j ij

   i j

   i j

   i j: jN poriN p

   =2tr[T x(Dp W p )xT ]

   Where = similarity objective for penalty graph

   i=set of positive sample set J=set of negative samples = standard deviation

   =Weighting matrix

   Weighting Matrix is calculated by using the

   Calculate the objective function of the intrinsic graph for unlabeled samples

   u i j ij

   u i j ij

   following formula

   S 1 *

   T x

   T x

   2

   2

   W u

   i j

   i j

   W p 1

   N p if

   2

   l(i)=1 and l(j)=-1,i N por j N p , 0 else

   i j: jNuoriNu

   ij j i

   After constructing the two graphs maximize the objective function of the penalty graph and minimize the objective function of the intrinsic graph.

   To overcome the Second drawback

   =tr[T x(Du Wu )xT]

   =tr[T x xT ]

   Where = similarity objective for unlabeled samples of intrinsic graph

   i=set of positive sample set J=set of negative samples

   = standard deviation

   =Weighting matrix

   Weighting Matrix is calculated by using the following formula

   of the remaining images in the top results are automatically labeled by the system as negative feedbacks.Based on the positive and negative feedbacks RF model can be

   W u 1

   exp(xi xj

   )if l(i)=1 and l(j)=-1,i Nuor j Nu ,0 else

   trained based on various existing

   2

   2

   ij u 2

   N

   j i techniques.Then all images will be resorted based on the new similarity metric.After

   The intrinsic graph of unlabeled samples are introduced into BMMA by using the following formula

   each round of iteration the user will check whether the results are satisfied.If the user is satisfied then the process is ended otherwise the feedback process repeats until the user is

   satisfied with the retrieval results.

   * =argmax tr[ T X(B – L –

   This will lead to SemiBMMA.

   * U)XT ]

2. 1. In content based image retrieval (CBIR) system when the query image is provided by the user the image retrieval system first extract the low-level features.Then all the images in the database are sorted based on a similarity metric,i.e.,Euclidean distance.If the user is satisfied with the results,the retrieval process is ended and the results are presented to the user.If the user is not satisfied with the first retrieval results then the user will label the most semantically relevant images as positive feedbacks in top retrieval results.All

      Fig : CBIR System Architecture

      We propose an effective method for retrieving the images based on user query and content.For this purpose two processes are there. They are (1) User login process

      (2)Retrieving process

      1. User login process

         Input : user enters username and password which are provided by the administrator

         Enter Username and Password

         Enter Username and Password

         System Behaviour : The details entered by the user are validated by the system in such away that whether the username and password are valid or not.If they are valid then it will display the output. Output : system will display the required retrieval page

         System Behaviour :Then the system first consider the content which is provided by the user and then based on the query image it will retrieve the relevant images.

         Server

         Server

         Upload some image and content

         Upload some image and content

         Web search engine

         Web search engine

         Output :sytem will provide relevant images.

         Database

         Display error message

         Display error message

         no Vali

         Yes

         Yes

         date

         Display retrieving page

         Display retrieving page

      2. Retrieving Process

         Input : user upload query image and enter required content which the user wants.

         Relevant images

         Relevant images

         Fig: Proposed System Architecture

3. • User enters username and password which is provided by the administrator.

   • If it is valid then it will displays required page.

   • User uploads a query image and enters some content.

   • System will retrieve relevant images based on the image and content.

In content based image retrieval(CBIR) system images are retrieved based on low-level visual features(color,shape,texturee.t.c) but it will not fill the gap between low-level visual features and high-level semantic concepts. To fill that gap SVM based RF is used as a powerful tool to improve the performance of CBIR system.But by directly using SVM based RF for image retrieval process two drawbacks are there.(1) it will consider positive and negative feedbacks equally.(2) it will not consider unlabeled samples.To overcome these two problems Biased maximum margin analysis(BMMA) and SemiBMMA are introduced.But in these two approaches images are retrieved based on the computation of the similarity between the users query and images via a query by example(QBE) system.But the main problem is that the extracted visual features are widely varied to capture the concept of the users query.

To solve these problem we propose an enhanced method in CBIR system with considering the ambiguity between words and to estimate the semantic similarity between words and entities. These two problems are solved by using web

search engines and then refines the search results.

[1]. Y. Rui, T. Huang, M. Ortega, and S. Mehrotra, Relevance feedback: A power tool in interactive content-based image retrieval, IEEE Trans.Circuits Syst. Video Technol., vol. 8, no.5, pp. 644655, Sep. 1998.

[2]. X. He, D. Cai, and J. Han, Learning a maximum margin subspace for image retrieval, IEEE Trans. Knowl. Data Eng., vol. 20, no. 2, pp.189201, Feb. 2008.

[3]. P. Hong, Q. Tian, and T. S. Huang, Incorporate support vector machines to content-based image retrieval with relevant feedback, in Proc. IEEE ICIP, Vancouver, BC, Canada, 2000, pp.750753.

[4]. X. Zhou and T. Huang,Relevance feedback for image retrieval: A comprehensive review, Multimedia Syst., vol. 8, no. 6, pp. 536544,Apr. 2003.

[5]. X. Zhou and T. Huang, Small sample learning during multimedia retrieval using BiasMap, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recog., 2001, vol. 1, pp. 1117. [6]. S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, Graph embedding and extensions: A general framework for dimensionality reduction, IEEE Trans.

Pattern Anal. Mach. Intell., vol. 29, no. 1, pp.4051, Jan. 2007.

[7]. S. Tong and E. Chang, Support vector machine active learning for image retrieval, in Proc. ACM Multimedia, 2001, pp. 107 118.

[8]. C. Hoi, C. Chan, K. Huang, M. R. Lyu, and I. King, Biased support vector machine for relevance feedback in image retrieval, in Proc,IJCNN, Budapest, Hungary, Jul. 25 29, 2004, pp. 31893194.