Tag-Based Social Image Retrieval Systems Using Neighbour Voting Algorithm

Tag-Based Social Image Retrieval Systems Using Neighbour Voting Algorithm

Vivek Sharma1, Deepak Sharma2, Prof. Gajendra Singp

1. M.Tech Scholar, Dept. of Computer Science and Engineering, SSSIST, Sehore, India

2. M.Tech Scholar, Dept. of Information Technology, SSSIST, Sehore, India

3. Professor & Head, Department of Computer Science and Engineering, SSSIST, Sehore, India

Abstract

Tags associated with social images are valuable information source for superior image search and retrieval experiences. Social image retrieval is important for exploiting the increasing amounts of amateur-tagged multimedia such as Flickr images. Intuitively, if different persons label similar images using the same tags, these tags are likely to reflect objective aspects of the visual content. Interpreting the relevance of a user-contributed tag with respect to the visual content of an image is an emerging problem in social image retrieval. An algorithm is proposed that scalably and reliably learns tag relevance by accumulating votes from visually similar neighbours. Treated as tag frequency, learned tag relevance is seamlessly embedded into current tag-based social image retrieval paradigms.

Keywords- tag relevance, image retrieval, neighbour voting ,user contributed tag, social image tagging.

1. Introduction.

   All Image sharing websites such as Flickr and Facebook are hosting billions of personal photos. Tagging is a significant feature of social bookmarking systems which enables users to add, annotate, edit and share bookmarks of a web documents. Social image tagging, assigning tags to images by common users, is reshaping the way people manage and access such large-scale visual content. Image tagging basically refers to a process of categorizing or mapping of images on the basis of their contents either visual or context. Along with the rapid growth of personal

   albums in social networking sites, it has been seen that tagging is the most promising and practical way to facilitate the huge photos database semantically searchable. To tag an image firstly the training set is manually tagged and then the tags of the testing set are automatically predicted.

   Image tagging can be done in two ways:-

   1. Manual Image Tagging

   2. Automatic Image Tagging.

      An image retrieval system is a computer system for browsing, searching and retrieving images from a large database of digital images.

      Improving Image Tagging- Image tagging can be improved by tagging the images on the basis of their features and tags should be relevant to the image and with the help of which image can be retrieved from pool of the databases.

      Improving Image Retrieval- Image retrieval can be improved on the basis of the content as well as the features, characteristics, color etc of the image.

2. Related Works.

   Most germane to this work are efforts in the traditional IR environment that rank documents matching a given query in descending order of relevance.

   Example dimensions are term weight (e.g., inverse document frequency), document-term weight (e.g., term frequency), and document length normalization. Each dimension may be realized by many alternative formulations. For instance, more than eight formulations for document length normalization are enumerated in. Due to the large number of combinations of alternative formulations, a subset of relevance measures was evaluated on the trec dataset.

   In this work, we share a similar objective to systematically explore and evaluate the relevance measures between tag queries and tagged social images. The major difference between our work and aforementioned efforts is that a textual document contains much redundancy of words to conveys its semantic whereas images are usually associated with only few tags. Besides, redundancy of tags is minimum in many social image tagging systems. Particularly, in Flickr, a tag cannot be assigned more than once to the same image. Moreover, the tags are assigned by different users with different motivations and different criteria for determining the degree of relatedness of an image to a tag. All these differences demand systematic investigation of the impact of different formulations on image search ranking. In the following, we first distinguish TagIR from automatic image annotation. Next, we address related work from a number of recent research efforts toward understanding image tagging, including: motivations for tagging[1, 3, 11], tagging systems[4,3], and tag types [5, 10, 12, 13] and tag

   relatedness [16,17] and tag representativeness [19].

3. Basic Idea Of Image Retrieval System.

General goal of image retrieval systems are: 1.It must able to process natural language query.

2.Search must be performed among annoted and non- annoted images and considers human visual perception. 3.It must take account the various features of an image. The images can be automatically indexed by summarizing their visual features in image retrieval systems. A feature is one of the important characteristic which captures a certain visual property of an image either globally for the entire image or locally for regions or objects. Color, texture and shape are commonly used features in systems.

Various techniques of image retrievals.

1. Content Based Image Retrieval-

   The content based image retrieval techniques use low- level image features."Content-based" means that the search analyzes the contents of the image rather than the metadata such as keywords, tags, or descriptions associated with the image. The term "content" in this context might refer to colors, shapes, textures, or any other information that can be derived from the image itself.

2. Text Based Image Retrieval-

   The text-based image retrieval techniques use keywords.Text-based image retrieval is used to retrieve the XML documents containing the images based on the textual information for a specific multimedia query.

   It is also called description-based image retrieval. To overcome the limitations of CBIR, TBIR represents the visual content of images by manually assigned tags or keywords.

   TBIR represents the visual content of images by manually assigned keywords/tag. It allows a user to present his/her information need as a textual query, and find the relevant images based on the match between the textual query and the manual annotations of images.

3. Multimodal Fusion Image Retrieval-

   The multimodal fusion techniques use combination of various image representative feature. Multimodal fusion image retrieval involves data fusion and machine learning algorithms. Data fusion, also known as combination of evidence, is a technique of merging multiple sources of evidence.

   By using multiple modalities,we can learn the skimming effect, chorus effect and dark horse effect.

   Advantages:

   1. Machine learning algorithms are used to study and classify the combination of modalities that represent images or regions.

   2. The skimming effect is used when the topranked documents are fused to increase the recall and precision of the retrieved documents.

4. Semantic Based Image Retrieval-

The semantic-based techniques use concepts. Image retrieval based on the semantic meaning of the images is currently being explored by many researchers. This is one of the efforts to close the semantic gap problem.In this context, there are two main approaches:

Annotating images or image segments with keywords through automatic image annotation or adopting the semantic web initiatives.

1. Improving Image Retrieval.

   Image retrieval can be improved on the basis of the content as well as the features, characteristics, color etc of the image. First of all the query image is loaded and then its neighbour images are retrieved on the basis of features it could be text based image retrieval or content based image retrieval in which retrieval of images is done on the basis of text or content of the image it can be anything like color, feature, characteristics. Retrieval of images can be done for labeled as well as for unlabelled images. In labeled image retrieval images are retrieved on the behalf of tags which differentiate each group from other. Image with similar features are

   grouped together and will be retrieved in a group only whenever a feature of the grouped is being called for images containing that feature, the whole group will be

   retrieved. While for unlabelled images retrieval is done again on the basis of grouping of images and tags are being predicted on the behalf of the characteristics of similar group images, tag prediction for unlabelled images is done with the help of the features of all the pictures which are being retrieved and the features which are present in all the retrieved images are considered as tags and are being predicted for the whole group together.

2. Image Retrieval Using Neighbour Voting Algorithim.

   To fulfill image tagging, the measurement should rank tags relevant with respect to an image ahead of tags irrelevant with respect to the image. From our earlier discussions we know that if different persons label visually similar images using the same tags, these tags are most probable to reflect objective aspects of the visual content. This suggests that the relevance of a tag given an image might be inferred from how visual neighbours of that image are tagged: the more regular the tag occurs in the neighbour set, the more relevant it might be, to the query image. Thus, a good tag relevance measurement should take into account the distribution of a tag in the neighbour set and in the entire collection, at the same time. Motivated by the informal analysis

   above, I propose a neighbour voting algorithm for learning tag relevance. Though the proposed algorithm is simple, I deem it important to gain insight into the rationale for the algorithm. The following two subsections explain it. Firstly in I have defined two conditions to describe the goal of tag relevance learning. After which, in I have provided a formal analysis of user tagging and content-based nearest neighbour searches. Then we observe how our algorithm is naturally derived from the analysis.

   Major notations which are used in the proposed algorithm:

   Proposed Algorithm:

   Input: A user tagged image.

   Output: (Tag relevance( t , I , k) , that is the tag relevance value of each tag t in I. Find the k-nearest visual neighbours of I from the collection with the unique user constraint that is a user has

   at most one image in the neighbour set.

   for tag t in tags of I do Tag relevance(t, I , k) 0 end for

   for image J in the neighbour set of I do for tag t in tags _ of _ J tags _ of _ I do

   Tag relevance(t, I , k) Tag relevance(t, I , k) +1 end for

   end for

   Tag relevance(t, I , k) Tag relevance(t, I , k) Prior (t, k)

   Tag relevance(t, I , k) max(Tag relevance(t, I , k).1)

   Flow Chart of Algorithm

3. Description of Proposed Algorithm.

   Image tagging can be done in two ways:

   1. Tag suggestion or prediction

   2. Tag based search button

First of all an input query image is loaded then we can search its neighbour images after that we get the top 20 neighbour images of the loaded query image retrieved on the basis of image ranking. Then in order to find tags for labeled images we enter user tag and then related to it we get the tag suggestion for labeled images. If images are unlabeled by clicking on the search button we can get the tag suggestion for unlabeled images also according to top rank priority. When a query image is loaded and we try to find its neighbour images then on the basis of image ranking the top 20 images are retrieved which are the most matching neighbour images of that query image.If we want to search tag suggestion for labeled images then after entering the user tag on the basis of which searching is done, after that tags matching with that entered tags are retrieved, the top 5 tags which matches with the entered user tag are retrieved on the basis of priority matching with the called tag.

Result Analysis:

For image retrieval, images relevant with respect to user queries should be ranked as high as possible. Meanwhile, ranking quality of the whole list is important not only for user browsing, but also for applications using search results as a starting point. For tag suggestion, tags relevant with respect to user images should be ranked as high as possible. Also, the candidate tag list should be short such that users pick out relevant tags easily and efficiently. Thus, the following two standard criteria are adopted to measure the different aspects of the performance. Given a ranked list of L instances where an instance is an image for image retrieval and a tag for tag suggestion, we measure

Precision: The proportion of relevant instances in the top n retrieved results, where n l. The percentage of no. of relevant images out of retrieved images is known as precision.

Average precision (AP): AP measures ranking quality of the whole list. Since it is an approximation of the area under the precision-recall curve [38], AP is commonly considered as a good combination of precision and recall, For evaluation of the overall performance, we use mean average precision abbreviated as MAP, a common measurement in information retrieval. MAP is the mean value of the AP over all queries in the image retrieval experiment and all test images in the tag suggestion experiments.

6.1 Comparison with Other Algorithms.

Table shows the comparison of various algorithms with the proposed method and the value of irrelevant images at different number of visual image search.

Algorithm	Among Top 20	Among Top 10	Among Top 5
Heuristic Rank	1.93	1.42	0.86
Visual Rank	0.83	0.47	0.3
Proposed Algorithm	0.74	0.45	0.4

Graph Shows the comparison of different algorithms:

CONCLUSION .

The algorithm produces a good tag relevance measurement for both image ranking and tag ranking. Also, since the proposed algorithm does not require any model training for any visual concept, it is efficient in handling large-scale image data sets.To verify the algorithm, three experiments were conducted on two thousand photos: one image ranking experiment and two tag ranking experiments. For the image ranking experiment, social image retrieval is improved by using learned tag relevance as updated tag frequency in a general tag-based retrieval framework. For the tag ranking experiments, two settings are considered, i.e., tag suggestion for labeled images and tag suggestion for unlabeled images. In the tag suggestion experiment for labeled images, the algorithm finds more tags which describe visual aspects of an image. In the tag suggestion experiment for unlabeled images, the algorithm compares favorably against two baselines. Specifically, we effectively restrain high frequency tags without overweighting rare tags. This study demonstrates that the proposed algorithm predicts more relevant tags even when the visual search is unsatisfactory. In short, all the three experiments show the general applicability o tag relevance learning for both image ranking and tag ranking.

FUTURE WORK.

We will try to reduce the waiting time while retrieving tag based search result. We will try to apply optimization technique in order to reduce the waiting time and try to increase our database in order to get more neighbour images of the tags

REFERENCES.

1. M. Ames and M. Naaman. Why we tag: motivations for annotation in mobile and online media. In Proceedings of the SIGCHI conference on Human factors in computing systems, San Jose, California, USA, 2007. ACM.

2. X. Li, C. G. M. Snoek, and M. Worring, Learning tag relevance by neighbour voting for social image retrieval, in Proc. ACM MIR, 2008, pp. 180187.

3. C. Marlow, M. Naaman, D. Boyd, and M. Davis. Ht06, tagging paper, taxonomy, flickr, academic article, to read.In Proceedings of the seventeenth conference on Hypertext and hypermedia (HYPERTEXT06), pages 3140,Odense, Denmark, 2006. ACM.

[4]. S. A. Golder and B. A. Huberman. Usage patterns of collaborative tagging systems. J. Inf. Sci., 32:198 208, 2006.

[5]. K. Bischoff, C. S. Firan, W. Nejdl, and R. Paiu. Can all tags be used for search? In Proceeding of the

17th ACM conference on Information and knowledge management

(CIKM08), pages 193202, Napa Valley, California, USA, 2008. ACM.

[6].Anil K. Jain, Fundamental Of Digital Image Processing, Pearson Education 1st edition, 1989.

[7]. Lin Chen, Dong Xu, Ivor W. Tsang, and Jiebo Luo, Tag-Based Image Retrieval Improved by Augmented Features and Group-Based Refinement IEEE Trans.

Multimedia,vol. 14, no. 4, Aug 2012.

[8]. K. Weinberger, M. Slaney, and R. van Zwol, Resolving tag ambiguity, in Proc. ACM Multimedia, 2008, pp. 111119.

[9]. Rafael C. Gonzalez and Richard E. Woods, Digital Image Processin,Pearson Edu 2nd edition, 2005

[10]. S. Overell, B. Sigurbj¨ornsson, and R. van Zwol. Classifying tags using open content resources. In Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM09), pages 6473, Barcelona, Spain, 2009. ACM.

[11]. A. Zollers. Emerging motivations for tagging: expression, performance, and activism. In Proc. of Tagging and Metadata for Social Information Organization Workshop with WWW, Banff, Alberta, Canada, 2007. ACM.

[12]. B. Sigurbj¨ornsson and R. van Zwol. Flickr tag recommendation based on collective knowledge. In Proceeding of the 17th international conference on World Wide Web (WWW08), pages 327336, Beijing, China, 2008. ACM.

[13]. B. Sigurbj¨ornsson and R. van Zwol. TagExplorer: Facetedbrowsing of flickr photos. Technical Report YL-2010-

005, Yahoo! Research, 2010.

[14]. M. E. I. Kipp and G. D. Campbell. Patterns and Inconsistencies in Collaborative Tagging Systems: An Examination of Tagging Practices,Annual General Meeting of the American Society for Information Science and Technology, 2006.

[15]H. Halpin, V. Robu, and H. Shepherd. The complex dynamics of collaborative tagging. Proceedings of the 16th International Conference on World Wide Web, pages 211220, 2007.

[16] X. Li, C. G. M. Snoek, and M. Worring. Learning social tag relevance by neighbour voting. IEEE Trans. Multimedia, 11(7):13101322, 2009.

[17]. X. Li, C. G. M. Snoek, and M. Worring.

Unsupervised

multi-feature tag relevance learning for social image retrieval. In Proceedings of the ACM International

Conference on Image and Video Retrieval (CIVR10), pages1017, Xian, China, 2010. ACM.

[19]. A. Sun and S. S. Bhowmick. Image tag clarity: in search of visual-representative tags for social images. In Proceedings of the first SIGMM workshop on Social media

(WSM09), pages 1926, Beijing, China, 2009. ACM. [20]. Dr.H.B.Kekre, Sudeep D. Thepade, Color Based Image Retrieval using Amendment Block Truncation Coding with YCbCr Color Space, International Journal on Imaging (IJI), Volume 2, Number A09, Autumn 2009, pp. 2-14.