Efficient Image Retrieval System Using Sketches

Efficient Image Retrieval System Using Sketches

Vol. 2 Issue 2, February- 2013

Efficient Image Retrieval System Using Sketches

B. Rajalingam M. E.,

Assistant Professor & Department of CSE, Aksheyaa College of Engineering, Puludivakkam, Madurandagam Taluk-607 102

AbstractIn the rising research areas of the digital image processing, content based image retrieval (CBIR) is one of the most popular used techniques. The goal of CBIR is to extract visual content of an image automatically like color, texture, or shape. Most of the image search tools, such as Google images, yahoo image search are based on textual annotation of images. Performances of these tools using text based search methods are not satisfactory. The most important task is to bridge the important gap between the sketch and colored image. Our work is to evaluate the problems and challenges concerned with the design and the creation of CBIR system which is based on sketch based image retrieval (SBIR). Design and task specific descriptors like Edge Histogram Descriptors (EHD), Histogram Oriented Gradient (HOG) and Scale Invariant Feature Transform (SIFT) are used in this project.

1. INTRODUCTION

   Sketch Based Image Retrieval (SBIR) System, which can retrieve the images using sketches in frequently used databases. The efficiency of searching in information set is a very important point of view. In case of texts we can search flexibly using keywords, but if we use images, we cannot apply dynamic methods. The human is able to recall visual information more easily. For example, the shape of an object, or arrangement of colors and objects. Using a sketch based system can be very important and efficient in many areas of the life. In some cases we can recall our minds with the help of figures or drawing. In the following paragraph some application possibilities are analysed.

   The CBIR systems have a big significance in the criminal investigation. The identification of unsubstantial images, tattoos and graffitis can be supported by these systems. Another possible application area of sketch based information retrieval is the searching of analogue circuit graphs from a big database. The user has to make a sketch of the analogue circuit, and the system can provide many similar circuits from the database. The Sketch-based image retrieval (SBIR) was introduced in QBIC [6] and Visual SEEK [17] systems.

   In these systems the user draws color sketches and blobs on the drawing area. The images were divided into grids, and the color and texture features were determined in these grids. The applications of grids were also used in other algorithms, for example in the edge

   histogram descriptor (EHD) method [4]. The disadvantage of these methods is that they are not invariant opposite rotation, scaling and translation.

   Lately the development of difficult and robust descriptors was emphasized. Another research approach is the application of fuzzy logic or neural networks. In these cases the purpose of the investment is the determination of suitable weights of image features[15].

2. PROPOSED WORK

   In this paper proposed work is to develop a content-based associative search engine, which databases are available for. The user has a drawing area, where he can load all shape of images, which are expected to occur in the given location and with a given size. The retrieval results are grouped by color for better clarity. Our most important task is to bridge the information gap between the sketches and the picture, which is helped by own pre- processing transformation process. In our system the iteration of the utilization process is possible, by the current results looking again, thus increasing the precision.

   A Sketch Based Image Retrieval (SBIR) System, which can retrieve using, sketches in frequently used databases. The user has loads an image, which is the base of the retrieval method. Using a sketch based system can be very important and efficient in many areas of the life. In some cases we can recall our minds with the help of figures or drawing. In these systems the user loads an image and blobs on the drawing area. The images were divided into grids, and the color and texture features were determined in these grids. First the user has to loads an image. When the drawing has been finished or the appropriate representative has been loaded, the retrieval process is started.

   The retrieved image first is pre-processed. After that the feature vector is generated, then using the retrieval subsystem a search is executed in the previously indexed database. As a result of searching a result set is raised, which appears in the user interface on a systematic form. Based on the result set we can again retrieve using another descriptor with different nature.

   The system was designed for databases containing relatively simple images, but even in such cases large differences can occur among images in simple size or resolution. In addition, some images may

   be noisier, the extent and direction of illumination may

   situations is that the background

   Vol. 2 Issue 2, February- 2013

   containing several

   vary, and so the feature vectors cannot be effectively compared.

   Fig. 1: The overall structure of the System

   A.Pre-Processing of Sketch Based Image Retrieval System

   Fig. 2: Steps for pre-processing

   In order to avoid it, a multistep pre-processing mechanism precedes the generation of descriptors. The input of the pre-processing subsystem is one image, and the output is the respective processed result set. The main problem during pre-processing of the color images real

   textures and changes generate unnecessary and variable- length edges. As a possible solution texture filters were analysed, for example the entropy calculation based filter. It gives very valuable results, if a textured object of little color stands in a homogenous background. Therefore, the classification of the image pixel intensities minimizes the number of the displayed colors.

   If only some intensity values represent the images, then according to our experience, the color based classification of result images can also be easily implemented. As an approximate method the uniform and minimum variance quantization were used. After the transformation step edges are detected, of which the smaller ones are filtered by morphological opening filter.

   1. The Feature Vector Preparation Subsystem

      In this subsystem the descriptor vectors representing the content of images are made. Basically three different methods were used, namely the edge histogram descriptor (EHD), the histogram of oriented gradients (HOG) and the scale invariant feature transform (SIFT). Our system works with databases containing simple images. But even in such cases, problems can occur, which must be handled. If the description method does not provide perfect error handling, that is expected to be robust to the image rotation, scaling and translation. Our task is to increase this safety. Another problem was encountered during the development and testing. Since own hand-drawn images are retrieved, an information gap arises between retrieved sketch and color images of database. While an image is rich of information, in contrast at a binary edge image only implicit content and explicit location of pixels can be known. This transformation step has to be incorporated into the method, or to be made during the pre-processing. As we wrote in the previous subsection, the images of database were transformed into edge images, so information was lost, however. In order to discover the iplicit content the 2- dimensional distance transform was used.

      Three Stages of Feature Vector Generation:

      1. Indexing

      2. Edge Histogram

      3. Color Layout

         1. Indexing

            Indexing the whole set of images using K-means Clustering algorithm. Indexing is done using an implementation of the Document Builder Interface. A simple approach is to use the Document Builder Factory, which creates Document Builder instances for all available features as well as popular combinations of features .In a content based image retrieval system, target images are sorted by feature similarities with respect to the query (CBIR).In this indexing, we propose to use K- means clustering for the classification of feature set obtained from the histogram. Histogram provides a set of features for proposed for Content Based Image Retrieval (CBIR).

            Hence histogram method further refines the histogram by splitting the pixels in a given bucket into

            several classes. Standard histograms, because of their efficiency and insensitivity to small changes, are widely used for content based image retrieval. But the main disadvantage of histograms is that many images of different appearances can have similar histograms because histograms provide coarse characterization of an image.

            The K-Means Algorithm:

            Algorithm: k-means. The k-means algorithm for partitioning based on the mean value of the objects in the cluster.

            Input: The number of clusters k and a database containing n objects.

            Output: A set of k clusters that minimizes the squared- error criterion.

            Method:

            1. arbitrarily choose k objects as the initial cluster canters:

            2. repeat

            3. re assign each object to the cluster to which the object is the most similar, based on the mean value of the objects in the cluster;

            4. Update the cluster means, i.e., calculate the mean value of the objects for each cluster;

            5. Until no change

         2. Edge Histogram:

            The EHD represents the spatial distribution of edges in an image. The extraction process of the EHD consists of the following stages:

            Fig. 3: Stages of Edge Histogram

         3. Color Layout

         Color is one of the most widely used features in image retrieval. It is robust to background complication and invariant of image size and Orientation.

         • Area of Matching,

         • Color Distance,

         • Spatial Distribution.

         Vol. 2 Issue 2, February- 2013

         Fig. 4: Color Layout

   2. The Retrieval Subsystem

      As the feature vectors are ready, the retrieval can start. For the retrieval the distance based search was used with Minkowski distance [13], and the classification-based retrieval [14].

   3. The Database Management Subsystem

      The images and their descriptors are stored and the necessary mechanism for subsequent processing is provided. This is the database management subsystem, which consists of three parts, the storage, the retrieval, and the data manipulation modules [3]. The storage module provides images, information and the associated feature vectors are uploaded to the database. The file name, size and format of the image are attached.

      The information related to the preparation is gathered, as the makers name, creation date, image title, the brand and type of recording unit. In addition, we may need more information of color depth, resolution, image dimension, vertical and horizontal resolution, possibly the origin of the image, so we take care of their storage. For storage the large images are reduced. The data is stored in a global, not scattered place in the hard disk. The retrieval results are obtained by usage of query module. The retrieval subsystem contacts the database, which provides the descriptors. For optimization it is already loaded at startup to a variable, data structure. If we have the result of retrieval, the database retrieves the result image using the primary key. In addition, statistics can be taken due to a variety of criteria.

   4. The Displaying Subsystem

   Because drawings are the basis of the retrieval, thus a drawing surface is provided, where they can be produced. Also a database is needed for search, which also must be set before the search. In case of large result set the systematic arrangement of search results makes much easier the overviews, so it is guaranteed. The methods in our system cannot work without parameters, and therefore an opportunity is provided to set these as well. The number of results to show in the user interface is an important aspect. Prima facie the first n pieces of results can be displayed, which conveniently can be placed in the user interface.

   This number depends on the resolution of the monitor, and for as much the large resolution monitors are widely used, so this number can move between 20 and 40. Another approach is to define the maximum

   number of results (n), but we also observe that how the goodness of individual results can vary. If the retrieval effectiveness is worse by only a given ratio, the image can be included in the display list. In our system the possible results are classified, and the Obtained clusters are displayed. Hence the color-based clustering for us is the best solution, so our choice was the k-means clustering method [1], which is perfectly suited for this purpose. The implemented user interface can be seen in Fig .7 and Fig .8..

3. EXPERIMENTAL RESULTS

   The system was tested with more than one sample database to obtain a more extensive description of its positive and negative properties. The Microsoft Research Cambridge Object

   Fig. 5: some sample images of the Microsoft Research Cambridge Object Recognition Image Database.

   Vol. 2 Issue 2, February- 2013

   Fig. 6: Some sample images of Flickr 160 database

   Fig. 7: The implemented user interface

   Recognition Image Database was used, which contains 209 realistic objects. All objects have been taken from 14 different orientations with 450×450 resolution. The images are stored in TIF format with 24 bits. This database is most often used in computer and psychology studies. Some images of this database can be seen in Fig.

   1. Another test database was the Flickr 160. This database was used before for measuring of a dictionary- based retrieval system [8]. 160 pieces of general-themed pictures have sorted from the photo sharing website called Flickr. The images can be classified into 5 classes based on their shape. A lot of images contain the same building and moments. The database is accompanied by examples, which is based on the retrieval. Since the test result are documented and the retrieved sketches are also available, so the two systems can be compared with each other. Some images of Flickr 160 database can be seen in Fig. 6.

      symposium on

      symposium on

      Sketches 9th IEEE Internationational Vol. 2 Issue 2, February- 2013

      Applied Machine Intelligence and Informatics,January 2011

      1. D. Comaniciu, and P. Meer, Robust analysis of feature spaces: color image segmentation, IEEE Conference on Computer Vision and Pattern Recognition, pp. 750755, June 1997.

      2. N. Dalal, and B. Triggs, Histograms of oriented gradients for human detection, IEEE Conference on Computer Vision and Pattern Recognition, pp. 886893, July 2005.

      3. T. Deselaers, D. Keysers, and H. Ney, Features for image retrieval: an experimental comparison, Information Retrieval, vol. 11, pp. 77107, December 2007.

         Fig 8: The Uploaded Input Image

         Fig. 8: The first four results can be seen in a separate window

4. CONCLUSIONS

Sketch Based Image Retrieval (SBIR) is an automatic process for searching relevant images based on image features and user inputs. JPEG is an international standard for multimedia content description, and it is an impotant achievement for CBIR. JPEG has a collection of effective descriptors for images, videos, audios and other multimedia contents. The drawn or uploaded image can be compared with color image or its edge representation. The simple smoothing and edge detection based method was used in this project. The Edge Histogram Descriptor (EHD) and Scale Invariant Feature Transform (SIFT) used for image retrieval process. Experimental results on two sample database showed good results. The results show that the sketch based system allows users an intuitive access to search tools.

REFERENCES

1. M. Eitz, K. Hildebrand, T. Boubekeur, and M. Alexa, An evaluation of descriptors for large-scale image retrieval from sketched feature lines, Computers and Graphics, vol. 34, pp. 482498, October 2010.

2. R. Fabbri, L.D.F. Costa, J.C. Torelli, and O.M. Bruno, 2D Euclidean distance transform algorithms: a comparative survey, ACM Computing Surveys, vol. 44, pp. 144, February 2008.

3. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Hiang, B. Dom, Query by image and video content: the QBIC system, IEEE Computer, vol. 28, pp. 2332, 2002.

4. Gy. Gy¨or¨ok, Embedded hybrid controller with programmable analog circuit, IEEE 14th International Conference on Intelligent Systems pp. 59.159.4, May 2010.

5. R. Hu, M. Barnard, and J. Collomosse, Gradient _eld descriptor for sketch based image retrieval and localization, International Conference on Image Processing, pp. 14, 2010.

6. A.K. Jain, J.E. Lee, and R. Jin, Sketch to photo matching: a feature-based approach, Proc. SPIE, Biometric Technology for Human Identification VII, vol. 7667, pp. 766702766702, 2010.

7. A.K. Jain, J.E. Lee, R. Jin, and N. Gregg, Graffiti-ID: matching retrieval of graffiti images, ACM MM, MiFor09, pp. 16, 2009.

8. A.K. Jain, J.E. Lee, R. Jin, and N. Gregg, Content based image retrieval: an application to tattoo images, IEEE International Conference on Image Processing, pp. 27452748, November 2009

9. T. Hashimoto, A. R¨ovid, G. Ohashi, Y. Ogura, H. Nakahara, and A.R. V´arkonyi-K´oczy, Edge detection based image retrieval method by sketches, Proc. of the International Symposium on Flexible Automation, pp. 14, 2006.

10. J.B. Kruskal, Nonmetric multidimensional scaling: a Vol. 2 Issue 2, February- 2013

    numerical method, Psychometrika, vol. 29, pp. 115129, 1964.

11. Y. Liu, and F. Dellaert, A classification based similarity metric for 3D image retrieval, IEEE Conference on Computer Vision and Pattern Recognition, pp. 800805, June 1998.

12. D.G. Lowe, Distinctive image features from scale- invariant keypoints, International Journal of Computer Vision, vol. 60, pp. 91110, 2004.

13. D.G. Lowe, Object Recognition from Local Scale- Invariant Features, IEEE International Conference on Computer Vision, vol. 2, p. 1150, 1999.

14. J.R. Smith, and S.F. Chang, Visual SEEK: a fully automated content based image query system, ACM Multimedia 96, pp. 9798, 1996.

Mr.B.Rajalingam, Assistant Professor, Department of Computer Science and Engineering, Aksheyaa College of Engineering, Puludivakkam, Madurandagam Taluk, Kanchipuram Dist, Tamil Nadu, India. He graduated B.Tech (IT) in Alpha College of Engineering Affiliated by Anna University, Chennai, Tamil Nadu and India. He got ME (CSE) degree from Annamalai University in 2012, Chidambaram, Tamil Nadu and India. He has been actively involved in coordinating and organizing plenty of national events such as seminars and workshops. He has also presented four papers during the conferences to his credit and meticulous guide to UG and

P.G engineering students. He can be reached at rajalingam35@gmail.com.