Content Based Color Image Retrieval Using SVM with Deep Learning Technic

Content Based Color Image Retrieval Using SVM with Deep Learning Technic

A. Gayathri1,

1Assistant Professor, Department of Information Technology,

M N M Jain Engineering College, Chennai, TamilNadu

A.Srinivasan2,

2Professor and Head, Department of Information Technology,

M N M Jain Engineering College, Chennai, TamilNadu

Abstract Content-based image retrieval aims to retrieve images from a database which are similar to each other in terms of their visual contents. Over the years, various feature descriptors based on color, texture and shape information have been proposed to characterize visual elements of image and video data. These features have subsequently been employed to improve the automated search and retrieval of image and videos in CBIR applications. The applications of CBIR are: users searching a particular image on the web, Various types of professionals like police force for picture recognition in crime prevention, Medicine diagnosis, Architectural and engineering design, Fashion and publishing, Geographical information, remote sensing systems and home entertainment. However, selecting and developing methods for correctly identifying and effectively integrating suitable visual features for a specific vision task remain a very challenging problem. The proposed method uses support vector machines for retrieval of images based on content. First, 190 dimensional feature vectors are extracted and similarity metrics is defined. Using the SVM learning techniques, the images which are similar to the query image are retrieved.

Keywords Content based image retrieval; SVM

1. INTRODUCTION

   Content Based Image Retrieval is a method of retrieving images from a database based on the features of the image. Features such as color, texture, shape and contrast are used in image retrieval. A query image is chosen as input. Images which are similar to the query image from the database are retrieved and displayed as output. The objective is to retrieve images from a database which are similar to each other in terms of their content. Selecting methods for correctly identifying and effectively integrating suitable features for a image to retrieve similar images from the database.

   • Select a query image.

   • Extract features of the query image and the candidate images.

   • Apply existing method to find the image similarity and retrieve images.

   • Apply SVM for classification and retrieve similar images.

   • Compare the existing and proposed method.

2. LITERATURE SURVEY

   There are various methods which have been proposed for content based image retrieval. In [1], hierarchical visual

   structuring, color perception and compressed sensing are used to perform CBIR. Using 2D CS measurement model, hierarchical HSV features and SGLCM texture features are extracted. These features are used to compute the similarity between the query image and the candidate images in the database. It required empirical setting of parameters in the feature fusion step and also its performance was inconsistent. In [6], a feature combination method with similarity measure for CBIR was proposed which improves the precision rate in comparison with the other state-of-art methods. In [3], a sparse multimodal learning approach was proposed to combine features using joint structured sparsity regularizations. The recall rate is high whereas problem exists with correlation. In [4], a Bayesian merging approach was used to down-weight the indexed feature in the intersection set to overcome the correlation problem. In [5], a coupled multi-index framework is proposed to overcome information loss using SIFT. This approach improves the recall rate.

   In recent years deep learning techniques have gained importance in the field of CBIR. In [2], a deep learning framework for CBIR by training large-scale deep convolutional neural networks for learning effective feature representations of images is introduced. The performance is inconsistent and retrieval time is high. There are many techniques used like Deep belief network, Boltzmann machines, Deep Boltzmann machines, Deep Neural Networks, Support Vector Machines, etc. In [7] k-NN classification technique is used for CBIR but only color features are used. Images are retrieved correctly using this technique but retrieval time is high. In [8] feature extraction was seen as the binary classification problem and SVM (Support Vector Machine) was used for solving this problem in CBIR. In this SVM is used as the classifier which is performing the task of classifying the image and this process of classification is given to all the traits of the image which are extracted after the feature extraction process. It is mostly used for estimating the highest margin hyper planes within the feature space which is also a high dimensional feature space. Very few features are extracted. So feature extraction process is a disadvantage. In [9] a cluster based support vector machine technique for the retrieval of images from the database. The aim is to retrieve images efficiently in large databases. The system has two phases: preprocessing phase and image retrieval phase. The retrieval accuracy is remarkable but the retrieval time taken is a disadvantage.

   However, research is still required to develop methods that are capable of retrieving images from the database accurately.

3. EXISTING SYSTEM

   In the existing system [1], hierarchical visual structuring, color perception and compressed sensing are used to perform CBIR. The image is converted from RGB to HSV and then discrete cubic partitioning is performed. Then hierarchical operators and SGLCM are defined in HSV space. Using 2D CS measurement model hierarchical HSV features and SGLCM texture features are extracted. These features are used to compute the similarity between the query image and the candidate images in the database.

   Firstly the query image is processed using hierarchical operators. Then each resultant hierarchical mapping matrix is processed to compute a two dimensional compressive sensing measurement. Then texture features are extracted using SGLCM. Candidate images are retrieved from the database for which color and texture features are extracted for comparison with the query image. A normalized similarity score is found for every image and the images are ranked based on similarity. The image with the highest similarity score is ranked first. The images are retrieved according to the rank.

   Corel 1000 dataset is being used in the existing system. Precision, Recall, F-Score and retrieval time is used for measuring the performance of the system. Compared to other approaches this method performs better with an average precision of 80.1. However it still has some limitations. It requires empirical setting of parameters in the feature fusion step and inconsistent performance is also observed. Dissimilar images are also retrieved at times. Thus, we have proposed a method for improving the performance and retrieval of images using k-NN and SVM learning techniques.

4. PROPOSED SYSTEM

   In our proposed system, we perform content based image retrieval using support vector machine. SVMs are supervised learning models used for classification and regression analysis.

   The Fig. 1 explains about the architecture of the proposed system. The features are extracted from the images. The classification is done using SVM and the similar images are retrieved as output.

   Fig. 1 System Architecture

   The system is divided into two main steps which are as follows.

   1. Featre Extraction

      In this step the raw pixel values are mapped into feature space. The below steps are applied to every image of our dataset to extract features of every image. The steps are following:

      • Color histogram is computed for every image using H, S, V component by uniformly quantizing it in 8,2 and 2 bins. This produces a vector of 32 elements.

      • The image is quantized into 64 elements by computing the auto-correlogram of every image in RGB space.

      • The mean and standard deviation is extracted for every RGB channel producing 6 vector elements.

      • We produce 48 elements by applying Gabor wavelet filters for each image across four scales and six orientations.

      • 40 feature elements are extracted from every image by applying wavelet transform with a three level decomposition.

      • Finally all the vectors are combined and a new vector is formed which comprises of 190 dimensions.

        Thus a feature vector database is formed for all the images of the database.

   2. Classification

      SVM technique is used for multiclass classification problem. It is used to classify the different sets of images present in our dataset. There are two approaches to solve this issue.

      1. One vs All approach

         This involves training a single classifier per class as positive samples and all other samples as negatives. This strategy requires the base classifiers to produce a real-valued confidence score for its decision.

      2. One vs One approach

      This trains n!/(nk)!k! binary classifiers for a k-way multiclass problem. Each receives the samples of a pair of classes from the original training set, and must learn to distinguish these two classes. A voting scheme is applied at prediction time. All n!/(nl)!k! classifiers are applied to an unseen sample. The class that gets the highest number of +1 predictions gets predicted by the combined classifier.

      Thus by this method the candidate images are retrieved which are similar to the query image.

      The Fig. 2 represents the data flow diagram of the system.

      Fig. 2 Data Flow Diagram of the proposed system.

5. IMPLEMENTATION

   We have used MATLAB R2016a to implement our approach for content based image retrieval. The following are the steps to be followed for implementation:

   • An image directory is created for processing the image and a database is created which consists of the features of the image.

   • We load our dataset.

   • We browse the query image from our dataset and select one.

   • We apply the above mentioned techniques and retrieve the images which are similar to the query image.

   The dataset being used is called as Wang database. It consists of 1000 images from 10 different categories where 100 images correspond to each category. The 10 categories or classes are: Africa, Beach, Monument, Bus, Dinosaur, Elephant, Flower, Horse, Mountain and Food. Fig. 3 represents one picture from every above mentioned category.

   Fig. 3 Sample images from every category of the dataset.

   Below are output results for existing and proposed system. Fig. 4 and Fig. 6 are of the existing system. Fig. 5 and Fig. 7 are of the proposed system.

   Fig. 4 Retrieved images in existing system when query image is from the Dinosaur class.

   Fig. 5 Retrieved images of proposed system when query image is from the Dinosaur class.

   Fig. 6 Retrieved images in existing system when query image is from the Bus class.

   Fig. 7 Retrieved images of proposed system when query image is from the Bus class.

   Thus the proposed system which uses SVM technique performs better compared to the existing method of similarity based ranking. The analysis is given below. In Fig. 6 there are few images not similar to the query image, but in Fig. 7 we notice that all the images belong to the same category as of the query image. Thus SVM classifier is proved to be better than the similarity based ranking.

6. EXPERIMENT ANALYSIS

   Evaluation of retrieval performance is a crucial problem in Content-Based Image Retrieval. Many different methods for measuring the performance of a system have been created and used by researchers. We have used the most common evaluation methods namely, Precision, Recall and F-Measure usually presented as a Precision, Recall and F-measure graph. Precision and recall alone contain insufficient information. We can always make recall value 1 just by retrieving all images. In a similar way precision value can be kept in a higher value by retrieving only few images or precision and recall should either be used together or the number of images retrieved should be specified. With this, the following formulae are used for finding Precision, Recall and F-measure values.

   The Precision is represented by P. The following formulae describe how precision can be calculated:

   P = no. of relevant images retrieved / number of images requested

   The F-measure is used to represents the harmonic mean of precision and recall i.e.

   F=2RP/R+P

   The F-measure (also F-score) is a score to measure the accuracy of the system.

   The TABLE I. shows the Precision and F-score results for the results obtained for the query image from every class using similarity based ranking i.e. existing system.

   TABLE I. Comparison of precision and F-score for query image from every class of the dataset using existing approach.

   CLASSES	PRECISION (PERCENTAGE)	F-SCORE (PERCENTAGE)
   Africa	65	78.8
   Beach	70	82.35
   Monument	45	62
   Bus	75	85.71
   Dinosaur	100	100
   Elephant	85	91.89
   Flower	100	100
   Horse	80	88.8
   Mountain	75	85.71
   Food	75	85.71

   The TABLE II. shows the Precision and F-score results for the results obtained for the query image from every class using Support Vector Machine i.e. proposed system.

   TABLE II. Comparison of precision and F-score for every class of

   dataset using SVM.

   CLASSES	PRECISION (PERCENTAGE)	F-SCORE (PERCENTAGE)
   Africa	95	97.43
   Beach	100	100
   Monument	100	100
   Bus	100	100
   Dinosaur	100	100
   Elephant	100	100
   Flower	100	100
   Horse	100	100
   Mountain	100	100
   food	100	100

   The TABLE III. shows the comparison of average Precision and average F-score values between the existing and the proposed approach.

   TABLE III. Comparison of the average precision and average F- score of existing and proposed approach.

   AVERAGE	SIMILARITY BASED RANKING	SUPPORT VECTOR MACHINE
   precision (percentage)	77	99.5
   f-score (percentage)	85.6	99.7

   Fig. 8 and Fig. 9 depict the increase in the precision ad F- score of the image retrieval compared to the existing approach. Thus the proposed SVM classifier performs better.

   Fig 8. Comparison of average precision of existing and proposed approach.

   Fig. 9 Comparison of average F-score of existing and proposed approach.

7. CONCLUSION AND FUTURE WORK

Thus, the aim is to retrieve similar images from the database for the given query image using support vectors. We see better performance and retrieval accuracy compared to the other state-of-art methods. But, if the SVM algorithm predicts the wrong class label for a query image, then the retrieved image will be from a wrong class since the wrong label was predicted. We hope to overcome this in the future.

REFERENCES

1. Yan Zhou, Fan-Zhi Zeng, Hui-min Zha, Paul Murray, Jinchang Ren. Hierarchical Visual Perception and Two-Dimensional Compressive Sensing for Effective Content-Based Color Image Retrieval. Springer, 2016.

2. Wan J et al. Deep learning for content-based image retrieval: a comprehensive study. In: Proceedings of the 22nd ACM international conference on multimedia, 2014; pp. 15766.

3. Wang H, Nie FP, Huang H et al. Heterogeneous visual features fusion via sparse multimodal machine. CVPR, 2013; pp. 3097102.

4. Zheng L, Wang S, Zhou W et al. Bayes merging of multiple vocabularies for scalable image retrieval. CVPR, 2014; pp. 196370, in Columbus, Ohio, USA, June 2014.

5. Zheng L, Wang S, Liu Z et al. Packing and padding: coupled multi- index for accurate image retrieval, CVPR, 2014; pp. 194754, in Columbus, Ohio, USA, June 2014.

6. Sadegh Fadaei, Rassoul Amirfattahi, Mohammad Reza Ahmadzadeh; A New Content-Based Image Retrieval System Based on Optimized Integration of DCD, Wavelet and Curvelet Features; IET Image Processing; 2016.

7. Ray-I Chang, Shu-Yu Lin, Jan-Ming Ho, Chi-Wen Fann and Yu-Chun Wang; A Novel Content Based Image Retrieval System using K- means/KNN with Feature Extraction; ComSIS Vol. 9, No. 4, Special Issue, December 2012.

8. Priyanka Sharma; Content Based Image Retrieval Using SVM.; International Journal of Computer Science Trends and Technology (IJCST), Volume 4 Issue 4, Jul – Aug 2016.

9. Sumiti Bansal, Er.Rishamjot Kaur; Content Based Image Retrieval Using C-SVM Technique.; International Journal of scientific research and management (IJSRM), volume 3 issue 4, April 2015.