Scalable Learning Based Framework for Pruning CBIR System Using Visual Art Image

Scalable Learning Based Framework for Pruning CBIR System Using Visual Art Image

Y. Poornima

Research Scholar, Dept of Computer Science.

Vinayaka Mission University Salem, India

1. Hiremath

   Dept. of Computer Science Gulbarga University Gulbarga, India

   Abstract The proposed system introduces a framework to perform visual art image retrieval that is conducted by feature extraction using visual descriptors and filtering of datasets. The training of the system is done using Support vector machine (SVM) and Principal Component Analysis (PCA) on the extracted features in order to maintain a better recall and precision rate in the retrieved results. The system mainly comprises two phases: dataset creation and retrieval system. The final preciseness of the retrieved results is achieved using relevance feedback. As the system is designed over content based retrieval system model, the problem of false positives images in retrieval system is addressed using relevance feedback. Finally, the proposed system is also compared with prior techniques of CBIR system where the results exhibit better accuracy in the retrieval process.

   Keywords-component; Visual Art Images, Content Based Image Retreival System, SVM, PCA, Visual Descriptors

   1. INTRODUCTION

      Content-based image retrieval has been proposed as a viable alternative to text-based image retrieval. Aiming at the retrieval of images on the basis of automatically extracted visual features such as intensity, color, texture and shape, the strength of content-based image retrieval mainly stems from its ability to search for an image depending on metrics for comparing image or structure properties that can match human judgments of similarity [1]. Response times in the interaction between computer systems and human users are of great importance to user satisfaction. At present, this fact is not widely explicitly addressed in CBIR: many authors discuss mechanisms for reducing search time, but few quote actual times. The goal should be a suitable trade-off between response time and the quality of the results. Most current CBIRSs represent images as points in a multidimensional feature space. Image similarity is defined as the Euclidean or Mahalanobis distance between points in this vector space. Many authors propose methods for reducing the time taken to search this space, such as dimensionality reduction using Principal Components Analysis (PCA) [2, 3], clustering [4, 5], or spatial search structures such as KD-trees [6]. The suitability of PCA as preprocessing for information retrieval has been challenged, since it can eliminate the rare feature variations which can be very useful for creating a specific query. The other techniques limit search by pruning the number of images for which distances are calculated. In an earlier study, the performances of several weighting schemes for CBIR were investigated, both with and without RF [7]. This study confirmed the efficacy of RF, and allowed the best-performing weighting method to be selected.

      In spite of this capacity, however, mainstream content-based methods do not satisfy the complex demands created by visual art imaging. In contrast to real-world images, visual art images have specic characteristics that highly impact the design of content-based image

      retrieval methods. For example, visual art images, unlike other images, pose additional challenges in that visual features of normal and complex structures are typically separated by only subtle differences in visual appearance. The success of content-based image retrieval hinges on the consideration of the typical visual art image characteristics. Tagare et al.[8] identify four characteristics of medical images: a) the semantics of medical knowledge extractable from images is imprecise, b) image information contains form and spatial data, which are not expressible in conventional language, c) a large part of image information is geometric and d) diagnostic inferences derived from images rest on an incomplete, continuously evolving model of normality. To explore the impact of these characteristics, Tagare et al.[8] categorize content-based image retrieval along three dimensions: i) the extent of understanding and reasoning about the image content, ii) the ease with which the query mechanism allows the user to specify what the user wants, iii) the extent of interaction required at data entry or during retrieval. Salient points are locations in an image where there is a signicant variation with respect to a chosen image feature. Since the set of salient points in an image capture important local characteristics of that image, they can form the basis of a good image representation for content-based image retrieval (CBIR). The features for a salient point should represent the local characteristic of that point so that the similarity between features indicates the similarity between the salient points. The proposed system introduces a framework to perform visual art image retrieval that is designed keeping scalability of dataset and efficiency of the retrieval result to preciseness. Finally the proposed system is compared with prior research work to demonstrate its efficiency.

   2. RELATED WORK

      There are large number of academic and commercial CBIR systems have been developed by universities, government organizations, companies, and hospitals. Comprehensive surveys of these techniques and systems used in visual art image retrieval systems are discussed in this section.

      The authors in [9] have focused on the nuclei detection on Hematoxilin eosin stained colon tissue sample images. It examines that how effectively the algorithms used during the process can be implemented to data parallel architectures, and is it worth using GPU (Graphic Processing Unit) instead of the CPU (Central Processing Unit).

      Liguang et al. [10] has demonstrated a software framework and database system for visual art image related to Chinese human genetic resources was proposed and implemented base on web database technique and SOA architecture (Service Oriented Architecture).

      Liviu Constantinescu et al. [11] has presented a system that allows visual art imaging data to be embedded within an online PHR (Personal Health Records), and from there to be accessed interactively, with full capacity for manipulation, by practitioners

      anywhere in the world, using any of a vast number of portable and handheld computing devices, in addition to the standard desktops.

      Puyan Mojabi and Joe LoVetri [12] have used the multiplicative regularized GaussNewton inversion (MR-GNI) method for microwave visual art imaging assuming the two-dimensional (2D) transverse magnetic (TM) illumination.

      Jayaram K. Udupa et al. [13] has presented in the approach implemented in the system consists of (i) segmentation and interpolation, (ii) surface tracking, and (iii) creation of rendered images.

      Deniz Sarioz et al. [14] has discussed their work toward obtaining topological and geometric descriptors in the form of Betti triple sequences from such a linear combination of blobs, and give some preliminary experimental results.

      Shih-Fu Chang et al. [15] have first discussed the scope, basic terminology, and potential applications of MPEG-7 standard, and then discussed the constituent components. Then also compared the relationship with other standards to highlight its capabilities.

      Mathias Lux et al. [16] has developed LIRe system for lucene image retrieval.

      Raul Alonso-Calvo et al. [17] has presented a cloud computing service prototype for storing, analyzing and retrieving images and its assoiated information. Image representation and processing is graph- based, region oriented. Thus, the system is able to extract and store image regions (particularly, piecewise-constant regions) with their relevant features and relationships. Then, it is able to process stored images or to retrieve them.

      Henning Muller et al. [18] has presented in this paper goal of the imagebased retrieval task was to retrieve an ordered set of images from the collection that best met the information need specified as a textual statement and a set of sample images, while the goal of the casebased retrieval task was to return an ordered set of articles (rather than images) that best met the information need provided as a description of a case.

      Thomas M. Lehmann [19] has presented a novel multi-step approach, which is specially designed for content-based image retrieval in medical applications (IRMA). In contrast to common approaches, the IRMA-concept is based on a conceptual and algorithmic separation of: (a) image categorization using global features, (b) geometry and contrast registration with respect to prototypes within the categories, (c) extraction of local features, (d) category and query dependent local feature selection, (e) index generation resulting in hierarchical multi-scale blob representations,

      (f) object identification that links a-prior knowledge on image content to the blobs, and (g) image retrieval processed on the abstract blob- level.

      Thomas M. Lehmann et al. [20] has presented a comparative evaluation of methods for automatic categorization of medical images. Automatic categorization is a first step towards the use of data mining methods on medical image databases and it is obviously necessary for medical applications such as cased-based reasoning.

      Yong Rui et al. [21] has proposed a relevance feedback based interactive retrieval approach, which effectively takes into account the above two characteristics in CBIR. During the retrieval process, the users high level query and perception subjectivity are captured by dynamically updated weights based on the users feedback.

      Si Yong Yeo [22] has demonstrated an active contour model using the level set method is proposed for N-dimensional segmentation of visual art images. The proposed model is based on an image attraction force derived from geometric interactions between the active contour model and the image object boundaries.

      Emrah Bostan et al. [23] has propose a novel statistical formulation of the image reconstruction problem from noisy linear measurements.

      They also derive an extended family of MAP estimators based on the theory of continuous-domain sparse stochastic processes.

      Kimberly Powell [24] has explored high performance computation and image processing in the radiology sector with the programmability of the GPU (Graphic Processing Unit) via CUDA (Compute Unified Device Architecture) will be presented, as well as the latest advances in GPU architecture (Graphic Processing Unit) by NVIDIA. GPUs have become a critical component in visual art imaging.

   3. ISSUES VISUALIZED

      The issue in focus of the proposed system is based on pruning problems from large datasets concerning visual art images retrieval system. Various issues visualized are related decision, optimization and computation. The Decision Problem are related to a given threshold to decide whether the dissimilarity is smaller than the threshold For a given threshold, the decision issues also poses obstruction to decide whether there exists a transformation such that the dissimilarity between the transformed pattern and the other pattern is smaller than the threshold. The Optimization Problem refers to find the transformation that minimizes the dissimilarity between the transformed pattern and the other pattern. The Computation Problem relates for computing the dissimilarity between the two patterns of visual art images. Sometimes the time complexities to solve such issues are rather high, so that it makes sense to devise approximation algorithms:

      • Shape Approximation and Simplification: These algorithms construct a shape of fewer elements (points, segments, triangles, etc.), that is still similar to the original. There are many heuristics for approximating polygonal curves and polyhedral surfaces. Optimal methods construct an approximation with the fewest elements given a maximal dissimilarity, or with the smallest dissimilarity given the maximal number of elements. It should be noted that checking the former dissimilarity is a decision problem, the latter is a computation problem.

      • Shape Retrieval: It is a search for all shapes in a typically large database of shapes that are similar to a query shape. Usually all shapes within a given distance from the query are determined (decision problem), or the first few shapes that have the smallest distance (computation problem)are. If the database is large, it may be infeasible to compute the similarity between the query and every database shape. An indexing structure can help to exclude large parts of the database from consideration at an early stage of the search, often using some form of triangle inequality property.

      • Shape Alignment and Registration:This transform one shape so that it best matches another shape (optimization problem), in whole or in part.

      • Approximate Optimization Problem: It is to find a transformation that gives dissimilarity between the two patterns that is within a constant multiplicative factor from the minimum dissimilarity.

      • Shape Recognition and Classification: It is to determine whether a given shape matches a model sufficiently close (decision problem), or which of k class representatives is most similar (k computation problems).

   4. PROPOSED SYSTEM

      The main aim is to design a framework for classification driven visual art image retrieval framework based on image filtering and similarity fusion by employing supervised learning technique. The

      proposed system highlights a learning-based retrieval framework that uses novel image filtering and similarity matching approaches. In this framework, several different image features are extracted to train multiclass support vector machines (SVMs) and perform similarity matching. Probabilities output from the SVM are considered as category-specific information for query and database images, and are used to first filter out irrelevant images before applying a linear combination of similarity matching functions. The features are finally unified by a dynamically weighted linear combination of similarity matching functions to overcome machine learning or user classification errors.

   5. FRAMEWORK DESIGN

      In the proposed method, we evaluate the effectiveness of the proposed retrieval approach, Exhaustive experiments were performed in a medical image collection. The collection comprises of 1000 visual art images from different datasets used for retrieval evaluation campaign in Yale University Art Gallery [25].

      Database Creation

      Perform Feature Extraction

      performing the similarity matching, the category of a query image is determined as probabilistic outputs or class confidence scores based on the classification subsystem. Next, this output is sent to the filtering subsystem to select candidate images for further similarity matching. In addition, based on the online category prediction, the predefined category-specific feature weights are utilized in a linear combination of similarity matching function as shown in the middle portion. After obtaining a ranked-based retrieval result (as shown in the left side of the Fig. 1), users next provide the feedback about relevant images and that information is utilized to update the final feature weights for the next retrieval iterations.

   6. EXPERIMENTAL RESULT ANALYSIS

      The proposed system is experimented on Intel core i3 processor with

      1. GHz and 4GB RAM. Matlab is considered for designing platform. Te performance of a classification and/or retrieval system depends on the underlying image representation, usually in the form of a feature vector. The Fig.2(a) shows the original visual art image where after performing feature extraction using CLD, the results can be seen in the Fig.2 (b) where the original image is divided into 8×8 non-overlapping blocks the Fig 2(c), shows CLD Coefficient. The Fig.2 (d) highlights the results of EHD. These outputs of feature vectors are then subjected to SVM learning algorithm, where the

         Color Layout Descriptor

         Edge Histogram Descriptor

         database is generated that retains all the empirical features extracted using CLD and EHD in arrays.

         This procedure of extracting features and training using SVM and PCA is repeated for all the groups and their respective images and

         Trained Dataset

         Trained Dataset

         SVM+PCA Training

         Retrieval Phase Query Image

         Perform Feature Extraction

         once done, it can be said that our dataset is completely trained and ready for retrieval operation. It should be noted that even in retrieval phase also, the query image (untrained image) is also subjected to feature extraction that is very similar to feature extraction done at the time of dataset creation or training phase.

         Color Layout Descriptor

         Perform Ranking

         Edge Histogram Descriptor

         Retrieved Similar Images

         1. (b)

      Perform RF

      Retreived Final Results

      Perform RF

      Retreived Final Results

      Fig. 1 Schematic diagram of the proposed system

      The schematic diagram of the proposed visual image retrieval framework is shown in Fig. 1. As it can be seen from the top portion of Fig. 1, various image features are extracted offline in a feature extraction subsystem and stored in a feature index for the database images. In addition, image features are combined and classified by the SVM and PCA (Principal Component Analysis) to generate a category index file where for each image the class confidence or probability scores are stored for later filtering purpose. For a query image, similar feature extraction is performed as database images as shown in the bottom portion of Fig. 1. However, instead of

      (c) (d)

      Fig. 2 Performing feature extraction using visual descriptors

      The next phase in this case will be to perform execution of the algorithms for performing similarity fusion and similarity matching. Once, similarity algorithm accomplishes, the next phase of the execution will extraction of all 0 1 similar images with highest rank. It should be known that this procedure is conducted considering the

      machine is taking decision about the search based on content based image retrieval. Therefore, the result is expected to have false positives. This step is conducted to understand the importance of relevance feedback in the area of machine learning in visual art image retrieval system as seen in Fig 3.

      Query Im Rank:1 Rank:2 Rank:3

      Rank 4 Rank:5 Rank:6 Rank:7

      Rank 8 Rank:9 Rank:10 Rank:11

      Fig. 3 Retreived visual art images without relevance feedback

      The Fig.4 shows that relevance feedback is very critical when it comes to machine learning and visual art image retrieval especially in context of Content based Image Retrieval System. Therefore, to aggregate the result, relevance feedback is performed by manually selecting all the images that visually maps with the queried subject image.

      Fig 4 Performing relevance feedback

      The prime assumption of this module is user is expected to rationally select the appropriate image that maps with queried image. The success rate of this module will depend upon precise selection of all relevant or similar images present in the form of queried image. It should also be noted that user may have the chances of making mistakes of choosing irrelevant images as relevant image. Consideration of this fact will exponentially alter the final retrieval results where one can expect presence of irrelevant images. In order to address this issue, the proposed system has considered two different types of visual descriptors (CLD/EHD) and training using unsupervised learning algorithm (SVM). This fact will mean that in case the user selects inappropriate image as similar images which performing relevance feedback, the imprecise image feature will never coincide with both features that are externally trained using

      SVM. Therefore, even after wrong selection, the program considers the image but attempts to show only the images with highest ranks, which will mean elimination of false positives as much as possible in final result of retrieval. This fact can be visualized in the Fig.5.

      Fig. 5 Final results of retrieval

   7. PERFORMANCE ANALYSIS

      For the purpose of comparative performance analysis, the proposed system is compared with certain prior research work that has claimed optimal output of pruning of CBIR system. We consider our database and experiment it using all the considered prior approach in CBIR. Following is the description of the approaches considered for performance evaluation.

      Narasimhan Approach [26]: It is a new partitional clustering algorithm based on the notion of contribution of a data point. The algorithm optimized on both the intra-cluster and inter-cluster similarity measures and required fewer passes with each pass having the same time complexity as that of the k-means algorithm. The clustering algorithm is applied to content-based image retrieval and the experiments revealed that the algorithm improves on recall at the cost of precision.

      Silakari Approach [27]: It is a framework of unsupervised clustering of images based on the color feature of image. Test has been performed on the feature database of color moments and BTC (Block Truncation Coding). K-means clustering algorithm is applied over the extracted dataset. Results are quite acceptable and showing that performance of BTC algorithm is better than color moments.

      Biswas Approach [28]: It is a system that is developed for retrieving images similar to a query image from a large set of distinct images. It follows an image segmentation based approach to extract the different features present in an image. These features are stored in vectors called feature vectors and compared to the feature vectors of query image and thus, the image database is sorted in decreasing order of similarity.

      It is to be noted that all the above mentioned techniques have similarity in pruning CBIR system, but accomplished with various techniques as mentioned in the Table 1:

      Table 1 Techniques considered

      Authors	Techniques Adopted
      Narasimhan [26]	Contribution Based Clustering
      Silakari [27]	-Block Truncation Coding -k-means clustering
      Biswas [28]	-k-means clustering -Discrete Wavelet Transform
      Proposed Method	-Visual Descriptors -Support Vector Machine -Principal Component Analysis -Relevance Feedback -Image Filtering

      The following performance measures were used to evaluate the performance of the algorithm,

      • Precision=( Total number of retrieved relevant images)/( Total number of retrieved images)

      • Recall=( Total number of retrieved relevant images)/( Total number of relevant images)

      Fig. 6 Comparative performance analysis

      The Fig. 6 shows the comparative performance analysis for three previous techniques with proposed technique. Narasimhan [26] has used an empirical technique that defines the contribution of a data point belonging to a cluster as the impact that it has on the quality of the cluster. To represent the visual content of an image, it has used a RGB color histogram. The color coordinates in the RGB color space are uniformly quantized into a number o bins. Therefore the precision is higher compared to other techniques (BTC and k-means). Silakari [27] has performed feature extraction using color moments and then BTC is applied along with k-means clustering. Biswas [28] has used discrete wavelet transformation for feature extraction followed by implementing k-means clustering algorithm. The k- means algorithm has been used to cluster the feature vectors into several classes with every class corresponding to one region in the segmented image. Although, this technique has improved the search results considerably and is also consistent with the human perception of an image, however, recall and precision rate is found to be very low compared to other works. Thus the feature extraction is better when visual descriptors are used as seen in work of Narasimhan [26]. Moreover, in the proposed learning based technique using SVM and PCA, the training is performed over features being extracted using two different visual descriptors (EHD and CLD). The recall and precision rates are increased compared to other works due to inclusion of relevance feedback.

   8. CONCLUSION

The proposed system introduces learning-based and groups-driven visual art image retrieval framework for various fine arts collections of different forms. In the present technique, the system directly associates categorization to retrieval. In this framework, the image group information is utilized directly to filter out irrelevant images and adjust the feature vectors in a linear combination of similarity matching. The system also uses the relevance feedback-based technique to update the feature weights based on positive user feedback. The results of retrieval system is satisfactory and obviously exhibits the advantage of searching images based on similarity index and filtering in terms of effectiveness and efficiency. Overall, this

retrieval framework is useful as a front end for large visual art databases where a search can be performed in diverse images for teaching, training and research purposes.

REFERENCES

1. Antani, S., Long, L.R., Thomas, G.R., Content-Based Image Retrieval for Large Visual art Image Archives, MEDINFO 2004

2. Thierry Pun and David McG. Squire. Statistical structuring of pictorial databases for content-based image retrieval systems. Pattern Recognition Letters, 17:12991310, 1996.

3. Stan Sclaroff, Leonid Taycher, and Marco La Cascia. ImageRover: a content-based browser for the world wide web. In IEEE Workshop on Content-Based Access of Image and Video Libraries, pages 29, San Juan, Puerto Rico, June 1997

4. Anil K. Jain and Aditya Vailaya. Image retrieval using color and shape. Pattern Recognition , 29(8):12331244, August 1996

5. Asha Vellaikal and C.-C. Jay Kuo. Hierarchical clustering techniques for image database organization and summarization. In C.-C. Jay Kuo, Shih-Fu Chang, and Sethuraman Panchanathan, editors, Multimedia Storage and Archiving Systems III (VV02) , volume 3527 of SPIE Proceedings , pages 6879, Boston, Massachusetts, USA, November 1998

6. David A. White and Ramesh Jain. Algorithms and strategies for similarity retrieval. Technical Report VCL-96-101, Visual Computing Laboratory, University of California, San Diego, 9500 Gilman Drive, Mail Code 0407, La Jolla, CA 92093-0407, July 1996.

7. David McG. Squire, Wolfgang Muller, and Henning Muller. Relevance feedback and term weighting schemes for content- based image retrieval. In Third International Conference On Visual Information Systems, Amsterdam, The Netherlands, June 24 1999. (to appear).

8. Tagare, H.D., Ja e, C.C., and Duncan, J. Medical image databases: A content-based retrieval approach. American Medical Informatics Association, 4:184198, 1997.

9. Remenyi, A. and Bandi, I. and Valcz, G. and Bodgdanov, P. and Kozlovszky, Miklós, Visual art image processing with GPGPU using CUDA. In: MIPRO 2011. 34th international convention. Proceedings. Vol. 1. MEET and GVS. Opatija, 2011.

10. Liguang, MA. Yanrong, C., Jianbang, H., Visual art Image Storage, Retrieval and Visualization Based-on Open Source Project, Congress on Image and Signal Processing, 2008

11. Constantinescu, L., Jinman, K., Dangan, F., Integration of Interactive Visual art Image Data Visualisation to Internet-Based Personal Health Records for Handheld Devices, e-Health Networking, Applications and Services, Healthcom ,11th International Conference, pp. 79-83, 2009

12. Puyan, M., Vetri, L.J, Microwave Visual art Imaging Using the Multiplicative Regularized GaussNewton Inversion, Antennas and Wireless Propagation Letters, IEEE, vol.8,pp.645-648,2009

13. Udupa.J.K, Raya.S.P, Barreit.W.A, A PC-based 3D imaging system for visual art data, Visualization in Visual art Computing, Proceedings of the First Conference, pp.294- 303,2009

14. Deniz, S., Herman, G.T., Kong, T.Y., A Technology for Retrieval of Volume Images from Visual art Databases, IEEE, 2004

15. Chang, S.F., Sikora, T., Puri, A., Overview of the MPEG-7 Standard, IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on MPEG-7, June 2001.

16. Lux, A., Chatzichristofis, S.A., LIRe: Lucene Image Retrieval – An Extensible Java CBIR Library, MM '08 Proceedings of the 16th ACM international conference on Multimedia, 2008

17. Calvo, R.A., Crespo, J., Marmol, M.M., Intriago, M., Castellanos, A.I., A Cloud Computing Service for Managing Visual art Image Collections, IEEE, 2012

18. Muller, H., Kalpathy, J., Eggel, I., Overview of the CLEF 2010 medical image retrieval track, Citeseer, 2010

19. Thomas M. Lehmann, Berthold Wein, Jörg Dahmen, Jörg Bredno, Frank Vogelsang, and Michael Kohnen, Content- Based Image Retrieval in Medical Applications: A Novel Multi- Step Approach, In Proceedings of SPIE: Storage and Retrieval for Media Databases 2000, Vol. 3972 (2000), pp. 312-320 Key: citeulike:2783702

20. Thomas M. Lehmann, Mark O. Gu¨lda, Thomas Deselaersb, Daniel Keysersb, Henning Schubertc, Klaus Spitzera, Hermann Neyb, Berthold B. Wein, Automatic categorization of medical images for content-based retrieval and data mining, Computerized Medical Imaging and Graphics 29 pp. 143 155,2005

21. Yong Rui, Thomas S. Huang, Michael Ortega, Sharad Mehrotra, Relevance Feedback A Power Tool for Interactive Content Based Image Retrieval,IEEE, Transactions on circuits and video technology

22. Si Yong Yeo, Implicit Active Contours for N-Dimensional Visual art Image Segmentation, 2012 IEEE International Conference on Systems, Man, and Cybernetics, pp.14- 17,COEX, Seoul, Korea,2012

23. Emrah Bostan, Ulugbek Kamilov, and Michael Unser, Reconstruction of visual art images and sparse stochastic modeling, pp.1-12, IEEE, 2012

24. Kimberly Powell, Visual art imaging ecosystem and the role of the GPU, pp.1291-1292, IEEE, 2009

25. http://guides.library.yale.edu/content.php?pid=47735&sid=3545 20

26. Narasimhan, H., Ramraj, P., Contribution-Based Clustering Algorithm for Content-Based Image Retrieval, 5th International Conference on Industrial and Information Systems, ICIIS 2010,

    Jul 29 – Aug 01, 2010

27. Silakari, S., Motwani, M., Maheshwari, M., Color Image Clustering using Block Truncation Algorithm, IJCSI International Journal of Computer Science Issues, Vol. 4, No. 2, 2009

28. Prasad, B. G., K. K. Biswas, and S. K. Gupta. "Region-based image retrieval using integrated color, shape, and location index." computer vision and image understanding 94.1 (2004): 193-233.

.