Region based Image Indexing and Retrieval System

Region based Image Indexing and Retrieval System

Amit G. Gaikwad1, Priti Rege2, Chetan Pardeshi3

Electronics and Telecommunication College of Engineering

Pune, India

AbstractContent based image retrieval (CBIR) system is developed for accessing digital images from large image database. To improve the performance of CBIR system, Region based approach for image retrieval system is gaining a considerable attention in research area. Region based image retrieval (RBIR) system focuses on content from regions of images. Although various RBIR techniques have been developed, there are still many problems not satisfactorily solved. In this paper, a effective region based image indexing and retrieval (RBIR) framework is proposed. The proposed approach employs fast and effective statistical region merging (SRM) algorithm to segment images into meaningful regions and each region is then represented using feature vector which describes the color and texture features of region. These feature vectors are used to perform image indexing and retrieval process. Further to improve the retrieval speed and performance, images with similar regions are grouped together. Color moment, HSV color histogram and autocolor correlogram is used to extract color features whereas Gabor function is used to extract texture features from region. Experimental results and performance measurement shows the efficiency and reliability of proposed RBIR system.

KeywordsContent based image retrieval (CBIR), statistical region merging(SRM), autocolor correlogram, color moment.

1. INTRODUCTION

   In recent year, with the advancement in internet and availability of multimedia devices, the size of digital archives such as digital image is increasing rapidly. It has recently attracted research efforts in providing tools for efficient image access and retrieval process for various applications such as publishing, medicine, crime prevention, geographical information etc. For this purpose many image retrieval systems have been developed. Early image retrieval system uses textual annotation of images (TBIR) for searching and retrieval. Here text descriptor are used to annotate images manually which are then used to store images in database and in retrieval process. But manual image annotation is tedious and labor intensive job [2]. Also annotation is subjective of human perception. Different people have different way of annotating an image, which limits the use of TBIR system. To overcome the problem associated with TBIR, content based image retrieval system (CBIR) was propose. CBIR uses visual contents for image indexing and retrieval. In CBIR image visual contents are described by feature vector.

   Although various CBIR techniques have been implemented and good performance were demonstrated, there are still no universally accepted system available for image retrieval which have satisfactorily solved CBIR problems. One such problem associated with CBIR system is the gap between the low level visual feature and human semantic interpretation

   of an image. To narrow down this gap, some approaches have been proposed to bridge the semantic gap. Such as in relevance feedback method system learns user intensions to modify the retrieval process to provide semantically and perceptually meaningful result [14]. Some existing CBIR system extracts visual content descriptor from the entire image that are called global descriptor feature. But in global feature representation of an image information about object shape, texture and location is discarded [3]. In some CBIR system image is divided into blocks and color information of each block is used for matching and retrieval process [4]. These methods are fails to give accurate match.

   To overcome the deficiencies of global and grid based methods, Region based image retrieval (RBIR) technique is used. In RBIR system, Image is represented at the object- level by using segmentation method to break image into objects (i.e. homogeneous regions) [6]. It contributes to more meaningful image retrieval as object level representation is intended to be close to the human perception. All region based techniques are based on image segmentation. Most of the RBIR systems are developed using watershed segmentation [13], K-means clustering [12] is used to segment an image into regions. However due to segmentation complexity these methods fail to gives exact matches during retrieval process.

   This paper presents a novel region based image indexing and retrieval framework which uses fast and effective statistical region merging (SRM) technique for segmentation of image into homogenous regions. SRM uses computationally simple algorithm giving reliable and fast segmentation of image into meaningful regions. After segmentation system, is trained by supervised learning algorithm. For that, each segmented region is then represented by means of a set of features such as color and texture. The images with similar regions are grouped together and their associated images ID is recorded into the region index file along with region feature vector. This approach provides faster retrieval as only region features are compared instead of whole images of image database. During retrieval process, to answer a query, the query image is segmented by the employed segmentation algorithm and feature vector for each segmented region is obtained and compared with all region feature vector of region database. Images associated with matched regions are returned to answer a query. The outline of the propose framework is illustrated in Figure.1.

   The paper is organized as follows. In section II, the employed image segmentation algorithm is presented. Section III details the used feature extraction methods. Section IV deal with the index and retrieval structure. Experimental results along with performance measure are given in section V. Finally conclusion is given in section VI.

   Fig. 1. Outline of the propose framework

2. IMAGE SEGMENTATION

   The first step in adopted region based approach is to segment input image into homogenous regions to extract meaningful objects. For efficient retrieval process the segmentation method should be simple and it should effectively render homogeneous regions in short time. In our model, statistical region merging (SRM) algorithm [5] is employed to segment an image. SRM algorithm is based on region growing and merging techniques, which uses statistical test to merge regions. The method model segmentation as an inference problem, where image is treated as an observed instance for an unknown theoretical image whose statistical (true) regions are to be reconstructed. In SRM algorithm it is assumed that:

   • Statistical pixel belongs to any region posses the same expectation inside that region and given any color of channel.

   • And the expectation of adjacent statistical regions differs in at least one color channel.

   SRM algorithm strongly depends on two essential components: a merging predicate which decides whether merging of regions is possible or not and order followed to inspect pair of adjacent regions.

   1. Merging predicate:

      Following merging predicate is used for RGB image [5].

      In our approach we have used and =20 to get better segmentation result for used image database. is the maximum intensity for each color channel and kept at g=256.

   2. Merging order:

      For an image I, there are N < 2 pairs of adjacent pixels based on 4- connectivity. Let SI be the set of these pairs, p and p be pixel in image I, and R(p) stands for region to which a pixel p belongs. RM algorithm first sort pairs of SI in increasing order of a real- valued function f(p,p) and the this order is transverse only ones, by performing merging test P(R(p), R(p)) for any pair of pixel (p,p) and merge R(p) and R(p) if predicate return true.

      Sorting function f(.) is defined as follows, where first local gradient is calculated between pixels and then maximum per- channel variation is computed.

      (3) SRM algorithm:

      1. * For 4 connectivity of pixel C4 *

      2. Find gradient magnitude for each pixel in 4-connectivity

      3. * Sort gradient magnitude in increasing order

      4. For total number of regions in image I

      5. Do.

         If R(pi) != R(pi) ( if two regions are not equal) Then if Merging predicate is true for (R(pi),R(pi))

         Then Merge two regions ( R(pi),R(pi))

         End

3. FEATURE EXTRACTION

   After segmentation feature vector (descriptor) is calculated from each region. These feature vectors act as description of the segmented region (object), which is then used during indexing and retrieval process. To describe each region or object briefly and effectively, we have used HSV color histogram, color moment and color correlogram to obtain color information [8]. Whereas Gabor function and wavelet transform is used to extract texture feature from region.

   1. Color feature extraction:

      1. HSV color Histogram: A histogram acts as a coarse representation of an object and describes color distribution within an object. It is rotation, translation and scaling invariant characteristic of an object. Among all color spaces HSV color

         (1)

         space is more close to the human perception [7]. Hence first image is converted into HSV color space, and then histogram is obtained by quantizing the colors within the region and finding occurrence of each color inside region. An HSV color histogram for a region is defined as:

         And

         (2)

         H = { f(h), f(s), f(v) } (4)

         Where R and R represent the two regions being tested,

         denotes the observed average for color channel a in region R and R|p| is the set of regions with p pixels. is the maximum probability when become false and generally kept very small. Parameter decides statistical complexity of the ideal segmentation of image. For large value of algorithm gives more number of segmentation for an image.

         Where (5)

         Where i represents a color in c color channel, C[i] is the number of pixels in color I, N is the number of quantized color levels.

         probability of getting a pixel whose color is j at a distance of d from a pixel in the image or region. Let I an m × n image and is quantized into m color C1, C2, Cm . Then the color correlogram of an image I is given by

         (8)

         (a)

         (b)

         Where is the color of pixel at a distance d from the pixel of color inside region. Then auto color correlogram is calculated for identical color ( ) by using

         ACC (i,j,d) = (9)

   2. Texture

      Texture is an important property to describe object uniquely and effectively. In proposed RBIR technique Gabor filter [8] is used to obtain the texture feature as its result has been proved to be well matched to human perceptual vision system.

      1. Gabor filter: Gabor filters are used to extract texture feature from image region using multiple orientation and scale approach. Bank of filters are built using Gabor function which is defined in 2-D [8] as:

         (10)

         (c)

         Fig. 2. Some color segmentation results. Left: original image. Right: its corresponding segmentation. (a)House. (b) Tiger. (c) Rose

      2. Color moment: Color moment is used to represent color distribution of region. It is proved that low order moments are capable of representing color distribution effectively and efficiently. Thus in proposed method only the first moment (mean) and second moment (variance) are calculated for each channel ( R,G,B) and added to feature vector.

         The first order moment is defined by:

         Where j = and are scaling parameter of

         filter, W specifies radial frequency of sinusoid and [0, ] define orientation of filter. Segmented region (assume to be rectangle) is convolved with the Gabor function for each orientation and scale to obtain an array of magnitudes, which represents the energy content for different scale and orientation. For a given region R(x, y) the convolution is given by:

         The magnitudes of the Gabor filters responses are:

         Where is the pixel color value of the k-th color

         channel and XY is total number of pixels in region.

         Similarly nth moment, n= 2, 3, of k-th color channel is given by

      3. Auto color correlogram: Auto color correlogram

   technique is based on color correlogram which express how the spatial correlation of pairs of color changes with distance [9]. Generally for an image, color correlogram is the

   By using and feature vector is constructed to define texture of region.

4. INDEXING AND RETRIEVAL SYSTEM

   A. Region database generation and image indexing

   In proposed system, region database is generated off-line. Each entry in region database holds the feature vector describing the particular class of object or region along with the images ID containing that region. For each image in the

   proposed method for different query images are shown in Figure.3.

   1. Performance evaluation

      Generally to measure retrieval performance of any CBIR system, precision rate (p) and recall rate (r) are used, which are defined as:

      image database image, ID is given which identify the position of image in image database.

      Firstly, an image containing a particular class of object is loaded into the system. System then segment that image into different region (corresponding to object) and extract feature

      p =

      r =

      (15)

      (16)

      vector for each region. Only selected regions are labeled (keywords for class of object) and their feature vectors are stored in region database. Also image holding that region is also stored in image database and its position ID is added to newly loaded region feature vectors. Further, the features defining different classes of objects (region) are added to region database by repeating above process. This approach has been proved to be very effective as only meaningful regions features are loaded into region database which are then used in image retrieval process. If any image containing object which is already defined in region database is uploaded, then system automatically indexes that image in image database and its position ID is added to the matched regions feature vector. During retrieval, above indexing approach helps user to add his image collection into system which are further used in retrieval.

   2. Image retrieval methodology

   Image retrieval is on-line process where user inserts a query image for which he wants similar images to be retrieved by system. As soon as query image is loaded, system segments it into meaningful regions and generate feature vector set for each region. System then compares each region feature vector of query image with all region feature vector of region database. While comparing, similarity measures is find out using Manhattan distance [10] function. If

   and are two feature vectors then Manhattan distance between them is given by:

   .. (14)

   Two regions are said to be similar if the distance between them is less than or equal to threshold (T). Threshold value is predefined in the system. All the candidate images of matched regions are retrieved to the user end.

5. EXPERIMENTAL RESULTS & PERFORMANCE EVALUATION

   In order to implement, test and validate our approach, we have used CORAL database consisting 1000 color images of different classes. This database was used to test many CBIR systems [17]. Frm this database, five categories of images including dinosaur, elephant, flowers, tigers were selected as benchmark images. JPEG format is used to store all images in image database.

   The proposed system is implemented using MATLAB 2013 software. A flexible user interface is designed for performing indexing and retrieval process. Results of

   Wh er e is total number of retrieved images, is the total number of relevant images in database, is the number of relevant images retrieved. Precision rate and recall rate of proposed system for different class of query image is given in following table 1. The result shows that the proposed system gives better matched results for query image.

   TABLE I. PERFORMANCE EVALUATION

   Query Image				p	r
   Dinasaur	13	12	12	0.923	1
   Yellow rose	8	6	6	0.75	1
   Orange rose	14	12	12	0.85	1

   (a)

   (b)

   Fig. 3. Some image retrieval results. Left: query image. Right: its corresponding matched images. (a) Dinosaur. (b) Yellow rose.

6. CONCLUSION

   A region based image indexing and retrieval system is proposed in this paper. System uses statistical region merging technique for image segmentation. SRM technique helps efficient and fast segmentation of image into meaningful regions which helps to improve the RBIR system accuracy. Proposed indexing and retrieval approach increase the speed of RBIR system. The experimental results show system efficiency, high retrieval ratio and less complexity.

7. REFERENCES

1. Y. Liu, D.S. Zhang, G. Lu, and W.Y. Ma, A survey of content based image retrieval with high-level semantics Pattern Recognition, vol.40,no. 1, pp.262-282,2007.

2. D. Brahmi, D. Ziou, Improving CBIR system by integrating semantics feratures, Proc. Of Computer assisted information retrieval, Vaucluse, France, pp.21-305, Apr.2004.

3. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Hungs, B. Com, Query by image and video content: The QBIC system, IEEE computer, vol. 28, no. 9, pp. 23-22, Sep-1995.

4. T.W. Chiang, T. Tasi, and M.J. Hsiao, Content based image retrieval using color histogram and Grid based clustering, Proc. Of Int. Conf. of Innovation and Management, Dec. 2006.

5. Richard Nock and Frank Nielsen Statistical region merging IEEE transactions on pattern analysis and machine intelligence, vol. 26, no. 11, nov 2004

6. D. Zhang, Improving image retrieval performance by using both color and texture features, ion Proc. IEEE 3rd Int. conf. image and graphics (ICIG04) Hong Kong, China, 2004, pp.172-175.

7. J. R. Smith, Integrated spatial and feature image system: Retrieval, analysis and compression, PhD dissertation, Columbia Univ., New York, USA, 1997.

8. Ryszard S. Choras Image feature extraction techniques and their applications for CBIR and biometrics system, Int. Journal of biology and biomedical engineering, jun. 2007.

9. Anucha Tungsathan Sarayut Intarasema, Spatral color indexing using acc algorithm 7th int. Conf. on ICT and Knowledge Engineering, 2009.

10. A Vadivel, A K Majumdar Performance comparison of distance metrics in content-based Image retrieval applications IIT, Kharagpur,

    India,2007

11. C. Chiang, Y. Hung, G. Lee, Region based image retrieval using color size of watershed regions, Journal of visual commu. and image representation, vol. 20, pp. 167-177,2009.

12. A. Jain, Data clusteriong: 50 years beyond k-means Pattern recognition letters, vol. 31, no.8, pp. 651-666, June 2010.

13. Niket Amoda, Ramesh K Kulkarni, Image segmentation and detection using watershed transform and region based image retrieval, IJETTCS, Vol. 2, Issue 2, March April 2013.

14. Y. Rui, T.S. Huang, M. Ortega, and S. Mehrotra, Relevance feedback: a power tool for interactive CBIR, IEEE Trans. on ceruits and system for video technology, vol. 8, no. 5, pp. 644-655, sep-1998.

15. A. Natsev, R. Rastogi, and K. Shim, WALRUS: A similarity retrieval algorithm for image databases, IEEE Trans. Knowl. and Data Eng., vol. 16, no. 3, pp. 301-316, Mar. 2004.

16. M.-J. Hsiao, Y.-P. Huang, T. Tsai, T.-W. Chiang, An efficient and flexible matching strategy for Content-based image retrieval, Life Science Journal, vol. 7, no. 1, pp.99-106, Mar. 2010.

17. http://wang.ist.psu.edu/jwang/test1.tar, Jan. Last accessed-2013.