Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Exploiting global and local features for image retrieval

全局和局部特征的图像检索

  • Published:
Journal of Central South University Aims and scope Submit manuscript

Abstract

Two lines of image representation based on multiple features fusion demonstrate excellent performance in image retrieval. However, there are some problems in both of them: 1) the methods defining directly texture in color space put more emphasis on color than texture feature; 2) the methods extract several features respectively and combine them into a vector, in which bad features may lead to worse performance after combining directly good and bad features. To address the problems above, a novel hybrid framework for color image retrieval through combination of local and global features achieves higher retrieval precision. The bag-of-visual words (BoW) models and color intensity-based local difference patterns (CILDP) are exploited to capture local and global features of an image. The proposed fusion framework combines the ranking results of BoW and CILDP through graph-based density method. The performance of our proposed framework in terms of average precision on Corel-1K database is 86.26%, and it improves the average precision by approximately 6.68% and 12.53% over CILDP and BoW, respectively. Extensive experiments on different databases demonstrate the effectiveness of the proposed framework for image retrieval.

摘要

两种基于多特征融合的图像检索方法具有非常好的性能。 但是, 这两种融合方法存在以下问题: 1) 在颜色空间中直接定义纹理结构的方法会增大对颜色特征的描述; 2) 提取多种特征再重新融合为一个向量的方法, 这种方法将有效的特征和无效的特征直接结合后, 无效的特征会降低检索性能。 针对以上问题, 提出一种新的混合框架用于彩色图像检索, 该框架使用词袋模型(bag-of-visual words, BoW)和颜色强度局部差分模式(color intensity-based local difference patterns, CILDP)分别提取图像的不同特征信息。 同时, 提出的融合框架利用 graph density 的方法将 BoW 和 CILDP 的排序结果进行有效融合, 利用该框架能够提高图像检索的精度。 在Corel-1K 数据库上, 返回 10 幅图像时, 提出的框架的平均精度为86.26%, 分别比 CILDP 和 BoW 提高了大约 6.68%和 12.53%。 在不同数据库上的大量实验也验证了该框架在图像检索上的有效性。

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. LIU Ying, ZHANG Deng-sheng, LU Guo-jun, MA Wei-ying. A survey of content-based image retrieval with high-level semantics [J]. Pattern Recognition, 2007, 40(1): 262–282.

    Article  MATH  Google Scholar 

  2. PENATTI O A B, SILVA F B, VALLE E, GOUET-BRUNET V, TORRES R D S. Visual word spatial arrangement for image retrieval and classification [J]. Pattern Recognition, 2014, 47(2): 705–720.

    Article  Google Scholar 

  3. DATTA R, JOSHI D, LI Jia, WANG J Z. Image retrieval: Ideas, influences, and trends of the new age [J]. ACM Computing Surveys (CSUR), 2008, 40, 2: 5.

    Article  Google Scholar 

  4. SWAIN M J, BALLARD D H. Color indexing [J]. International Journal of Computer Vision, 1991, 7(1): 11–32.

    Article  Google Scholar 

  5. STRICKER M A, ORENGO M. Similarity of color images [C]//Proceedings of IS&T/SPIE's Symposium on Electronic Imaging: Science & Technology. Bellingham: SPIE, 1995: 381–392.

    Google Scholar 

  6. HUANG Jing, KUMAR S R, MITRA M, ZHU Wei-jing, ZABIH R. Image Indexing Using Color Correlograms [C]//Proceedings of 1997 IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 1997: 762–768.

    Google Scholar 

  7. PASS G, ZABIH R, MILLER J. Comparing images using color coherence vectors [C]//Proceedings of the Fourth ACM International Conference on Multimedia. New York: ACM, 1997: 65–73.

    Google Scholar 

  8. OJALA T, PIETIKAINEN M, MAENPAA T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971–987.

    Article  MATH  Google Scholar 

  9. HUANG Di, SHAN Cai-feng, ARDABILIAN M, WANG Yun-hong, CHEN Li-ming. Local binary patterns and its application to facial image analysis: A survey [J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2011, 41(6): 765–781.

    Article  Google Scholar 

  10. HEIKKILÄ M, PIETIKÄINEN M, SCHMID C. Description of interest regions with local binary patterns [J]. Pattern Recognition, 2009, 42(3): 425–436.

    Article  MATH  Google Scholar 

  11. GUO Yi-mo, ZHAO Guo-ying, PIETIKÄINEN M. Texture Classification using a Linear Configuration Model based Descriptor [C]//Proceedings of the British Machine Vision Conference (BMVC). Dundee, United Kingdom: Citeseer, 2011: 1–10.

    Google Scholar 

  12. TAN Xiao-yang, TRIGGS B. Enhanced local texture feature sets for face recognition under difficult lighting conditions [J]. IEEE Transactions on Image Processing, 2010, 19(6): 1635–1650.

    Article  MathSciNet  MATH  Google Scholar 

  13. GUO Zhen-hua, ZHANG Lei, ZHANG D. A completed modeling of local binary pattern operator for texture classification [J]. IEEE Transactions on Image Processing, 2010, 19(6): 1657–1663.

    Article  MathSciNet  MATH  Google Scholar 

  14. MURALA S, WU Q M. Local mesh patterns versus local binary patterns: Biomedical image indexing and retrieval [J]. IEEE Journal of Biomedical and Health Informatics, 2014, 18(3): 929–938.

    Article  Google Scholar 

  15. ZHANG Gang, MA Zong-min, DENG Li-guo, XU Chang-ming. Novel histogram descriptor for global feature extraction and description [J]. Journal of Central South University of Technology, 2010, 17: 580–586.

    Article  Google Scholar 

  16. LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91–110.

    Article  Google Scholar 

  17. BAY H, ESS A, TUYTELAARS T, VAN GOOL L. Speeded-up robust features (SURF) [J]. Computer Vision and Image Understanding, 2008, 110(3): 346–359.

    Article  Google Scholar 

  18. RUBLEE E, RABAUD V, KONOLIGE K, BRADSKI G. ORB: An efficient alternative to SIFT or SURF [C]//Proceedings of 2011 IEEE International Conference on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2011: 2564–2571.

    Chapter  Google Scholar 

  19. SIVIC J, ZISSERMAN A. Video Google: A text retrieval approach to object matching in videos [C]//Proceedings of Ninth IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE, 2003: 1470–1477.

    Chapter  Google Scholar 

  20. SHEN Guan-lin, WU Xiao-jun. Content based image retrieval by combining color, texture and Centrist [C]//2013 Constantinides International Workshop on Signal Processing (CIWSP 2013). Stevenage, GBR: IET, 2013: 1–4.

    Google Scholar 

  21. ELALAMI M E. A novel image retrieval model based on the most relevant features [J]. Knowledge-Based Systems, 2011, 24(1): 23–32.

    Article  Google Scholar 

  22. SUBRAHMANYAM M, WU Q M J, MAHESHWARI R P, BALASUBRAMANIAN R. Modified color motif co-occurrence matrix for image indexing and retrieval [J]. Computers & Electrical Engineering, 2013, 39(3): 762–774.

    Article  Google Scholar 

  23. LIU Guang-hai, YANG Jing-yu. Image retrieval based on the texton co-occurrence matrix [J]. Pattern Recognition, 2008, 41(12): 3521–3527.

    Article  MATH  Google Scholar 

  24. LIU Guang-hai, ZHANG Lei, HOU Ying-kun, LI Zuo-yong, YANG Jing-yu. Image retrieval based on multi-texton histogram [J]. Pattern Recognition, 2010, 43(7): 2380–2389.

    Article  MATH  Google Scholar 

  25. LIU Guang-hai, YANG Jing-yu. Content-based image retrieval using color difference histogram [J]. Pattern Recognition, 2013, 46(1): 188–198.

    Article  Google Scholar 

  26. ZHANG Shao-ting, YANG Ming, COUR T, YU Kai, METAXAS D N. Query specific rank fusion for image retrieval [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(4): 803–815.

    Article  Google Scholar 

  27. SINGHA M, HEMACHANDRAN K. Performance analysis of color spaces in image retrieval [J]. Assam University Journal of Science and Technology, 2011, 7(2): 94–104.

    Google Scholar 

  28. VADIVEL A, SURAL S, MAJUMDAR A K. An integrated color and intensity co-occurrence matrix [J]. Pattern Recognition Letters, 2007, 28(8): 974–983.

    Article  Google Scholar 

  29. LIU Li, ZHAO Ling-jun, LONG Yun-li, KUANG Gang-yao, FIEGUTH P. Extended local binary patterns for texture classification [J]. Image and Vision Computing, 2012, 30(2): 86–99.

    Article  Google Scholar 

  30. CROSS G R, JAIN A K. Markov random field texture models [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,, 1983, 1: 25–39.

    Article  Google Scholar 

  31. GUO Zhen-hua, ZHANG Lei, ZHANG D. Rotation invariant texture classification using LBP variance (LBPV) with global matching [J]. Pattern Recognition, 2010, 43(3): 706–719.

    Article  MATH  Google Scholar 

  32. ZHANG Xiao-fan, DOU Hang, JU Tao, ZHANG Shao-ting. Fusing heterogeneous features for the image-guided diagnosis of intraductal breast lesions [C]//Proceedings of 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). Piscataway, NJ: IEEE, 2015: 1288–1291.

    Chapter  Google Scholar 

  33. WANG J Z, LI Jia, WIEDERHOLD G. SIMPLIcity: Semantics-sensitive integrated matching for picture libraries [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(9): 947–963.

    Article  Google Scholar 

  34. LI Jia, WANG J Z. Automatic linguistic indexing of pictures by a statistical modeling approach [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(9): 1075–1088.

    Article  Google Scholar 

  35. ANTANI S, KASTURI R, JAIN R. A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video [J]. Pattern Recognition, 2002, 35(4): 945–965.

    Article  MATH  Google Scholar 

  36. LIU Guang-hai, LI Zuo-yong, ZHANG Lei, XU Yong. Image retrieval based on micro-structure descriptor [J]. Pattern Recognition, 2011, 44(9): 2123–2133.

    Article  Google Scholar 

  37. BESIRIS D, ZIGOURIS E. Dictionary-based color image retrieval using multiset theory [J]. Journal of Visual Communication and Image Representation, 2013, 24(7): 1155–1167.

    Article  Google Scholar 

  38. LANCE G N, WILLIAMS W T. Mixed-data classificatory programs I-agglomerative systems [J]. Australian Computer Journal, 1967, 1(1): 15–20.

    Google Scholar 

  39. ELALAMI M E. A new matching strategy for content based image retrieval system [J]. Applied Soft Computing, 2014, 14(1): 407–418.

    Article  Google Scholar 

  40. JALAB H. Image retrieval system based on color layout descriptor and Gabor filters [C]//Proceedings of 2011 IEEE Conference on Open Systems (ICOS). Piscataway, NJ: IEEE, 2011: 32–36.

    Chapter  Google Scholar 

  41. HIREMATH P S, PUJARI J. Content based image retrieval using color, texture and shape features [C]//Proceedings of 2007 International Conference on Advanced Computing and Communications (ADCOM 2007). Piscataway, NJ: IEEE, 2007: 780–784.

    Google Scholar 

  42. BANERJEE M, KUNDU M K, MAJI P. Content-based image retrieval using visually significant point features [J]. Fuzzy Sets and Systems, 2009, 160(23): 3323–3341.

    Article  MathSciNet  Google Scholar 

  43. GUO Jing-ming, PRASETYO H, SU Huai-sheng. Image indexing using the color and bit pattern feature fusion [J]. Journal of Visual Communication and Image Representation, 2013, 24(8): 1360–1379.

    Article  Google Scholar 

  44. IRTAZA A, JAFFAR M A, ALEISA E, CHOI T S. Embedding neural networks for semantic association in content based image retrieval [J]. Multimedia Tools and Applications, 2014, 72(2): 1911–1931.

    Article  Google Scholar 

  45. ZENG Shan, HUANG Rui, WANG Hai-bing, KANG Zhen. Image retrieval using spatiograms of colors quantized by Gaussian Mixture Models [J]. Neurocomputing, 2016, 171(1): 673–684.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, 116024, China

    Li Li  (李莉) & Lin Feng  (冯林)

  2. School of Innovation and Entrepreneurship, Dalian University of Technology, Dalian, 116024, China

    Lin Feng  (冯林), Jun Wu  (吴俊) & Mu-xin Sun  (孙木鑫)

  3. School of Control Science and Engineering, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, 116024, China

    Sheng-lan Liu  (刘胜蓝)

Authors
  1. Li Li  (李莉)
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Feng  (冯林)
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Wu  (吴俊)
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mu-xin Sun  (孙木鑫)
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sheng-lan Liu  (刘胜蓝)
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Feng  (冯林).

Additional information

Foundation item: Projects(61370200, 61672130, 61602082) supported by the National Natural Science Foundation of China; Project(1721203049-1) supported by the Science and Technology Research and Development Plan Project of Handan, Hebei Province, China

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Feng, L., Wu, J. et al. Exploiting global and local features for image retrieval. J. Cent. South Univ. 25, 259–276 (2018). https://doi.org/10.1007/s11771-018-3735-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11771-018-3735-6

Key words

关键词

Search

Navigation