research-article

Multi-level contour combination features for shape recognition

Authors:

Chengzhuan Yang,

Hui WeiAuthors Info & Claims

Volume 229, Issue C

https://doi.org/10.1016/j.cviu.2023.103650

Published: 01 March 2023 Publication History

Abstract

We present a novel multi-level contour combination feature for shape recognition. This combination feature effectively solves large intra-class changes and nonlinear deformations of object shapes, thereby enhancing the performance of shape recognition. First, we divide the shape contour into two levels: the sampling points and the contour fragments, where sampling points are used to describe the detailed information of a shape and contour fragments are used to represent the global feature of a shape. Second, we employ the Fisher vector (FV) approach to encode the local sampling point feature and contour fragment feature as high-level characteristics. Finally, we combine the high-level characteristics after FV encoding and perform shape recognition through a linear support vector machine (SVM) model. The proposed method has been assessed on three benchmark shape datasets, including the Animal, MPEG-7,and ETH-80 datasets. Our method achieves 92.70%, 99.26% and 98.32% classification accuracy on the Animal, MPEG-7, and ETH-80 datasets, respectively. In addition, our method can also be applied to the classification of objects in real-word scenes. We combine the Weizmann Horse and the ETHZ Cow real-world scene datasets, and our method achieves 99.25% classification accuracy on the combined dataset. The recognition results of our approach are better than prior state-of-the-art shape recognition methods, which demonstrate the effectiveness and superiority of our approach.

Highlights

•

We propose a novel multi-level contour combination feature method.

•

Our method solves the intra-class variation and complex deformation of a shape.

•

We are the first to study the shape from the multi-level perspective of contours.

•

The performance of our method exceeds the prior state-of-the-art approaches.

References

[1]

Bai X., Liu W., Tu Z., Integrating contour and skeleton for shape classification, in: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, IEEE, 2009, pp. 360–367.

[2]

Belongie S., Malik J., Puzicha J., Shape matching and object recognition using shape contexts, IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002) 509–522.

Digital Library

[3]

Bicego M., Lovato P., A bioinformatics approach to 2D shape classification, Comput. Vis. Image Underst. 145 (2016) 59–69.

[4]

Borenstein, E., Sharon, E., Ullman, S., 2004. Combining top-down and bottom-up segmentation. In: Conference on Computer Vision and Pattern Recognition Workshop.

[5]

Chen Y.W., Chen Y.Q., Invariant description and retrieval of planar shapes using radon composite features, IEEE Trans. Signal Process. 56 (2008) 4762–4771.

[6]

Crammer K., Singer Y., On the algorithmic implementation of multiclass kernel-based vector machines, J. Mach. Learn. Res. 2 (2001) 265–292.

Digital Library

[7]

Daliri M.R., Torre V., Robust symbolic representation for shape recognition and retrieval, Pattern Recognit. 41 (2008) 1782–1798.

[8]

Daliri M.R., Torre V., Shape recognition based on kernel-edit distance, Comput. Vis. Image Underst. 114 (2010) 1097–1103.

[9]

Eslami S.M.A., Heess N., Williams C.K.I., Winn J., The shape boltzmann machine: A strong model of object shape, Int. J. Comput. Vis. 107 (2014) 155–176.

[10]

Fan R.E., Chang K.W., Hsieh C.J., Wang X.R., Lin C.J., Liblinear: A library for large linear classification, J. Mach. Learn. Res. 9 (2008) 1871–1874.

Digital Library

[11]

Frankle J., Carbin M., The lottery ticket hypothesis: Finding sparse, trainable neural networks, 2019.

[12]

Govindaraj P., Sudhakar M., Hexagonal grid based triangulated feature descriptor for shape retrieval, Pattern Recognit. Lett. 116 (2018) 157–163.

[13]

Guo Y., Liu Y., Oerlemans A., Lao S., Wu S., Lew M.S., Deep learning for visual understanding: A review, Neurocomputing 187 (2016) 27–48.

Digital Library

[14]

He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778.

[15]

Hu R.X., Jia W., Ling H., Zhao Y., Gui J., Angular pattern and binary angular pattern for shape retrieval, IEEE Trans. Image Process. 23 (2014) 1118–1127.

[16]

Hu R.X., Jia W., Zhao Y., Gui J., Perceptually motivated morphological strategies for shape retrieval, Pattern Recognit. 45 (2012) 3222–3230.

[17]

Jayasumana, S., Salzmann, M., Li, H., Harandi, M., 2013. A framework for shape analysis via hilbert space embedding. In: IEEE International Conference on Computer Vision. pp. 1249–1256.

[18]

Ke, Q., Li, Y., 2014. Is rotation a nuisance in shape recognition?. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4146–4153.

[19]

Khotanzad A., Hong Y.H., Invariant image recognition by zernike moments, IEEE Trans. Pattern Anal. Mach. Intell. 12 (1990) 489–497.

Digital Library

[20]

Krizhevsky, A., Sutskever, I., Hinton, G.E., 2012. Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems. pp. 1097–1105.

[21]

Kurnianggoro L., Wahyono, Jo K.H., A survey of 2D shape representation: Methods, evaluations, and future research directions, Neurocomputing 300 (2018).

[22]

Latecki L.J., Lakämper R., Convexity rule for shape decomposition based on discrete contour evolution, Comput. Vis. Image Underst. 73 (1999) 441–454.

[23]

Latecki L.J., Lakamper R., Eckhardt T., Shape descriptors for non-rigid shapes with a single closed contour, in: Proceedings IEEE Conference on Computer Vision and Pattern Recognition, 2000, IEEE, 2000, pp. 424–429.

[24]

Leibe B., Schiele B., Analyzing appearance and contour based methods for object categorization, IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2003, pp. II–409–15.

[25]

Li, C., Stevens, A., Chen, C., Pu, Y., Gan, Z., Carin, L., 2016. Learning weight uncertainty with stochastic gradient mcmc for shape classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 5666–5675.

[26]

Li C., You X., Hamza A.B., Zeng W., Zhou L., Distinctive parts for shape classification, in: 2011 International Conference on Wavelet Analysis and Pattern Recognition, ICWAPR, IEEE, 2011, pp. 97–102.

[27]

Lim K.L., Galoogahi H.K., Shape classification using local and global features, in: 2010 Fourth Pacific-Rim Symposium on Image and Video Technology, PSIVT, IEEE, 2010, pp. 115–120.

[28]

Lin M., Zhang Y., Li Y., Chen B., Chao F., Wang M., Li S., Tian Y., Ji R., 1Xn pattern for pruning convolutional neural networks, IEEE Trans. Pattern Anal. Mach. Intell. (2022).

[29]

Ling H., Shape classification using inner-distance, IEEE Trans. Pattern Anal. Mach. Intell. 29 (2007) 286–299.

[30]

Macrini D., Dickinson S., Fleet D., Siddiqi K., Object categorization using bone graphs, Comput. Vis. Image Underst. 115 (2011) 1187–1206.

[31]

Mingqiang Y., Kidiyo K., Joseph R., A survey of shape feature extraction techniques, Pattern Recognit. 15 (2008) 43–90.

[32]

Mirehi N., Tahmasbi M., Targhi A.T., New graph-based features for shape recognition, Soft Comput. 25 (2021) 7577–7592.

[33]

Patel, V., Mujumdar, N., Balasubramanian, P., Marvaniya, S., Mittal, A., 2019. Data augmentation using part analysis for shape classification. In: IEEE Winter Conference on Applications of Computer Vision. WACV.

[34]

Perronnin F., Sánchez J., Mensink T., Improving the fisher kernel for large-scale image classification, in: European Conference on Computer Vision, Springer, 2010, pp. 143–156.

[35]

Porikli F., Shan S., Snoek C., Sukthankar R., Wang X., Deep learning for visual understanding: Part 2 [from the guest editors], IEEE Signal Process. Mag. 35 (2018) 17–19.

[36]

Ramesh B., Xiang C., Lee T.H., Shape classification using invariant features and contextual information in the bag-of-words model, Pattern Recognit. 48 (2015) 894–906.

[37]

Sánchez J., Perronnin F., Mensink T., Verbeek J., Image classification with the fisher vector: Theory and practice, Int. J. Comput. Vis. 105 (2013) 222–245.

[38]

Sebastian T.B., Klein P.N., Kimia B.B., Recognition of shapes by editing shock graphs, IEEE Trans. Pattern Anal. Mach. Intell. 26 (2004) 550–571.

[39]

Shen W., Du C., Jiang Y., Zeng D., Zhang Z., Bag of shape features with a learned pooling function for shape recognition, Pattern Recognit. Lett. 106 (2018) 33–40.

[40]

Shi B., Zhang D., Dai Q., Zhu Z., Mu Y., Wang J., Informative dropout for robust representation learning: A shape-bias perspective, in: International Conference on Machine Learning, PMLR, 2020, pp. 8828–8839.

[41]

Shin H.C., Roth H.R., Gao M., Lu L., Xu Z., Nogues I., Yao J., Mollura D., Summers R.M., Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning, IEEE Trans. Med. Imaging 35 (2016) 1285–1298.

[42]

Simonyan, K., Zisserman, A., 2015. Very deep convolutional networks for large-scale image recognition. In: ICLR 2015 : International Conference on Learning Representations. pp. 1097–1105.

[43]

Sokic E., Konjicija S., Novel fourier descriptor based on complex coordinates shape signature, in: 2014 12th International Workshop on Content-Based Multimedia Indexing, CBMI, IEEE, 2014, pp. 1–4.

[44]

Sun K.B., Super B.J., Classification of contour shapes using class segment sets, in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, IEEE, 2005, pp. 727–733.

[45]

Tan T., Qian Y., Hu H., Zhou Y., Ding W., Yu K., Adaptive very deep convolutional residual network for noise robust speech recognition, IEEE/ACM Trans. Audio Speech Lang. Process. 26 (2018) 1393–1405.

[46]

Wang J., Bai X., You X., Liu W., Latecki L.J., Shape matching and classification using height functions, Pattern Recognit. Lett. 33 (2012) 134–143.

Digital Library

[47]

Wang X., Feng B., Bai X., Liu W., Latecki L.J., Bag of contour fragments for robust shape classification, Pattern Recognit. 47 (2014) 2116–2125.

Digital Library

[48]

Wang B., Gao Y., Hierarchical string cuts: A translation, rotation, scale and mirror invariant descriptor for fast shape retrieval, IEEE Trans. Image Process. 23 (2014) 4101–4111.

[49]

Wang B., Gao Y., Structure integral transform versus radon transform: A 2D mathematical tool for invariant shape recognition, IEEE Trans. Image Process. 25 (2016) 5635–5648.

[50]

Wang B., Gao Y., A novel line integral transform for 2D affine invariant shape retrieval, in: European Conference on Computer Vision, Springer, 2020, pp. 596–611.

[51]

Wang B., Shen W., Liu W., You X., Bai X., Shape classification using tree-unions, in: 2010 20th International Conference on Pattern Recognition, ICPR, IEEE, 2010, pp. 983–986.

[52]

Xu C., Liu J., Tang X., 2D shape matching by contour flexibility, IEEE Trans. Pattern Anal. Mach. Intell. 31 (2009) 180–186.

Digital Library

[53]

Yang C., Plant leaf recognition by integrating shape and texture features, Pattern Recognit. 112 (2021).

[54]

Yang C., Fang L., Wei H., Learning contour-based mid-level representation for shape classification, IEEE Access 8 (2020) 157587–157601.

[55]

Yang C., Fang L., Yu Q., Wei H., A learning robust and discriminative shape descriptor for plant species identification, IEEE/ACM Trans. Comput. Biol. Bioinform. (2022).

[56]

Yang, C., Wei, H., Yu, Q., 2016. Multiscale triangular centroid distance for shape-based plant leaf recognition. In: European Conference on Artificial Intelligence. pp. 269–276.

[57]

Yang C., Yu Q., Multiscale fourier descriptor based on triangular features for shape retrieval, Signal Process., Image Commun. 71 (2019) 110–119.

[58]

Yang C., Yu Q., Invariant multiscale triangle feature for shape recognition, Appl. Math. Comput. 403 (2021).

Digital Library

[59]

Young T., Hazarika D., Poria S., Cambria E., Recent trends in deep learning based natural language processing, IEEE Comput. Intell. Mag. 13 (2018) 55–75.

[60]

Yu, X., Xiong, S., Gao, Y., Yuan, X., 2019. Contour covariance: A fast descriptor for classification. In: IEEE International Conference on Image Processing. ICIP.

[61]

Zeng Y., Feng M., Lu H., Yang G., Borji A., An unsupervised game-theoretic approach to saliency detection, IEEE Trans. Image Process. 27 (2018) 4545–4554.

[62]

Zhou K., Yang Y., Qiao Y., Xiang T., Domain generalization with mixstyle, 2021.

[63]

Zhuang F., Qi Z., Duan K., Xi D., Zhu Y., Zhu H., Xiong H., He Q., A comprehensive survey on transfer learning, Proc. IEEE 109 (2020) 43–76.

Cited By

Ma GWang XLiu XLi ZWan Z(2023)Computing 2D Skeleton via Generalized Electric PotentialPattern Recognition and Computer Vision10.1007/978-981-99-8549-4_29(346-357)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-8549-4_29

Recommendations

Shape recognition by bag of skeleton-associated contour parts

We associate a contour point with an object thickness value.We propose a new descriptor to combine both contour and skeleton information.Our method achieves the state-of-the-arts results on several shape dataset. Contour and skeleton are two ...
Invariant multiscale triangle feature for shape recognition
Abstract
Shape is an important visual characteristic in representing an object, and it is also an important part of human visual information. Shape recognition is an important research direction in pattern recognition and image understanding. ...
Shape recognition using spectral features

The classification of planar shapes using spectral features is presented in this paper. The contour of a planar shape is represented by the magnitude and phase of radial vectors drawn from a centroid, and they are modeled by an autoregressive process. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computer Vision and Image Understanding

Computer Vision and Image Understanding Volume 229, Issue C

Mar 2023

175 pages

ISSN:1077-3142

Issue’s Table of Contents

Elsevier Inc.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 March 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ma GWang XLiu XLi ZWan Z(2023)Computing 2D Skeleton via Generalized Electric PotentialPattern Recognition and Computer Vision10.1007/978-981-99-8549-4_29(346-357)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-8549-4_29

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents