Abstract
In the problems of image retrieval and annotation, complete textual tag lists of images play critical roles. However, in real-world applications, the image tags are usually incomplete, thus it is important to learn the complete tags for images. In this paper, we study the problem of image tag complete and proposed a novel method for this problem based on a popular image representation method, convolutional neural network (CNN). The method estimates the complete tags from the convolutional filtering outputs of images based on a linear predictor. The CNN parameters, linear predictor, and the complete tags are learned jointly by our method. We build a minimization problem to encourage the consistency between the complete tags and the available incomplete tags, reduce the estimation error, and reduce the model complexity. An iterative algorithm is developed to solve the minimization problem. Experiments over benchmark image data sets show its effectiveness.
J.-Y. Wang—The study was supported by the open research program of Jiangsu Key Laboratory of Big Data Analysis Technology, Nanjing University of Information Science and Technology, China (Grant No. KBDat1602).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cai, W.: Class D power amplifier for medical application. Inf. Eng. Int. J. (IEIJ) 4(2), 9–15 (2016)
Cai, W.: Low power SI based power amplifier for healthcare application. Int. J. Pharm. Pharm. Sci. 8(9), 307–309 (2016)
Cai, W., Huang, L., Wen, W.: 2.4 GHZ class AB power amplifier for healthcare application. Int. J. Biomed. Eng. Sci. (IJBES) (2016). arXiv preprint arXiv:1605.02455
Cai, W., Zhou, X., Cui, X.: Optimization of a GPU implementation of multi-dimensional RF pulse design algorithm. In: 2011 5th International Conference on Bioinformatics and Biomedical Engineering, (iCBBE), pp. 1–4. IEEE (2011)
Cheng, L., Kotoulas, S., Ward, T.E., Theodoropoulos, G.: Robust and efficient large-large table outer joins on distributed infrastructures. In: Silva, F., Dutra, I., Santos Costa, V. (eds.) Euro-Par 2014. LNCS, vol. 8632, pp. 258–269. Springer, Cham (2014). doi:10.1007/978-3-319-09873-9_22
Cheng, L., Kotoulas, S., Ward, T.E., Theodoropoulos, G.: Robust and skew-resistant parallel joins in shared-nothing systems. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1399–1408. ACM (2014)
Feng, Z., Feng, S., Jin, R., Jain, A.K.: Image tag completion by noisy matrix recovery. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 424–438. Springer, Cham (2014). doi:10.1007/978-3-319-10584-0_28
Fu, J., Wu, Y., Mei, T., Wang, J., Lu, H., Rui, Y.: Relaxing from vocabulary: robust weakly-supervised deep learning for vocabulary-free image tagging. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 11–18-December-2015, pp. 1985–1993. doi:10.1109/ICCV.2015.230 (2016)
Geng, Y., Liang, R.Z., Li, W., Wang, J., Liang, G., Xu, C., Wang, J.Y.: Learning convolutional neural network to maximize pos@top performance measure. In: ESANN (2016)
Hobbs, K.H., Zhang, P., Shi, B., Smith, C.D., Liu, J.: Quad-mesh based radial distance biomarkers for alzheimer’s disease. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 19–23. IEEE (2016)
King, D.R., Li, W., Squiers, J.J., Mohan, R., Sellke, E., Mo, W., Zhang, X., Fan, W., DiMaio, J.M., Thatcher, J.E.: Surgical wound debridement sequentially characterized in a porcine burn model with multispectral imaging. Burns 41(7), 1478–1487 (2015)
Li, Q., Zhou, X., Gu, A., Li, Z., Liang, R.Z.: Nuclear norm regularized convolutional max pos@top machine. Neural Comput. Appl. 1–10 (2016)
Li, W., Mo, W., Zhang, X., Lu, Y., Squiers, J.J., Sellke, E.W., Fan, W., DiMaio, J.M., Thatcher, J.E.: Burn injury diagnostic imaging device’s accuracy improved by outlier detection and removal. In: SPIE Defense+ Security, p. 947206. International Society for Optics and Photonics (2015)
Li, W., Mo, W., Zhang, X., Squiers, J.J., Lu, Y., Sellke, E.W., Fan, W., DiMaio, J.M., Thatcher, J.E.: Outlier detection and removal improves accuracy of machine learning approach to multispectral burn diagnostic imaging. J. Biomed. Optics 20(12), 121305 (2015)
Li, X., Zhang, Y.J., Shen, B., Liu, B.D.: Low-rank image tag completion with dual reconstruction structure preserved. Neurocomputing 173, 425–433 (2016)
Liang, R.Z., Shi, L., Wang, H., Meng, J., Wang, J.J.Y., Sun, Q., Gu, Y.: Optimizing top precision performance measure of content-based image retrieval by learning similarity function. In: 2016 23st International Conference on Pattern Recognition (ICPR). IEEE (2016)
Liang, R.Z., Xie, W., Li, W., Wang, H., Wang, J.J.Y., Taylor, L.: A novel transfer learning method based on common space mapping and weighted domain matching. In: 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI), pp. 299–303. IEEE (2016)
Lin, Z., Ding, G., Hu, M., Lin, Y., Sam Ge, S.: Image tag completion via dual-view linear sparse reconstructions. Comput. Vis. Image Underst. 124, 42–60 (2014)
Lin, Z., Ding, G., Hu, M., Wang, J., Ye, X.: Image tag completion via image-specific and tag-specific linear sparse reconstructions. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1618–1625 (2013)
Lopes, A., de Aguiar, E., De Souza, A., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017)
Ma, J., Wu, F., Zhu, J., Xu, D., Kong, D.: A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics 73, 221–230 (2017)
Mao, H., Liu, H., Shi, P.: Neighbor-constrained active contour without edges. In: Mathematical Methods in Biomedical Image Analysis, pp. 1–7 (2008)
Mao, H., Liu, H., Shi, P.: A convex neighbor-constrained active contour model for image segmentation, pp. 793–796 (2010)
Mo, W., Mohan, R., Li, W., Zhang, X., Sellke, E.W., Fan, W., DiMaio, J.M., Thatcher, J.E.: The importance of illumination in a non-contact photoplethysmography imaging system for burn wound assessment. In: SPIE BiOS, p. 93030M. International Society for Optics and Photonics (2015)
Shen, W., Wang, J.: Transaction costs-aware portfolio optimization via fast löwner-john ellipsoid approximation. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 1854–1860. AAAI Press (2015)
Shen, W., Wang, J.: Portfolio blending via Thompson sampling. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 1983–1989. AAAI Press (2016)
Shen, W., Wang, J.: Portfolio selection via subset resampling. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press (2017)
Shen, W., Wang, J., Jiang, Y.G., Zha, H.: Portfolio choices with orthogonal bandit learning. In: Proceedings of the Twenty-Fourth International Conference on Artificial Intelligence, pp. 974–980. AAAI Press (2015)
Shen, W., Wang, J., Ma, S.: Doubly regularized portfolio with risk minimization. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 1286–1292. AAAI Press (2014)
Shi, B., Chen, Y., Zhang, P., Smith, C.D., Liu, J., Initiative, A.D.N., et al.: Nonlinear feature transformation and deep fusion for alzheimer’s disease staging analysis. Pattern Recogn. 63, 487–498 (2017)
Tan, M., Hu, Z., Wang, B., Zhao, J., Wang, Y.: Robust object recognition via weakly supervised metric and template learning. Neurocomputing 181, 96–107 (2016)
Thatcher, J.E., Li, W., Rodriguez-Vaqueiro, Y., Squiers, J.J., Mo, W., Lu, Y., Plant, K.D., Sellke, E., King, D.R., Fan, W., et al.: Multispectral and photoplethysmography optical imaging techniques identify important tissue characteristics in an animal model of tangential burn excision. J. Burn Care Res. 37(1), 38–52 (2016)
Wu, L., Jin, R., Jain, A.: Tag completion for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 716–727 (2013)
Xia, Z., Feng, X., Peng, J., Wu, J., Fan, J.: A regularized optimization framework for tag completion and image retrieval. Neurocomputing 147(1), 500–508 (2015)
Yang, W., Chen, Y., Liu, Y., Zhong, L., Qin, G., Lu, Z., Feng, Q., Chen, W.: Cascade of multi-scale convolutional neural networks for bone suppression of chest radiographs in gradient domain. Med. Image Anal. 35, 421–433 (2017)
Yang, X., Yang, F.: Completing tags by local learning: a novel image tag completion method based on neighborhood tag vector predictor. Neural Comput. Appl. 27(8), 2407–2416 (2016)
Zhang, P., Kong, X.: Detecting image tampering using feature fusion. In: International Conference on Availability, Reliability and Security, ARES 2009, pp. 335–340. IEEE (2009)
Zhang, P., Shi, B., Smith, C.D., Liu, J.: Nonlinear metric learning for semi-supervised learning via coherent point drifting. In: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 314–319. IEEE (2016)
Zhao, J.Y., Tang, M., Tong, R.F.: Connectivity-based segmentation for GPU-accelerated mesh decompression. J. Comput. Sci. Technol. 27(6), 1110–1118 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Geng, Y. et al. (2017). A Novel Image Tag Completion Method Based on Convolutional Neural Transformation. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10614. Springer, Cham. https://doi.org/10.1007/978-3-319-68612-7_61
Download citation
DOI: https://doi.org/10.1007/978-3-319-68612-7_61
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68611-0
Online ISBN: 978-3-319-68612-7
eBook Packages: Computer ScienceComputer Science (R0)