Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Integrating the edge intelligence technology into image composition: A case study

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

Image composition has important research significance and application value in image aesthetics and daily life. However, it faces problems, such as complex network structures, numerous parameters, round-trip delays, and difficult deployment. Consequently, we designed an image composition guidance system (ICGS) on mobile devices to help users capture photos with more aesthetic value through image composition. The first step was to build a lightweight object-detection model. Compressing the model by optimizing the structure and parameters reduces the size of the network model, and solves the difficult deployment problem. Second, we customized the mobile camera application development and deployed a deep learning model for this application. By reading the lighting model, we realized automatic composition guidance (ACG). Simultaneously, for scenes without objects to be detected, we designed a manual composition guidance (MCG) based on the target tracking algorithm to lock any area for composition. Furthermore, the experimental results show that the aesthetic scores of the guided photos improve, which is more in line with public aesthetics. In addition, the application’s real-time performance, stability, and response time have also reached high standards.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

Not applicable.

References

  1. Yang H, Shi P, He S, Pan D, Ying Z, Lei L (2019) A comprehensive survey on image aesthetic quality assessment, in 2019 IEEE/ACIS 18th International Conference on Computer and Information Science (ICIS)pp.294-299

  2. Zhou Z, Chen X, Li E, Zeng L, Luo K, Zhang J (2019) Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proc IEEE 107(8):1738–1762

    Article  Google Scholar 

  3. Cao K, Liu Y, Meng G, Sun Q (2020) An overview on edge computing research. IEEE Access 8:85714–85728

    Article  Google Scholar 

  4. Chen J, Bai G, Liang S, Li Z (2016) Automatic image cropping: A computational complexity study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 507-515

  5. Zhang W, Zhuang P, Sun HH, Li G, Kwong S, Li C (2022) Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement. IEEE Trans Image Process 31:3997–4010

    Article  Google Scholar 

  6. Sugimoto Y, Imaizumi S, Lossless A (2021) Image processing Mmethod with contrast and saturation enhancement. In 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP) pp.1-6

  7. Li J, Datta R, Joshi D, Wang JZ (2006) Studying aesthetics in photographic images using a computational approach. Lect Notes Comput Sci 3953:288–301

    Article  Google Scholar 

  8. Wei Z, Zhang J, Shen X, Lin Z, Mech R, Hoai M, Samaras D (2018) Good view hunting: Learning photo composition from dense view pairs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 5437-5446

  9. Zhang X, Li Z, Jiang J (2020) Emotion attention-aware collaborative deep reinforcement learning for image cropping. IEEE Trans Multimedia 23:2545–2560

    Article  Google Scholar 

  10. Wang W, Shen J, Ling H (2018) A deep network solution for attention and aesthetics aware photo cropping. IEEE Trans Pattern Anal Mach Intell 41(7):1531–1544

    Article  Google Scholar 

  11. Li D, Wu H, Zhang J, Huang K (2019) Fast a3rl: Aesthetics-aware adversarial reinforcement learning for image cropping. IEEE Trans Image Process 28(10):5105–5120

    Article  MathSciNet  MATH  Google Scholar 

  12. Matsubara Y, Callegaro D, Baidya S, Levorato M, Singh S (2020) Head network distillation: Splitting distilled deep neural networks for resource-constrained edge computing systems. IEEE Access 8:212177–212193

    Article  Google Scholar 

  13. Gou J, Sun L, Yu B, Wan S, Tao D (2022) Hierarchical Multi-Attention Transfer for Knowledge Distillation. ACM Transactions on Multimedia Computing, Communications and Applications

    Book  Google Scholar 

  14. Liu Z, Sun M, Zhou T, Huang G, Darrell T (2018) Rethinking the value of network pruning, arXiv preprint http://arxiv.org/abs/1810.05270arXiv:1810.05270

  15. Polino A, Pascanu R, Alistarh D (2018) Model compression via distillation and quantization, arXiv preprint http://arxiv.org/abs/1802.05668

  16. Zhou Y, Moosavi-Dezfooli SM, Cheung SM, Frossard P (2018) Adaptive quantization for deep neural network. In Proceedings of the AAAI Conference on Artificial Intelligence 32(1)

  17. Li S (2020) Tensorflow lite: On-device machine learning framework. J. Comput. Res. Dev 57:1839

    Google Scholar 

  18. David R, Duke J, Jain A et al (2021) Tensorflow lite micro: Embedded machine learning for tinyml systems. Proceedings of Machine Learning and Systems (PMLR) 3:800–811

    Google Scholar 

  19. Dai J (2020) Real-time and accurate object detection on edge device with TensorFlow Lite. J Phys: Conf Ser 1651(1)

    Google Scholar 

  20. Zhang W, Li Z, Sun HH, Zhang Q, Zhuang P, Li C (2022) SSTNet: Spatial, spectral, and texture aware attention network using hyperspectral image for corn variety identification. IEEE Geosci Remote Sens Lett 19:1–5

    Google Scholar 

  21. Wan S, Qi L, Xu X, Tong C, Gu Z (2020) Deep learning models for real-time human activity recognition with smartphones. Mob Netw Appl 25:743–755

    Article  Google Scholar 

  22. Yuan P, Huang R (2021) Integrating the device-to-device communication technology into edge computing: A case study. Peer Peer Netw Appl 14(2):599–608

    Article  MathSciNet  Google Scholar 

  23. Hosang J, Benenson R, Schiele B (2017) Learning non-maximum suppression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.4507-4515

  24. Loshchilov I, Hutter F (2021) Sgdr: Stochastic gradient descent with warm restarts, arXiv preprint http://arxiv.org/abs/1608.03983

  25. Sunaryono D, Siswantoro J, Anggoro R (2021) An Android based course attendance system using face recognition. J King Saud Univ Comput Inform Sci 33(3):304–312

    Google Scholar 

  26. Chen C, Wang C, Liu B, He C, Cong L, Wan S (2023) Edge intelligence empowered vehicle detection and image segmentation for autonomous vehicles. IEEE Trans Intell Trans Syst

  27. Murray N, Marchesotti L, Perronnin F (2012) AVA: A large-scale database for aesthetic visual analysis, in 2012 IEEE conference on computer vision and pattern recognition pp. 2408-2415

  28. Hosu V, Goldlucke B, Saupe D (2019) Effective aesthetics prediction with multi-level spatially pooled features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 9375-9383

Download references

Funding

This study was supported in part by the National Natural Science Foundation of China under Grants 62072159, U1804164, 61902112, in part by the Science and Technology Foundation Project of Henan Province under Grant 222102210011 and in part by the Science and Technology Foundation of Henan Educational Committee under Grants 19A510015, 20A520019 and 20A520020.

Author information

Author notes

  1. Zhao Han and Xiaoyan Zhao contributed equally on this work.

Authors and Affiliations

  1. School of Computer and Information Engineering, Henan Normal University, Xinxiang, Henan, 453007, China

    Peiyan Yuan, Zhao Han & Xiaoyan Zhao

  2. Big Data Engineering Laboratory for Teaching Resources, Xinxiang, Henan, China

    Peiyan Yuan, Zhao Han & Xiaoyan Zhao

Authors
  1. Peiyan Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhao Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoyan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The three authors contributed equally to this work.

Corresponding author

Correspondence to Peiyan Yuan.

Ethics declarations

Ethics approval

Not applicable.

Consent for publication

The authors agree to publish this work.

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, P., Han, Z. & Zhao, X. Integrating the edge intelligence technology into image composition: A case study. Peer-to-Peer Netw. Appl. 16, 1641–1651 (2023). https://doi.org/10.1007/s12083-023-01480-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-023-01480-2

Keywords

Search

Navigation