Nothing Special   »   [go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Towards automatic placement of media objects in a personalised TV experience

  • Special Issue Article
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

In this paper, we describe a Proof-of-Concept implementation of an automated system to drive one aspect of a personalised, object-based TV experience, on sports content, such as football and rugby. Our Proof-of-Concept uses a sequence of analytics processes, which can be performed offline or in real time, to allow a new media object to be automatically positioned on a multi-angle broadcast video sequence without occluding any key action, thus enabling additional graphic or video content to be used to personalise the broadcast for individual viewers. First, an object detection algorithm using a deep neural network model detects the players and ball, and its filtered output defines the region of interest within each frame of the video sequence. To avoid occlusion of key action by the media object, the remaining space within the sequence of frames is analysed to propose suitable locations. Our algorithm applies layout rules to ensure the object is placed in the best position based on broadcaster-defined templates (e.g. top-left, top-right, bottom-right and bottom-left). The output shows that our Proof-of-Concepts capable of processing video sequences with multiple camera angles and proposing the start time and duration of the media object without occluding any key action.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Armstrong, M.: Object-based media: A toolkit for building responsive content. In Proceedings of the 32nd International BCS Human Computer Interaction (HCI) Conference, Belfast, UK (2018)

  2. Howells, E., Jackson, D.: Object-based media report. Ofcom, London (2021)

    Google Scholar 

  3. Netflix, "Black Mirror: Bandersnatch," Netflix, 2018. [Online]. Available: https://www.netflix.com/gb/title/80988062. Accessed 11 June 2022

  4. Walker, J., Williams, D., Kegel, I., Gower, A., Jansen, J., Lomas, M., Fjellsten, S.: 2-IMMERSE: a platform for production, delivery, and orchestration of distributed media applications. SMPTE Motion Imaging J 128(7), 45–51 (2019)

    Article  Google Scholar 

  5. Cox, J., Brooks, M., Forrester, I., Armstrong, M.: Moving object-based media production from one-off examples to scalable workflows. SMPTE Motion Imaging J 127(4), 32–37 (2018)

    Article  Google Scholar 

  6. Röggla, T., Li, J., Jansen, J., Fjellsten, S., Kegel, I., Pilgrim, L., Trimby, M., Williams, D., Cesar, P.: From the lab to the OB truck: Object-based broadcasting at the FA Cup in Wembley Stadium. In CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK (2019)

  7. Ibrahim, M., Lohmar, T., El-Essaili, A., d'Allonnes, A.:TV graphics personalization using in-band events. In Proceedings of the In-Programme Personalization for Broadcast (IPP4B) Workshop, ACM TVX2017, Hilversum, The Netherlands (2017)

  8. ITU-R: Artificial intelligence systems for programme production and exchange. BT Series (2019)

  9. Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: a benchmark. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA (2009)

  10. Dollar, P., Wojek, C., Schiele, B., Perona, P.: Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34(4), 743–761 (2012)

    Article  Google Scholar 

  11. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    Article  MATH  Google Scholar 

  12. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), San Diego, USA (2005)

  13. Dollar, P., Appel, R., Belongie, S., Perona, P.: Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1532–1545 (2014)

    Article  Google Scholar 

  14. Carranza-García, M., Torres-Mateo, J., Lara-Benítez, P., García-Gutiérrez, J.: On the performance of one-stage and two-stage object detectors in autonomous vehicles using camera data. Remote Sensing 13(1), 89 (2021)

    Article  Google Scholar 

  15. Corcoll Andreu, O.: Semantic image cropping. Queen Mary University of London (2018)

  16. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: SSD: Single shot multibox detector. In: Computer Vision – ECCV 2016: 14th European Conference, Part 1, Amsterdam, The Netherlands (2016)

  17. Redmon, J., Farhadi, A.: YOLO9000: Better, faster, stronger. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA (2017)

  18. Jiao, L., Zhang, F., Liu, F., Yang, S., Li, L., Feng, Z., Qu, R.: A survey of deep learning-based object detection. IEEE Access 7, 128837–128868 (2019)

    Article  Google Scholar 

  19. Tan, L., Huangfu, T., Wu, L., Chen, W.: Comparison of RetinaNet, SSD, and YOLO v3 for real-time pill identification. BMC Med. Inform. Decis. Mak. 21, 324 (2021)

    Article  Google Scholar 

  20. Morera, Á., Sánchez, Á., Moreno, A.B., Sappa, Á.D., Vélez, J.F.: SSD vs. YOLO for detection of outdoor urban advertising panels under multiple variabilities. Sensors 20(16), 4587 (2020)

    Article  Google Scholar 

  21. Alkentar, S., Alsahwa, B., Assalem, A., Karakolla, D.: Practical comparation of the accuracy and speed of YOLO, SSD and Faster RCNN for drone detection. J. Eng. 27(8), 19–31 (2021)

    Article  Google Scholar 

  22. NVIDIA, "NVIDIA DeepStream SDK Developer Guide," 08 March 2020. [Online]. Available: https://docs.nvidia.com/metropolis/deepstream/5.0/dev-guide/index.htmlpage/DeepStream_Development_Guide/deepstream_performance.htmlwwpID0E0YD0HA. Accessed 11 June 2022

  23. Zhong, Y., Wang, J., Peng, J., Zhang, L.: Anchor box optimization for object detection. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass, USA (2020)

  24. Stats Perform, "World Leaders in Sport Data," 2022. [Online]. Available: https://www.statsperform.com/opta/. Accessed 11 June 2022

  25. Sani, Y., Mauthe, A., Edwards, C.: Adaptive bitrate selection: a survey. IEEE Commun. Surv. Tutorials 19(4), 2985–3014 (2017)

    Article  Google Scholar 

  26. Fautier, T.: How OTT services can match the quality of broadcast. SMPTE Motion Imaging J. 129(3), 16–25 (2020)

    Article  Google Scholar 

  27. Jackson,: Broadcast lag in live online TV sport streaming frustrates fans. ISP Review, 10 June 2021. [Online]. Available: https://www.ispreview.co.uk/index.php/2021/06/broadcast-lag-in-live-online-tv-sport-streaming-frustrates-fans.html. Accessed 11 June 2022

  28. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy (2017)

  29. NVIDIA, "DeepStream SDK," 2022. [Online]. Available: https://developer.nvidia.com/deepstream-sdk. Accessed 11 June 2022

  30. NVIDIA, "DetectNet_v2 TAO Toolkit 3.0 documentation," 25 August 2021. [Online]. Available: https://docs.nvidia.com/tao/tao-toolkit/text/object_detection/detectnet_v2.html. Accessed 12 November 2021

  31. BT Sport, "Man City vs Liverpool (1-1, 5-4 on pens) | 2019 Community Shield highlights," BT Sport, 4 August 2019. [Online]. Available: https://www.youtube.com/watch?v=k9_tz9bi3rs. Accessed 11 June 2022

  32. Apache Software Foundation: Kafka 3.0 Documentation. 2017. [Online]. Available: https://kafka.apache.org/documentation/. Accessed 11 June 2022

  33. Armstrong, M., Brown, A., Crabb, M., Hughes, C.J., Jones, R., Sandford, J.: Understanding the diverse needs of subtitle users in a rapidly evolving media landscape. SMPTE Motion Imaging J. 125(9), 33–41 (2016)

    Article  Google Scholar 

  34. Law, E.L.-C., van Schaik, P.: Modelling user experience - an agenda for research and practice. Interact. Comput. 22(5), 313–322 (2010)

    Article  Google Scholar 

  35. Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A.P.O.S., Kort, J.: Understanding, scoping and defining user experience: A survey approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'09), Boston, USA (2009)

  36. Likert, R.: A technique for the measurement of attitudes. Arch. Psychol. 22(140), 5–55 (1932)

    Google Scholar 

  37. Kruger, J.-L., Doherty, S., Fox, W., de Lissa, P.: Multimodal measurement of cognitive load during subtitle processing: Same-language subtitles for foreign-language viewers. In: Lacruz, I., Jääskeläinen, R. (eds.) Innovation and expansion in translation process research, pp. 267–294. John Benjamins Publishing Company, Amsterdam, Netherlands (2018)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. British Telecommunications Plc, Ipswich, UK

    Brahim Allan, Ian Kegel & Sri Harish Kalidass

  2. Ulster University, Belfast, Northern Ireland

    Andriy Kharechko, Michael Milliken, Sally McClean, Bryan Scotney & Shuai Zhang

Authors
  1. Brahim Allan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ian Kegel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sri Harish Kalidass
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andriy Kharechko
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Milliken
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sally McClean
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bryan Scotney
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shuai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brahim Allan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Allan, B., Kegel, I., Kalidass, S.H. et al. Towards automatic placement of media objects in a personalised TV experience. Multimedia Systems 28, 2175–2192 (2022). https://doi.org/10.1007/s00530-022-00974-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00974-y

Keywords

Search

Navigation