Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Event-Driven Serverless Pipelines for Video Coding and Quality Metrics

Published: 21 March 2023 Publication History

Abstract

Nowadays, the majority of Internet traffic is multimedia content. Video streaming services are in high demand by end users and use HTTP Adaptive Streaming (HAS) as transmission protocol. HAS splits the video into non-overlapping chunks and each video chunk can be encoded independently using different representations. Therefore, these encode tasks can be parallelized and Cloud computing can be used for this. However, in the most extended solutions, the infrastructure must be configured and provisioned in advance. Recently, serverless platforms have made posible to deploy functions that can scale from zero to a configurable maximum. This work presents and analyses the behavior of event-driven serverless functions to encode video chunks and to compute, optionally, the quality of the encoded videos. These functions have been implemented using an adapted version of embedded Tomcat to deal with CloudEvents. We have deployed these event-driven serverless pipelines for video coding and quality metrics on an on-premises serverless platform based on Knative on one master node and eight worker nodes. We have tested the scalability and resource consumption of the proposed solution using two video codecs: x264 and AV1, varying the maximum number of replicas and the resources allocated to them (fat and slim function replicas). We have encoded different 4K videos to generate multiple representations per function call and we show how it is possible to create pipelines of serverless media functions. The results of the different tests carried out show the good performance of the serverless functions proposed. The system scales the replicas and distributes the jobs evenly across all the replicas. The overall encoding time is reduced by 18% using slim replicas but fat replicas are more adequate in live video streaming as the encoding time per chunk is reduced. Finally, the results of the pipeline test show an appropriate distribution and chaining among the available replicas of each function type.

References

[1]
Ao, L., Izhikevich, L., Voelker, G.M., Porter, G.: Sprocket: a serverless video processing framework. In: Proceedings of the ACM Symposium on Cloud Computing, SoCC ’18, pp. 263–274. Association for Computing Machinery, New York, NY, USA. (2018)
[2]
CloudEvents: Cloudevents project. https://cloudevents.io/. Accessed 01 Aug 2022 (2022)
[3]
Dong, Y., Zhang, X., Zhao, Y., Song, L.: A containerized media cloud for video transcoding service. In: 2018 IEEE International Conference on Consumer Electronics (ICCE), pp. 1–4. (2018)
[4]
FFmpeg: Ffmpeg tool. https://ffmpeg.org/ . Accessed 01 Aug 2022 (2022)
[5]
Fouladi, S., Wahby, R. S., Shacklett, B., Balasubramaniam, K. V., Zeng, W., Bhalerao, R., Sivaraman, A., Porter, G., Winstein, K.: Encoding, fast and slow: Low-latency video processing using thousands of tiny threads. In: 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17), pp. 363–376 (2017)
[6]
Guo, L., De Cock, J., Aaron, A.: Compression performance comparison of X264, X265, Libvpx and Aomenc for On-demand adaptive streaming applications. In: 2018 Picture Coding Symposium (PCS), pp. 26–30. (2018)
[7]
Gutiérrez-Aguado, J.: Adapting embeded Tomcat to develop event-driven serverless functions. In: JCIS2022. SISTEDES. http://hdl.handle.net/11705/JCIS/2022/040 (2022)
[8]
Gutiérrez-Aguado, J., Peña-Ortiz, R., Garcia-Pineda, M., Claver, J.M.: A cloud-based distributed architecture to accelerate video encoders. Applied Sciences 10(15). https://www.mdpi.com/2076-3417/10/15/5070 (2020)
[9]
Gutiérrez-Aguado J, Peña-Ortiz R, García-Pineda M, and Claver JM Cloud-based elastic architecture for distributed video encoding: evaluating H.265, VP9, and AV1 J. Netw. Comput. Appl. 2020 171 102782 https://www.sciencedirect.com/science/article/pii/S1084804520302563
[10]
IBM Watson Media Support Center: Internet connection and recommended encoding settings – IBM Watson Media. https://support.video.ibm.com/hc/en-us/articles/207852117-Internet-connection-and-recommended-encoding-settings
[11]
Jangda, A., Pinckney, D., Brun, Y., Guha, A.: Formal foundations of serverless computing. Proc. ACM Program. Lang. 3(OOPSLA). (2019)
[12]
Jeon M, Kim N, and Lee B Mapreduce-based distributed video encoding using content-aware video segmentation and scheduling IEEE Access 2016 4 6802-6815
[13]
Kemp, S.: Digital 2022: Global Overview Report. https://datareportal.com/reports/digital-2022-global-overview-report (2022)
[14]
Kerdranvat M, Chen Y, Jullian R, Galpin F, and François E The video codec landscape in 2020 ITU Journal: ICT Discoveries 2020 3 1 73-83 http://handle.itu.int/11.1002/pub/8153d78c-en
[15]
Kim M, Cui Y, Han S, and Lee H Towards efficient design and implementation of a hadoop-based distributed video transcoding system in cloud computing environment Int. J. Multimed. Ubiquit. Eng. 2013 8 2 213-224
[16]
Knative: Knative is an open-source enterprise-level solution to build serverless and event driven applications. https://knative.dev/docs/. Accessed 01 Aug 2022 (2022)
[17]
Lederer, S.: Optimal adaptive streaming formats mpeg-dash & hls segment length. Bitmovin . https://bitmovin.com/mpeg-dash-hls-segment-length/ (2020)
[18]
Li, J., Kulkarni, S.G., Ramakrishnan, K.K., Li, D.: Analyzing open-source serverless platforms: characteristics and performance. International Conferences on Software Engineering and Knowledge Engineering .. (2021)
[19]
Li, Z., Aaron, A., Katsavounidis, I., Moorthy, A., Manohara, M.: Toward a practical perceptual video quality metric. The Netflix Tech. Blog 6(2) (2016)
[20]
Li, Z., Duanmu, Z., Liu, W., Wang, Z.: AVC, HEVC, VP9, AVS2 or AV1? - a comparative study of state-of-the-art video encoders on 4K videos. In: ICIAR (2019)
[21]
Marathe, N., Gandhi, A., Shah, J. M.: Docker Swarm and Kubernetes in cloud computing environment. In: 2019 3Rd International Conference on Trends in Electronics and Informatics (ICOEI), pp. 179–184 (2019),
[22]
Martins H, Araujo F, and da Cunha PRBenchmarking serverless computing platformsJournal of Grid Computing2020184691-709https://doi.org/10.1007/s10723-020-09523-1
[23]
Moravcik, M., Kontsek, M.: Overview of docker container orchestration tools. In: 2020 18th International Conference on Emerging Elearning Technologies and Applications (ICETA), pp. 475–480. (2020)
[24]
Pereira, R., Azambuja, M., Breitman, K., Endler, M.: An architecture for distributed high performance video processing in the cloud. In: 2010 IEEE 3Rd International Conference on Cloud Computing, pp. 482–489 (2010),
[25]
Pääkkönen, P., Heikkinen, A., Aihkisalo, T.: Architecture for predicting live video transcoding performance on docker containers. In: 2018 IEEE International Conference on Services Computing (SCC), pp. 65–72 (2018),
[26]
Risco S, Moltó G, Naranjo DM, and Blanquer I Serverless workflows for containerised applications in the cloud continuum Journal of Grid Computing 2021 19 3 30
[27]
Sameti, S., Wang, M., Krishnamurthy, D.: Stride: distributed video transcoding in spark. In: 2018 IEEE 37th International Performance Computing and Communications Conference (IPCCC), pp. 1–8 (2018),
[28]
Sameti, S., Wang, M., Krishnamurthy, D.: Contrast: container-based transcoding for interactive video streaming. In: NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium, pp. 1–9 (2020),
[29]
Seufert M, Egger S, Slanina M, Zinner T, Hoßfeld T, and Tran-Gia P A survey on quality of experience of http adaptive streaming IEEE Communications Surveys & Tutorials 2014 17 1 469-492
[30]
Sharma, P., Chaufournier, L., Shenoy, P., Tay, Y.C.: Containers and virtual machines at scale: a comparative study. In: Proceedings of the 17th International Middleware Conference, Middleware ’16. Association for Computing Machinery, New York, NY, USA (2016),
[31]
Song, C., Shen, W., Sun, L., Lei, Z., Xu, W.: Distributed video transcoding based on mapreduce. In: 2014 IEEE/ACIS 13th International Conference on Computer and Information Science (ICIS), pp. 309–314 (2014),
[32]
Taibi D, Spillner J, and Wawruch KServerless computing-where are we now, and where are we heading?IEEE Softw.202138125-31https://doi.org/10.1109/MS.2020.3028708
[33]
Wang, L., Li, M., Zhang, Y., Ristenpart, T.: Swift, M.: Peeking behind the curtains of serverless platforms. In: 2018 USENIX Annual Technical Conference (USENIX ATC 18), pp. 133–146. USENIX Association, Boston, MA. https://www.usenix.org/conference/atc18/presentation/wang-liang (2018)

Cited By

View all
  • (2024)Streamlining Cloud-Native Application Development and Deployment with Robust EncapsulationProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698552(847-865)Online publication date: 20-Nov-2024
  • (2024)Towards GPU-enabled serverless cloud edge platforms for accelerating HEVC video codingCluster Computing10.1007/s10586-024-04692-028:1Online publication date: 5-Nov-2024

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Journal of Grid Computing
Journal of Grid Computing  Volume 21, Issue 2
Jun 2023
286 pages

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 21 March 2023
Accepted: 14 January 2023
Received: 01 August 2022

Author Tags

  1. Serverless
  2. Function as a service
  3. CloudEvents
  4. Video coding
  5. Quality metrics
  6. HTTP adaptive streaming

Qualifiers

  • Research-article

Funding Sources

  • Universitat de Valencia

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Streamlining Cloud-Native Application Development and Deployment with Robust EncapsulationProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698552(847-865)Online publication date: 20-Nov-2024
  • (2024)Towards GPU-enabled serverless cloud edge platforms for accelerating HEVC video codingCluster Computing10.1007/s10586-024-04692-028:1Online publication date: 5-Nov-2024

View Options

View options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media