research-article

DAO: Dynamic Adaptive Offloading for Video Analytics

Authors:

Zhisheng YanAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 3017 - 3025

https://doi.org/10.1145/3503161.3548249

Published: 10 October 2022 Publication History

Abstract

Offloading videos from end devices to edge or cloud servers is the key to enabling computation-intensive video analytics. To ensure the analytics accuracy at the server, the video quality for offloading must be configured based on the specific content and the available network bandwidth. While adaptive video streaming for user viewing has been widely studied, none of the existing works can guarantee the analytics accuracy at the server in bandwidth- and content-adaptive way. To fill in this gap, this paper presents DAO, a dynamic adaptive offloading framework for video analytics that jointly considers the dynamics of network bandwidth and video content. DAO is able to maximize the analytics accuracy at the server by adapting the video bitrate and resolution dynamically. In essence, we shift the context of adaptive video transport from traditional DASH systems to a new dynamic adaptive offloading framework tailored for video analytics. DAO is empowered by some new discoveries about the inherent relationship between analytics accuracy, video content, bitrate, and resolution, as well as by an optimization formulation to adapt the bitrate and resolution dynamically. Results from the real-world implementation of object detection tasks show that DAO's performance is close to the theoretical bound, achieving 20% bandwidth saving and 59% category-wise mAP improvement compared to conventional DASH schemes.

Supplementary Material

MP4 File (MM22-fp2082.mp4)

Presentation video

Download
22.89 MB

References

[1]

[n. d.]. DASH. ([n. d.]). https://us.hikvision.com/sites/default/files/tb/tb_bit_ rate_chart_120115us_0.pdf

[2]

[n. d.]. YouTube-VOS A Large-Scale Benchmark for Video Object Segmentation. https://youtube-vos.org/. Accessed: 2022-04-01.

[3]

2017. The Wonder Shaper. (2017). http://lartc.org/wondershaper

[4]

M. Abomhara, O.O. Khalifa, O. Zakaria, A.A. Zaidan, B.B. Zaidan, and A. Rame. 2010. Video Compression Techniques: An Overview. Journal of Applied Sciences, 10: 1834--1840 1 (2010).

[5]

Shivang Aggarwal, Sibendu Paul, Pranab Dash, Nuka Saranya Illa, Y. Charlie Hu, Dimitrios Koutsonikolas, and Zhisheng Yan. 2020. How to Evaluate Mobile 360 Video Streaming Systems?. In ACM International Workshop on Mobile Computing Systems and Applications (HotMobile).

Digital Library

[6]

Fabrice Bellard. https://www.ffmpeg.org. (FFmpeg https://www.ffmpeg.org).

[7]

Lahiru D Chamain, Sen-ching Samson Cheung, and Zhi Ding. 2019. Quannet: Joint image compression and classification over channels with limited bandwidth. In 2019 IEEE International Conference on Multimedia and Expo (ICME). 338--343.

[8]

Lahiru D Chamain, Fabien Racapé, Jean Bégaint, Akshay Pushparaja, and Simon Feltman. 2021. End-to-end optimized image compression for machines, a study. In 2021 Data Compression Conference (DCC). 163--172.

[9]

Bo Chen, Zhisheng Yan, Hongpeng Guo, Zhe Yang, Ahmed Ali-Eldin, Prashant Shenoy, and Klara Nahrstedt. 2021. Deep Contextualized Compressive Offloading for Images. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems.

Digital Library

[10]

Bo Chen, Zhisheng Yan, and Klara Nahrstedt. 2022. Context-aware Image Compression Optimization for Visual Analytics Offloading. In ACM Multimedia Systems Conference (MMSys).

[11]

Tiffany Yu-Han Chen, Lenin Ravindranath, Shuo Deng, Paramvir Bahl, and Hari Balakrishnan. 2015. Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems (SenSys '15). 155--168. 6 (2015).

[12]

Samuel F. Dodge and Lina Karam. 2017. A Study and Comparison of Human and Deep Learning Recognition Performance under Visual Distortions. 2017 26th International Conference on Computer Communication and Networks (ICCCN) (2017), 1--7.

[13]

Samuel F. Dodge and Lina Karam. 2019. Human and DNN Classification Performance on Images With Quality Distortions. ACM Transactions on Applied Perception (TAP) 16 (2019), 1 -- 17.

Digital Library

[14]

Kuntai Du, Ahsan Pervaiz, Xin Yuan, Aakanksha Chowdhery, Qizheng Zhang, Henry Hoffmann, and Junchen Jiang. 2020. Server-driven video streaming for deep learning inference. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication (SIGCOMM). 557--570.

[15]

Fanyi Duanmu, Eymen Kurdoglu, S Amir Hosseini, Yong Liu, and YaoWang. 2017. Prioritized Buffer Control in Two-tier 360 Video Streaming. In ACM Workshop on Virtual Reality and Augmented Reality Network.

Digital Library

[16]

Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. 2010. The PASCAL Visual Object Classes (VOC) Challenge. International Journal of Computer Vision 88, 2 (2010), 303--338.

Digital Library

[17]

Leonardo Galteri, Marco Bertini, Lorenzo Seidenari, and Alberto Del Bimbo. 2018. Video compression for object detection algorithms. In 2018 24th International Conference on Pattern Recognition (ICPR). 3007--3012.

[18]

Song Han, Huizi Mao, and William J Dally. 2016. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. In International Conference on Learning Representations (ICLR).

[19]

Puneet Jain, Justin Manweiler, and Romit Roy Choudhury. 2016. Low Bandwidth Offload for Mobile AR. In ACM International on Conference on Emerging Networking EXperiments and Technologies (CoNEXT).

[20]

Jean Le Feuvre and Cyril Concolato. 2016. Tiled-based Adaptive Streaming Using MPEG-DASH. In ACM Multimedia Systems Conference (MMSys). 41:1--41:3.

[21]

Zhi Li, Xiaoqing Zhu, Joshua Gahm, Rong Pan, Hao Hu, Ali C Begen, and David Oran. 2014. Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE Journal on Selected Areas in Communications 32, 4 (2014), 719--733.

[22]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. 740--755.

[23]

Fang Liu, Yeting Guo, Zhiping Cai, Nong Xiao, and Ziming Zhao. 2019. Edgeenabled disaster rescue: a case study of searching for missing people. ACM Transactions on Intelligent Systems and Technology (TIST) 10, 6 (2019), 1--21.

Digital Library

[24]

Luyang Liu, Hongyu Li, and Marco Gruteser. 2019. Edge assisted real-time object detection for mobile augmented reality. In The 25th Annual International Conference on Mobile Computing and Networking. 1--16.

Digital Library

[25]

Qiang Liu, Siqi Huang, Johnson Opadere, and Tao Han. 2018. An Edge Network Orchestrator for Mobile Augmented Reality. In IEEE International Conference on Computer Communications (INFOCOM).

[26]

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. SSD: Single shot multibox detector In European Conference on Computer Vision (ECCV).

[27]

Guo Lu,Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, and Zhiyong Gao. 2019. Dvc: An end-to-end deep video compression framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11006-- 11015.

[28]

Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 197--210.

Digital Library

[29]

Jacob Mattingley, YangWang, and Stephen Boyd. 2011. Receding horizon control. IEEE Control Systems Magazine 31, 3 (2011), 52--65.

[30]

Afshin Taghavi Nasrabadi, Anahita Mahzari, Joseph D. Beshay, and Ravi Prakash. 2017. Adaptive 360-Degree Video Streaming using Scalable Video Coding. In ACM International Conference on Multimedia (MM).

Digital Library

[31]

Viet-Anh Nguyen, Yap-Peng Tan, and Weisi Lin. 2008. Adaptive downsampling/ upsampling for better video compression at low bit rate. In 2008 IEEE International Symposium on Circuits and Systems (ISCAS). 1624--1627.

[32]

Chrisma Pakha, Aakanksha Chowdhery, and Junchen Jiang. 2018. Reinventing video streaming for distributed vision analytics. In Proceedings of the 10th USENIX Conference on Hot Topics in Cloud Computing (HotCloud'18) 5 (2018).

[33]

Martin L Puterman. 1990. Markov decision processes. Handbooks in operations research and management science 2 (1990), 331--434.

[34]

Yanyuan Qin, Shuai Hao, Krishna R Pattipati, Feng Qian, Subhabrata Sen, Bing Wang, and Chaoqun Yue. 2019. Quality-aware strategies for optimizing ABR video streaming QoE and reducing data usage. In Proceedings of the 10th ACM Multimedia Systems Conference. 189--200.

Digital Library

[35]

Xukan Ran, Haoliang Chen, Xiaodan Zhu, Zhenming Liu, and Jiasi Chen. 2018. DeepDecision: A Mobile Deep Learning Framework for Edge Video Analytics. In IEEE International Conference on Computer Communications (INFOCOM).

[36]

Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. ArXiv abs/1804.02767 (2018).

[37]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, et al. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 3 (2015), 211--252.

Digital Library

[38]

Liyang Sun, Fanyi Duanmu, Yong Liu, YaoWang, Yinghua Ye, Hang Shi, and David Dai. 2018. Multi-path multi-tier 360-degree video streaming in 5G networks. In ACM Multimedia Systems Conference (MMSys). 162--173.

Digital Library

[39]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1--9.

[40]

H. To, S. H. Kim, and C. Shahabi. 2015. Effectively crowdsourcing the acquisition and analysis of visual data for disaster response. IEEE International Conference on Big Data (Big Data), pp. 697--706, 11 (2015).

Digital Library

[41]

Fei-Yue Wang, Jun Jason Zhang, Xinhu Zheng, Xiao Wang, Yong Yuan, Xiaoxiao Dai, Jie Zhang, and Liuqing Yang. 2016. Where does AlphaGo go: From churchturing thesis to AlphaGo thesis and beyond. IEEE/CAA Journal of Automatica Sinica 3, 2 (2016), 113--120.

[42]

Xiufeng Xie and Kyu-Han Kim. 2019. Source compression with bounded dnn perception loss for iot edge computer vision. In The 25th Annual International Conference on Mobile Computing and Networking. 1--16.

Digital Library

[43]

Zhisheng Yan and Chang Wen Chen. 2016. RnB: Rate and Brightness Adaptation for Rate-Distortion-Energy Tradeoff in HTTP Adaptive Streaming over Mobile Devices. In ACM International Conference on Mobile Computing and Networking (MobiCom).

Digital Library

[44]

Shuochao Yao, Yiran Zhao, Huajie Shao, ShengZhong Liu, Dongxin Liu, Lu Su, and Tarek Abdelzaher. 2018. Fastdeepiot: Towards understanding and optimizing neural network execution time on mobile and embedded devices. In Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems. 278--291.

Digital Library

[45]

Jun Yi, Md Reazul Islam, Shivang Aggarwal, Dimitrios Koutsonikolas, Y Charlie Hu, and Zhisheng Yan. 2020. An Analysis of Delay in Live 360 Video Streaming Systems. In ACM International Conference on Multimedia (MM).

Digital Library

[46]

Jun Yi, Shiqing Luo, and Zhisheng Yan. 2019. A Measurement Study of YouTube 360° Live Video Streaming. In ACM Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV).

Digital Library

[47]

Shanhe Yi, Zijiang Hao, Qingyang Zhang, Quan Zhang, Weisong Shi, and Qun Li. 2017. Lavea: Latency-aware video analytics on edge computing platform. In ACM/IEEE Symposium on Edge Computing (SEC).

Digital Library

[48]

Tan Zhang, Aakanksha Chowdhery, Paramvir (Victor) Bahl, Kyle Jamieson, and Suman Banerjee. 2015. The Design and Implementation of a Wireless Video Surveillance System. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (MobiCom '15). 426--438. 7 (2015).

Digital Library

Cited By

Yan SKan NLi CDai WZou JXiong HCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Task-Oriented Multi-Bitstream Optimization for Image Compression and Transmission via Optimal TransportProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681523(3695-3703)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681523
Ghasemi MKostic ZGhaderi JZussman GGanesan DLane NShi W(2024)EdgeCloudAI: Edge-Cloud Distributed Video AnalyticsProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3698857(1778-1780)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3698857
Xiao XZuo YYan MWang WHe JZhang Q(2024)Task-Oriented Video Compressive Streaming for Real-Time Semantic SegmentationIEEE Transactions on Mobile Computing10.1109/TMC.2024.344618523:12(14396-14413)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3446185
Show More Cited By

Index Terms

DAO: Dynamic Adaptive Offloading for Video Analytics
1. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming

Recommendations

Large-scale Video Analytics with Cloud–Edge Collaborative Continuous Learning
Deep learning–based video analytics demands high network bandwidth to ferry the large volume of data when deployed on the cloud. When incorporated at the edge side, only lightweight deep neural network (DNN) models are affordable due to computational ...
Towards cloud-edge collaborative online video analytics with fine-grained serverless pipelines
MMSys '21: Proceedings of the 12th ACM Multimedia Systems Conference

The ever-growing deployment scale of surveillance cameras and the users' increasing appetite for real-time queries have urged online video analytics. Synergizing the virtually unlimited cloud resources with agile edge processing would deliver an ideal ...
Big Data Analytics

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

SF OAC grants

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
411
Total Downloads

Downloads (Last 12 months)115
Downloads (Last 6 weeks)10

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yan SKan NLi CDai WZou JXiong HCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Task-Oriented Multi-Bitstream Optimization for Image Compression and Transmission via Optimal TransportProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681523(3695-3703)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681523
Ghasemi MKostic ZGhaderi JZussman GGanesan DLane NShi W(2024)EdgeCloudAI: Edge-Cloud Distributed Video AnalyticsProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3698857(1778-1780)Online publication date: 4-Dec-2024
https://dl.acm.org/doi/10.1145/3636534.3698857
Xiao XZuo YYan MWang WHe JZhang Q(2024)Task-Oriented Video Compressive Streaming for Real-Time Semantic SegmentationIEEE Transactions on Mobile Computing10.1109/TMC.2024.344618523:12(14396-14413)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3446185
Dai PChao YWu XLiu KGuo S(2024)Context-Aware Offloading for Edge-Assisted On-Device Video Analytics Through Online Learning ApproachIEEE Transactions on Mobile Computing10.1109/TMC.2024.341860823:12(12761-12777)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3418608
Yang PCheng YZhang NCheng QYu L(2024)Adaptive Network Configuration for Efficient and Accurate Neural Video InferenceIEEE Transactions on Cognitive Communications and Networking10.1109/TCCN.2023.332087910:1(263-276)Online publication date: Feb-2024
https://doi.org/10.1109/TCCN.2023.3320879
Zhang WJing YZhang YLin TYan J(2024)Retina-U: A Two-Level Real-Time Analytics Framework for UHD Live Video StreamingIEEE Transactions on Broadcasting10.1109/TBC.2023.334564670:2(429-440)Online publication date: Jun-2024
https://doi.org/10.1109/TBC.2023.3345646
Wang CYang PHou JLiu ZZhang N(2024)Dependence-Aware Multitask Scheduling for Edge Video Analytics With Accuracy GuaranteeIEEE Internet of Things Journal10.1109/JIOT.2024.339729611:16(26970-26983)Online publication date: 15-Aug-2024
https://doi.org/10.1109/JIOT.2024.3397296
Li TLi QZhang MYuan ZJiang Y(2024)Adaptive Streaming Continuous Learning System for Video Analytics2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682886(1-10)Online publication date: 19-Jun-2024
https://doi.org/10.1109/IWQoS61813.2024.10682886
Peng HZhan YLi PXia Y(2024)Tangram: High-Resolution Video Analytics on Serverless Platform with SLO-Aware Batching2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00066(645-655)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCS60910.2024.00066
Beye FBabazaki YAndo ROshiba TNihei KTakahashi K(2024)SwitchingNet: Edge-Assisted Model Switching for Accurate Video Recognition Over Best-Effort Networks2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454650(37-43)Online publication date: 6-Jan-2024
https://doi.org/10.1109/CCNC51664.2024.10454650
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten