Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3581783.3612585acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Edge-Assisted On-Device Model Update for Video Analytics in Adverse Environments

Published: 27 October 2023 Publication History

Abstract

While large deep neural networks excel at general video analytics tasks, the significant demand on computing capacity makes them infeasible for real-time inference on resource-constrained end cameras. In this paper, we propose an edge-assisted framework that continuously updates the lightweight model deployed on the end cameras to achieve accurate predictions in adverse environments. This framework consists of three modules, namely, a key frame extractor, a trigger controller, and a retraining manager. The low-cost key frame extractor obtains frames that can best represent the current environment. Those frames are then transmitted and buffered as the retraining data for model update at the edge server. Once the trigger controller detects a significant accuracy drop in the selected frames, the retraining manager outputs the optimal retraining configuration balancing the accuracy and time cost. We prototype our system on two end devices of different computing capacities with one edge server. The results demonstrate that our approach significantly improves accuracy across all tested adverse environment scenarios (up to 24%) and reduces more than 50% of the retraining time compared to existing benchmarks.

References

[1]
Ganesh Ananthanarayanan, Paramvir Bahl, Peter Bodík, Krishna Chintalapudi, Matthai Philipose, Lenin Ravindranath, and Sudipta Sinha. 2017. Real-time video analytics: The killer app for edge computing. computer 50, 10 (2017), 58--67.
[2]
Romil Bhardwaj, Zhengxu Xia, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Nikolaos Karianakis, Kevin Hsieh, Paramvir Bahl, and Ion Stoica. 2022. Ekya: Continuous learning of video analytics models on edge compute servers. In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22). 119--135.
[3]
Christopher Canel, Thomas Kim, Giulio Zhou, Conglong Li, Hyeontaek Lim, David G Andersen, Michael Kaminsky, and Subramanya Dulloor. 2019. Scaling video analytics on constrained edge nodes. Proceedings of Machine Learning and Systems 1 (2019), 406--417.
[4]
Jiasi Chen and Xukan Ran. 2019. Deep learning with edge computing: A review. Proc. IEEE 107, 8 (2019), 1655--1674.
[5]
Tiffany Yu-Han Chen, Lenin Ravindranath, Shuo Deng, Paramvir Bahl, and Hari Balakrishnan. 2015. Glimpse: Continuous, real-time object recognition on mobile devices. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. 155--168.
[6]
Xu Chen, Chenqiang Gao, Feng Yang, Xiaohan Wang, Yi Yang, and Yahong Han. 2021. Video-to-Image casting: A flatting method for video analysis. In Proceedings of the 29th ACM International Conference on Multimedia. 4958--4966.
[7]
Yan Cheng, Peng Yang, Ning Zhang, and Jiawei Hou. 2023. Edge-Assisted Lightweight Region-of-Interest Extraction and Transmission for Vehicle Perception. In GLOBECOM 2023-2023 IEEE Global Communications Conference. IEEE, to appear.
[8]
Xiangxiang Dai, Peng Yang, Xinyu Zhang, Zhewei Dai, and Li Yu. 2022. RESPIRE: Reducing Spatial-Temporal Redundancy for Efficient Edge-Based Industrial Video Analytics. IEEE Transactions on Industrial Informatics 18, 12 (2022), 9324--9334.
[9]
Kuntai Du, Ahsan Pervaiz, Xin Yuan, Aakanksha Chowdhery, Qizheng Zhang, Henry Hoffmann, and Junchen Jiang. 2020. Server-driven video streaming for deep learning inference. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 557--570.
[10]
Hongpeng Guo, Shuochao Yao, Zhe Yang, Qian Zhou, and Klara Nahrstedt. 2021. CrossRoI: Cross-camera region of interest optimization for efficient real time video analytics at scale. In Proceedings of the 12th ACM Multimedia Systems Conference. 186--199.
[11]
Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. 2020. Ghostnet: More features from cheap operations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1580--1589.
[12]
Lu He, Qianyu Zhou, Xiangtai Li, Li Niu, Guangliang Cheng, Xiao Li, Wenxuan Liu, Yunhai Tong, Lizhuang Ma, and Liqing Zhang. 2021. End-to-end video object detection with spatial-temporal transformers. In Proceedings of the 29th ACM International Conference on Multimedia. 1507--1516.
[13]
Yuanyi He, Peng Yang, Tian Qin, and Ning Zhang. 2023. End-Edge Coordinated Joint Encoding and Neural Enhancement for Low-Light Video Analytics. In GLOBECOM 2023--2023 IEEE Global Communications Conference. to appear.
[14]
Jiawei Hou, Peng Yang, Tian Qin, and Wen Wu. 2023. Edge-Coordinated On-Road Perception for Connected Autonomous Vehicles Using Point Cloud. In Proceedings of 31st Biennial Symposium on Communications. to appear.
[15]
Chien-Chun Hung, Ganesh Ananthanarayanan, Peter Bodik, Leana Golubchik, Minlan Yu, Paramvir Bahl, and Matthai Philipose. 2018. Videoedge: Processing camera streams using hierarchical clusters. In 2018 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 115--131.
[16]
Samvit Jain, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, and Joseph Gonzalez. 2019. Scaling video analytics systems to large camera deployments. In Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications. 9--14.
[17]
Samvit Jain, Xun Zhang, Yuhao Zhou, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Paramvir Bahl, and Joseph Gonzalez. 2020. Spatula: Efficient cross-camera video analytics on large camera networks. In 2020 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 110--124.
[18]
Junchen Jiang, Ganesh Ananthanarayanan, Peter Bodik, Siddhartha Sen, and Ion Stoica. 2018. Chameleon: scalable adaptation of video analytics. In Proceedings of the 2018 conference of the ACM special interest group on data communication. 253--266.
[19]
Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. 2017. Noscope: optimizing neural network queries over video at scale. arXiv preprint arXiv:1703.02529 (2017).
[20]
Mehrdad Khani, Ganesh Ananthanarayanan, Kevin Hsieh, Junchen Jiang, Ravi Netravali, Yuanchao Shu, Mohammad Alizadeh, and Victor Bahl. 2023. {RECL}: Responsive {Resource-Efficient} Continuous Learning for Video Analytics. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI. 917--932.
[21]
Mehrdad Khani, Pouya Hamadanian, Arash Nasr-Esfahany, and Mohammad Alizadeh. 2021. Real-time video inference on edge devices via adaptive model streaming. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4572--4582.
[22]
Dawei Li, Serafettin Tasci, Shalini Ghosh, Jingwen Zhu, Junting Zhang, and Larry Heck. 2019. RILOD: Near real-time incremental learning for object detection at the edge. In Proceedings of the 4th ACM/IEEE Symposium on Edge Computing. 113--126.
[23]
Yiting Li, Haisong Huang, Qingsheng Xie, Liguo Yao, and Qipeng Chen. 2018. Re-search on a surface defect detection algorithm based on MobileNet-SSD. Applied Sciences 8, 9 (2018), 1678.
[24]
Yuanqi Li, Arthi Padmanabhan, Pengzhan Zhao, Yufei Wang, Guoqing Harry Xu, and Ravi Netravali. 2020. Reducto: On-camera filtering for resource-efficient real-time video analytics. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 359--376.
[25]
Jie Lin, Peng Yang, Wen Wu, Ning Zhang, Tao Han, and Li Yu. 2021. Edge learning for low-latency video analytics: Query scheduling and resource allocation. In 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems (MASS). IEEE, 252--259.
[26]
Jie Lin, Peng Yang, Ning Zhang, Feng Lyu, Xianfu Chen, and Li Yu. 2022. Low-latency edge video analytics for on-road perception of autonomous ground vehicles. IEEE Transactions on Industrial Informatics 19, 2 (2022), 1512--1523.
[27]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 740--755.
[28]
Shengzhong Liu, Tianshi Wang, Jinyang Li, Dachun Sun, Mani Srivastava, and Tarek Abdelzaher. 2022. Adamask: Enabling machine-centric video streaming with adaptive frame masking for dnn inference offloading. In Proceedings of the 30th ACM international conference on multimedia. 3035--3044.
[29]
Davide Maltoni and Vincenzo Lomonaco. 2019. Continuous learning in single-incremental-task scenarios. Neural Networks 116 (2019), 56--73.
[30]
Rakesh Mehta and Cemalettin Ozturk. 2018. Object detection at 200 frames per second. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 0--0.
[31]
Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. 1953. Equation of state calculations by fast computing machines. The journal of chemical physics 21, 6 (1953), 1087--1092.
[32]
Ravi Teja Mullapudi, Steven Chen, Keyi Zhang, Deva Ramanan, and Kayvon Fatahalian. 2019. Online model distillation for efficient video inference. In Proceedings of the IEEE/CVF International conference on computer vision. 3573--3582.
[33]
Taslim Murad, Anh Nguyen, and Zhisheng Yan. 2022. DAO: Dynamic Adaptive Offloading for Video Analytics. In Proceedings of the 30th ACM International Conference on Multimedia. 3017--3025.
[34]
Shadi A Noghabi, Landon Cox, Sharad Agarwal, and Ganesh Ananthanarayanan. 2020. The emerging landscape of edge computing. GetMobile: Mobile Computing and Communications 23, 4 (2020), 11--20.
[35]
NVIDIA Corporation. 2017. NVIDIA Jetson TX2 Developer Kit. Retrieved 2017 from https://www.nvidia.com/en-us/autonomousmachines/embedded-systems/ jetson-tx2.
[36]
NVIDIA Corporation. 2019. NVIDIA Jetson Nano Developer Kit. Retrieved 2019 from https://developer.nvidia.com/embedded/jetson-nano-developer-kit.
[37]
Iyiola E Olatunji and Chun-Hung Cheng. 2019. Video analytics for visual surveil-lance and applications: An overview and survey. Machine Learning Paradigms: Applications of Learning and Analytics in Intelligent Systems (2019), 475--515.
[38]
Sibendu Paul, Utsav Drolia, Y Charlie Hu, and Srimat T Chakradhar. 2021. Aqua: Analytical quality assessment for optimizing video analytics systems. In 2021 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 135--147.
[39]
Zhaobo Qi, Shuhui Wang, Chi Su, Li Su, Weigang Zhang, and Qingming Huang. 2020. Modeling temporal concept receptive field dynamically for untrimmed video analysis. In Proceedings of the 28th ACM International Conference on Multimedia. 3798--3806.
[40]
Xukan Ran, Haolianz Chen, Xiaodan Zhu, Zhenming Liu, and Jiasi Chen. 2018. Deepdecision: A mobile deep learning framework for edge video analytics. In IEEE INFOCOM 2018-IEEE conference on computer communications. IEEE, 1421--1429.
[41]
Daniel Rivas, Francesc Guim, Jordà Polo, Pubudu M Silva, Josep Ll Berral, and David Carrera. 2022. Towards automatic model specialization for edge video analytics. Future Generation Computer Systems 134 (2022), 399--413.
[42]
Nishu Singla. 2014. Motion detection based on frame difference method. Interna-tional Journal of Information & Computation Technology 4, 15 (2014), 1559--1565.
[43]
Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning. PMLR, 6105--6114.
[44]
TensorRT 2018. TensorRT: Inference Accelerator for Precision Inference at Scale. Retrieved 2018 from https://developer.nvidia.com/tensorrt
[45]
Chengzhi Wang, Peng Yang, Jie Lin, Wen Wu, and Ning Zhang. 2022. Object-Based Resolution Selection for Efficient Edge-Assisted Multi-Task Video Analytics. In GLOBECOM 2022--2022 IEEE Global Communications Conference. IEEE, 5081--5086.
[46]
Can Wang, Sheng Zhang, Yu Chen, Zhuzhong Qian, Jie Wu, and Mingjun Xiao. 2020. Joint configuration adaptation and bandwidth allocation for edge-based real-time video analytics. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 257--266.
[47]
Lin Wang and Kuk-Jin Yoon. 2021. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE transactions on pattern analysis and machine intelligence 44, 6 (2021), 3048--3068.
[48]
Shangguang Wang, Yan Guo, Ning Zhang, Peng Yang, Ao Zhou, and Xuemin Shen. 2019. Delay-aware microservice coordination in mobile edge computing: A reinforcement learning approach. IEEE Transactions on Mobile Computing 20, 3 (2019), 939--951.
[49]
Yue Wang, Xiaofeng Tao, Xuefei Zhang, Ping Zhang, and Y Thomas Hou. 2019. Cooperative task offloading in three-tier mobile computing networks: An ADMM framework. IEEE Transactions on Vehicular Technology 68, 3 (2019), 2763--2776.
[50]
Wen Wu, Peng Yang, Weiting Zhang, Conghao Zhou, and Xuemin Shen. 2020. Accuracy-guaranteed collaborative DNN inference in industrial IoT via deep reinforcement learning. IEEE Transactions on Industrial Informatics 17, 7 (2020), 4988--4998.
[51]
Peng Yang, Jiawei Hou, Li Yu, Wenxiong Chen, and Ye Wu. 2023. Edge-coordinated energy-efficient video analytics for digital twin in 6G. China Communications 20, 2 (2023), 14--25.
[52]
Peng Yang, Feng Lyu, Wen Wu, Ning Zhang, Li Yu, and Xuemin Sherman Shen. 2019. Edge coordinated query configuration for low-latency and accurate video analytics. IEEE Transactions on Industrial Informatics 16, 7 (2019), 4855--4864.
[53]
Yolov5 v6.2 2022. YOLOv5 by Ultralytics. Retrieved Feb, 2022 from https://github. com/ultralytics/yolov5
[54]
Xiao Zeng, Biyi Fang, Haichen Shen, and Mi Zhang. 2020. Distream: scaling live video analytics with workload-adaptive distributed edge intelligence. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems. 409--421.
[55]
Ben Zhang, Xin Jin, Sylvia Ratnasamy, John Wawrzynek, and Edward A Lee. 2018. Awstream: Adaptive wide-area streaming analytics. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. 236--252.
[56]
Miao Zhang, Fangxin Wang, and Jiangchuan Liu. 2022. Casva: Configuration-adaptive streaming for live video analytics. In IEEE INFOCOM 2022-IEEE Conference on Computer Communications. IEEE, 2168--2177.
[57]
Wuyang Zhang, Zhezhi He, Luyang Liu, Zhenhua Jia, Yunxin Liu, Marco Gruteser, Dipankar Raychaudhuri, and Yanyong Zhang. 2021. Elf: accelerate high-resolution mobile deep vision with content-aware parallel offloading. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 201--214.
[58]
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. 2018. Shufflenet: An ex-tremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6848--6856.

Cited By

View all
  • (2024)AxiomVision: Accuracy-Guaranteed Adaptive Visual Model Selection for Perspective-Aware Video AnalyticsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681269(7229-7238)Online publication date: 28-Oct-2024
  • (2024)Adaptive Network Configuration for Efficient and Accurate Neural Video InferenceIEEE Transactions on Cognitive Communications and Networking10.1109/TCCN.2023.332087910:1(263-276)Online publication date: Feb-2024
  • (2024)Dependence-Aware Multitask Scheduling for Edge Video Analytics With Accuracy GuaranteeIEEE Internet of Things Journal10.1109/JIOT.2024.339729611:16(26970-26983)Online publication date: 15-Aug-2024
  • Show More Cited By

Index Terms

  1. Edge-Assisted On-Device Model Update for Video Analytics in Adverse Environments

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '23: Proceedings of the 31st ACM International Conference on Multimedia
    October 2023
    9913 pages
    ISBN:9798400701085
    DOI:10.1145/3581783
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 October 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. edge computing
    2. model update
    3. neural networks
    4. video analytics

    Qualifiers

    • Research-article

    Funding Sources

    • Natural Science Foundation of Hubei Province of China
    • Young Elite Scientists Sponsorship Program by CAST
    • Natural Science Foundation of China
    • Fundamental Research Funds for the Central Universities, HUST

    Conference

    MM '23
    Sponsor:
    MM '23: The 31st ACM International Conference on Multimedia
    October 29 - November 3, 2023
    Ottawa ON, Canada

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)402
    • Downloads (Last 6 weeks)23
    Reflects downloads up to 20 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)AxiomVision: Accuracy-Guaranteed Adaptive Visual Model Selection for Perspective-Aware Video AnalyticsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681269(7229-7238)Online publication date: 28-Oct-2024
    • (2024)Adaptive Network Configuration for Efficient and Accurate Neural Video InferenceIEEE Transactions on Cognitive Communications and Networking10.1109/TCCN.2023.332087910:1(263-276)Online publication date: Feb-2024
    • (2024)Dependence-Aware Multitask Scheduling for Edge Video Analytics With Accuracy GuaranteeIEEE Internet of Things Journal10.1109/JIOT.2024.339729611:16(26970-26983)Online publication date: 15-Aug-2024
    • (2024)DACAPO: Accelerating Continuous Learning in Autonomous Systems for Video Analytics2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00093(1246-1261)Online publication date: 29-Jun-2024
    • (2024)Bio-CEC: A Secure and Efficient Cloud-Edge Collaborative Biometrics System using Cancelable Biometrics2024 IEEE International Conference on Web Services (ICWS)10.1109/ICWS62655.2024.00102(833-843)Online publication date: 7-Jul-2024
    • (2024)Edge-Assisted Low-Latency Object Detection for Networked Vehicles Using Point Cloud2024 International Conference on Cloud and Network Computing (ICCNC)10.1109/ICCNC63989.2024.00017(49-56)Online publication date: 31-May-2024
    • (2023)Edge-Assisted Lightweight Region-of-Interest Extraction and Transmission for Vehicle PerceptionGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10436797(1054-1059)Online publication date: 4-Dec-2023
    • (2023)End-Edge Coordinated Joint Encoding and Neural Enhancement for Low-Light Video AnalyticsGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10436757(7363-7368)Online publication date: 4-Dec-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media