Distributed Edge Inference: an Experimental Study on Multiview Detection
Article No.: 30, Pages 1 - 6
Abstract
Computing is evolving rapidly to cater to the increasing demand for sophisticated services, and Cloud computing lays a solid foundation for flexible on-demand provisioning. However, as the size of applications grows, the centralised client-server approach used by Cloud computing increasingly limits the applications' scalability. To achieve ultra-scalability, cloud/edge/fog computing converges into the compute continuum, completely decentralising the infrastructure to encompass universal, pervasive resources. The compute continuum makes devising applications benefitting from this complex environment a challenging research problem. We put the opportunities the compute continuum offers to the test through a real-world multi-view detection model (MvDet) implemented with the FastFL C/C++ high-performance edge inference framework. Computational performance is discussed considering many experimental scenarios, encompassing different edge computational capabilities and network bandwidths. We obtain up to 1.92x speedup in inference time over a centralised solution using the same devices.
References
[1]
[n. d.]. OpenSignal Italy - Mobile Network Experience Report - November 22. https://www.opensignal.com/reports/2022/11/italy/mobile-network-experience. Accessed: 2023-09-21.
[2]
Marco Aldinucci, Marco Danelutto, Peter Kilpatrick, et al. 2017. Fastflow: High-Level and Efficient Streaming on Multicore. Wiley-Blackwell, 261--280.
[3]
Marco Aldinucci, Sergio Rabellino, Marco Pironti, et al. 2018. HPC4AI: an AI-on-demand federated platform endeavour. In Proceedings of the 15th ACM International Conference on Computing Frontiers. Association for Computing Machinery, Ischia, Italy, 279--286.
[4]
Faisal Alsakran, Gueltoum Bendiab, Stavros Shiaeles, et al. 2019. Intrusion Detection Systems for Smart Home IoT Devices: Experimental Comparison Study. In International Symposium Security in Computing and Communications (SSCC) (Communications in Computer and Information Science, Vol. 1208). Springer, 87--98.
[5]
Amine Benelallam, Massimo Tisi, Jesús Sánchez Cuadrado, et al. 2016. Efficient model partitioning for distributed model transformations. In Proceedings of the 2016 ACM SIGPLAN International Conference on Software Language Engineering. 226--238.
[6]
Gary Bradski, Adrian Kaehler, et al. 2000. OpenCV. Dr. Dobb's journal of software tools 3, 2 (2000).
[7]
Tatjana Chavdarova, Pierre Baqué, Andrii Maksai, et al. 2018. WILDTRACK: A Multi-camera HD Dataset for Dense Unscripted Pedestrian Detection. IEEE Conference On Computer Vision And Pattern Recognition (CVPR), 5030--5039.
[8]
Bart Cox, Robert Birke, and Lydia Y. Chen. 2022. Memory-aware and context-aware multi-DNN inference on the edge. Pervasive Mob. Comput. 83 (2022), 101594.
[9]
Francois Fleuret, Jerome Berclaz, Richard Lengagne, et al. 2007. Multicamera people tracking with a probabilistic occupancy map. IEEE transactions on pattern analysis and machine intelligence 30, 2 (2007), 267--282.
[10]
Tiago Gomes, Sandro Pinto, Adriano Tavares, et al. 2015. Towards an FPGA-based edge device for the Internet of Things. In 2015 IEEE 20th Conference on Emerging Technologies & Factory Automation (ETFA). IEEE, 1--4.
[11]
Jianping Gou, Baosheng Yu, Stephen J Maybank, et al. 2021. Knowledge distillation: A survey. International Journal of Computer Vision 129 (2021), 1789--1819.
[12]
Luca Greco, Gennaro Percannella, Pierluigi Ritrovato, et al. 2020. Trends in IoT based solutions for health care: Moving AI to the edge. Pattern recognition letters 135 (2020), 346--353.
[13]
Jan Hosang, Rodrigo Benenson, and Bernt Schiele. 2017. Learning non-maximum suppression. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4507--4515.
[14]
Yunzhong Hou, Liang Zheng, and Stephen Gould. 2020. Multiview Detection with Feature Perspective Transformation. In Computer Vision - ECCV 2020, Vol. 12352. Springer International Publishing, Cham, 1--18.
[15]
Tzu-Hsiang Hsu, Yen-Cheng Chiu, Wei-Chen Wei, et al. 2019. Ai edge devices using computing-in-memory and processing-in-sensor: from system to device. In 2019 IEEE International Electron Devices Meeting (IEDM). IEEE, 22--5.
[16]
Dragi Kimovski, Roland Mathá, Josef Hammer, et al. 2021. Cloud, Fog, or Edge: Where to Compute? IEEE Internet Computing 25, 4 (2021), 30--36.
[17]
Gianluca Mittone, Nicolò Tonci, Robert Birke, et al. 2023. Experimenting with Emerging RISC-V Systems for Decentralised Machine Learning. In 20th ACM International Conference on Computing Frontiers (CF '23). ACM, Bologna, Italy.
[18]
Ravi Teja Mullapudi, Steven Chen, Keyi Zhang, et al. 2019. Online model distillation for efficient video inference. In Proceedings of the IEEE/CVF International conference on computer vision. 3573--3582.
[19]
Arun Narayanan, Arthur Sousa De Sena, Daniel Gutierrez-Rojas, et al. 2020. Key Advances in Pervasive Edge Computing for Industrial Internet of Things in 5G and Beyond. IEEE Access 8 (2020), 206734--206754.
[20]
Wanli Ouyang, Xingyu Zeng, and Xiaogang Wang. 2015. Partial occlusion handling in pedestrian detection with a deep model. IEEE Transactions on Circuits and Systems for Video Technology 26, 11 (2015), 2123--2137.
[21]
Roberto G Pacheco, Rodrigo S Couto, and Osvaldo Simeone. 2021. Calibration-aided edge inference offloading via adaptive model partitioning of deep neural networks. In ICC 2021-IEEE International Conference on Communications. IEEE, 1--6.
[22]
Claus Pahl, Sven Helmer, Lorenzo Miori, et al. 2016. A Container-Based Edge Cloud PaaS Architecture Based on Raspberry Pi Clusters. In 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW). 117--124.
[23]
Adam Paszke, Sam Gross, Francisco Massa, et al. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024--8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
[24]
Amir M Rahmani, Tuan Nguyen Gia, Behailu Negash, et al. 2018. Exploiting smart e-Health gateways at the edge of healthcare Internet-of-Things: A fog computing approach. Future Generation Computer Systems 78 (2018), 641--658.
[25]
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, et al. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1, 2 (2022), 3.
[26]
Partha Pratim Ray. 2023. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems 3 (2023), 121--154. https://www.sciencedirect.com/science/article/pii/S266734522300024X
[27]
Jonah Sengupta, Rajkumar Kubendran, Emre Neftci, et al. 2020. High-speed, real-time, spike-based object tracking and path prediction on google edge TPU. In 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, 134--135.
[28]
Suhail Mohmad Shah and Vincent KN Lau. 2021. Model compression for communication efficient federated learning. IEEE Transactions on Neural Networks and Learning Systems (2021).
[29]
Hang Su, Subhransu Maji, Evangelos Kalogerakis, et al. 2015. Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision. 945--953.
[30]
Tianxiang Tan and Guohong Cao. 2020. FastVA: Deep learning video analytics through edge processing and NPU in mobile. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 1947--1956.
[31]
Nicolò Tonci, Massimo Torquati, Gabriele Mencagli, et al. 2022. Distributed-Memory FastFlow Building Blocks. Int. J. Parallel Program. 51, 1 (2022), 1--21.
[32]
Xiaowei Xu, Yukun Ding, Sharon Xiaobo Hu, et al. 2018. Scaling for edge inference of deep neural networks. Nature Electronics 1, 4 (2018), 216--222.
[33]
Yang You, Zhao Zhang, Cho-Jui Hsieh, et al. 2019. Fast deep neural network training on distributed systems and cloud TPUs. IEEE Transactions on Parallel and Distributed Systems 30, 11 (2019), 2449--2462.
[34]
Minchen Yu, Zhifeng Jiang, Hok Chun Ng, et al. 2021. Gillis: Serving large neural networks in serverless functions with automatic model partitioning. In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS). IEEE, 138--148.
[35]
Ruichi Yu, Ang Li, Chun-Fu Chen, et al. 2018. Nisp: Pruning networks using neuron importance score propagation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9194--9203.
[36]
Shaojun Zhang, Wei Li, Yongwei Wu, et al. 2018. Enabling Edge Intelligence for Activity Recognition in Smart Homes. In 2018 IEEE 15th International Conference on Mobile Ad Hoc and Sensor Systems (MASS). 228--236.
[37]
Shifeng Zhang, Longyin Wen, Xiao Bian, et al. 2018. Occlusion-aware R-CNN: Detecting pedestrians in a crowd. In Proceedings of the European conference on computer vision (ECCV). 637--653.
[38]
Menglong Zhu, Konstantinos G Derpanis, Yinfei Yang, et al. 2014. Single image 3D object detection and pose estimation for grasping. In 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 3936--3943.
Index Terms
- Distributed Edge Inference: an Experimental Study on Multiview Detection
Recommendations
Deviceless edge computing: extending serverless computing to the edge of the network
SYSTOR '17: Proceedings of the 10th ACM International Systems and Storage ConferenceThe serverless paradigm has been rapidly adopted by developers of cloud-native applications, mainly because it relieves them from the burden of provisioning, scaling and operating the underlying infrastructure. In this paper, we propose a novel ...
Supporting Multi-Provider Serverless Computing on the Edge
ICPP Workshops '18: Workshop Proceedings of the 47th International Conference on Parallel ProcessingServerless computing has recently emerged as a new execution model for cloud computing, in which service providers offer compute runtimes, also known as Function-as-a-Service (FaaS) platforms, allowing users to develop, execute and manage application ...
A Unified Model for the Mobile-Edge-Cloud Continuum
Special Issue on Fog, Edge, and Cloud IntegrationTechnologies such as mobile, edge, and cloud computing have the potential to form a computing continuum for new, disruptive applications. At runtime, applications can choose to execute parts of their logic on different infrastructures that constitute ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
December 2023
502 pages
ISBN:9798400702341
DOI:10.1145/3603166
- Co-chairs:
- Omer Rana,
- Massimo Villari,
- Program Co-chairs:
- Song Fu,
- Lorenzo Carnevale
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 April 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- EuroHPC-JU Horizon2020 programme (the European PILOT)
- Spoke FutureHPC & BigData? of the ICSC - European Union ? NextGenerationEU
Conference
UCC '23
Sponsor:
UCC '23: IEEE/ACM 16th International Conference on Utility and Cloud Computing
December 4 - 7, 2023
Taormina (Messina), Italy
Acceptance Rates
Overall Acceptance Rate 38 of 125 submissions, 30%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 133Total Downloads
- Downloads (Last 12 months)133
- Downloads (Last 6 weeks)30
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in