Cited By
View all- Weiss MTonella P(2023)Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural NetworksACM Transactions on Software Engineering and Methodology10.1145/361759333:1(1-29)Online publication date: 23-Nov-2023
CheapET-3: cost-efficient use of remote DNN models
Pages 1811 - 1813
Abstract
On complex problems, state of the art prediction accuracy of Deep Neural Networks (DNN) can be achieved using very large-scale models, consisting of billions of parameters. Such models can only be run on dedicated servers, typically provided by a 3th party service, which leads to a substantial monetary cost for every prediction. We propose a new software architecture for client-side applications, where a small local DNN is used alongside a remote large-scale model, aiming to make easy predictions locally at negligible monetary cost, while still leveraging the benefits of a large model for challenging inputs. In a proof of concept we reduce prediction cost by up to 50% without negatively impacting system accuracy.
References
[1]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, and Amanda Askell. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33 (2020), 1877–1901.
[2]
William Fedus, Barret Zoph, and Noam Shazeer. 2021. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.
[3]
Raul Sena Ferreira, Jean Arlat, Jeremie Guiochet, and Helene Waeselynck. 2021. Benchmarking Safety Monitors for Image Classifiers with Machine Learning. In 2021 IEEE 26th Pacific Rim International Symposium on Dependable Computing (PRDC). IEEE. https://doi.org/10.1109/prdc53464.2021.00012
[4]
Dan Hendrycks and Kevin Gimpel. 2016. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. arXiv:1610.02136v3.
[5]
Jens Henriksson, Christian Berger, Markus Borg, Lars Tornberg, Cristofer Englund, Sankar Raman Sathyamoorthy, and Stig Ursing. 2019. Towards Structured Evaluation of Deep Neural Network Supervisors. In 2019 IEEE International Conference On Artificial Intelligence Testing (AITest). IEEE. https://doi.org/10.1109/aitest.2019.00-12
[6]
Jens Henriksson, Christian Berger, Markus Borg, Lars Tornberg, Sankar Raman Sathyamoorthy, and Cristofer Englund. 2019. Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks. In 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA). 113–120.
[7]
Manzoor Hussain, Nazakat Ali, and Jang-Eui Hong. 2022. DeepGuard: a framework for safeguarding autonomous driving systems from inconsistent behaviour. Automated Software Engineering, 29, 1 (2022), 1–32. https://doi.org/10.1007/s10515-021-00310-0
[8]
Seah Kim and Shin Yoo. 2020. Evaluating surprise adequacy for question answering. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. 197–202. https://doi.org/10.1145/3387940.3391465
[9]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25 (2012).
[10]
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Portland, Oregon, USA. 142–150. http://www.aclweb.org/anthology/P11-1015
[11]
Prasanta Chandra Mahalanobis. 1936. On the generalized distance in statistics.
[12]
Zilun Peng and Akshay Budhkar. 2021. GPT-Neo vs. GPT-3: Are Commercialized NLP Models Really That Much Better? https://medium.com/georgian-impact-blog/gpt-neo-vs-gpt-3-are-commercialized-nlp-models-really-that-much-better-f4c73ffce10b
[13]
Andrea Stocco, Michael Weiss, Marco Calzana, and Paolo Tonella. 2020. Misbehaviour prediction for autonomous driving systems. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. ACM, 12 pages. https://doi.org/10.1145/3377811.3380353
[14]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30 (2017).
[15]
Michael Weiss, Rwiddhi Chakraborty, and Paolo Tonella. 2021. A Review and Refinement of Surprise Adequacy. In 2021 IEEE/ACM Third International Workshop on Deep Learning for Testing and Testing for Deep Learning (DeepTest). 17–24. https://doi.org/10.1109/DeepTest52559.2021.00009
[16]
Michael Weiss, André García Gómez, and Paolo Tonella. 2022. A Forgotten Danger in DNN Supervision Testing: Generating and Detecting True Ambiguity. arXiv preprint arXiv:2207.10495.
[17]
Michael Weiss and Paolo Tonella. 2021. Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring. In 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 24–35. https://doi.org/10.1109/icst49551.2021.00015
[18]
Michael Weiss and Paolo Tonella. 2021. Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification. In 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST). 436–441. https://doi.org/10.1109/ICST49551.2021.00056
[19]
Michael Weiss and Paolo Tonella. 2022. Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study). In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022). Association for Computing Machinery, New York, NY, USA. 139–150. isbn:9781450393799 https://doi.org/10.1145/3533767.3534375
[20]
Yan Xiao, Ivan Beschastnikh, David S. Rosenblum, Changsheng Sun, Sebastian Elbaum, Yun Lin, and Jin Song Dong. 2021. Self-Checking Deep Neural Networks in Deployment. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 372–384. https://doi.org/10.1109/icse43902.2021.00044
Index Terms
- CheapET-3: cost-efficient use of remote DNN models
Index terms have been assigned to the content through auto-classification.
Recommendations
Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks
Recent decades have seen the rise of large-scale Deep Neural Networks (DNNs) to achieve human-competitive performance in a variety of AI tasks. Often consisting of hundreds of million, if not hundreds of billion, parameters, these DNNs are too large to be ...
Software Cost Estimation Models Using Radial Basis Function Neural Networks
Software Process and Product MeasurementRadial Basis Function Neural Networks (RBFN) have been recently studied due to their qualification as an universal function approximation. This paper investigates the use of RBF neural networks for software cost estimation. The focus of this study is on ...
Engineering Cost Prediction Model Based on DNN
A DNN-based cost prediction method is proposed for the difficult problem of cost calculation in engineering cost accounting, combined with deep neural networks. Firstly, we introduce the basic information of artificial neural network and select the DNN ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
November 2022
1822 pages
ISBN:9781450394130
DOI:10.1145/3540250
- General Chair:
- Abhik Roychoudhury,
- Program Chairs:
- Cristian Cadar,
- Miryung Kim
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 November 2022
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
ESEC/FSE '22: 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
November 14 - 18, 2022
Singapore, Singapore
Acceptance Rates
Overall Acceptance Rate 112 of 543 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 47Total Downloads
- Downloads (Last 12 months)9
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Nov 2024
Other Metrics
Citations
Cited By
View all- Weiss MTonella P(2023)Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural NetworksACM Transactions on Software Engineering and Methodology10.1145/361759333:1(1-29)Online publication date: 23-Nov-2023
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in