Eco-Friendly AI: A Guide to Energy-Efficient Techniques Across the AI Life-Cycle

1. AI is booming – and helping planet Earth burn faster. en-US. Apr. 2023. https://cybernews.com/editorial/chatgpt-carbon-footprint/ (visited on 07/02/2023)

2. AWS Events. AWS Re:Invent 2022 - [NEW LAUNCH!] Introducing AWS Inferentia2-based EC2 Inf2 Instances (CMP334). Dec. 2022. (Visited on 06/09/2024)

   Google Scholar 

3. Bian, J., et al.: CAFE: Carbon-Aware Federated Learning in Geograph- ically Distributed Data Centers. Feb. 2024. arXiv: 2311.03615 [cs]. (Visited on 02/21/2024)

   Google Scholar 

4. Bigcode/Starcoder Hugging Face. https://huggingface.co/bigcode/starcoder. Aug. 2023. (Visited on 04/14/2024)

5. Boutros, A., et al.: Beyond Peak Performance: Comparing the Real Performance of AI-Optimized FPGAs and GPUs. In: 2020 International Conference on Field-Programmable Technology (ICFPT). Maui, HI, USA: IEEE, pp. 10–19 (2020). isbn: 978–1–66542–302–1. https://doi.org/10.1109/ICFPT51103.2020.00011. (Visited on 04/19/2024)

6. Brownlee, J.: Data Preparation for Machine Learning: Data Cleaning, Feature Selection, and Data Transforms in Python. Machine Learning Mastery (2020)

   Google Scholar 

7. Cai, H., et al.: Once-for-All: Train One Network and Specialize it for Efficient Deployment (2020). In: arXiv preprint arXiv:1908.09791

8. Choi, Y., Kim, Y., Rhu, M:. LazyBatching: An SLA- aware Batching System for Cloud Machine Learning Inference (2020). arXiv:2010.13103 [cs]. (Visited on 04/19/2024)

9. Davy, B.: Building an AWS EC2 Carbon Emissions Dataset (2021). https://medium.com/teads-engineering/building-an-aws-ec2-carbon-emissions-dataset-3f0fd76c98ac

10. Dhilleswararao, P., et al.: Efficient hardware architectures for accelerating deep neural networks: survey. IEEE Access 10, 131788–131828 (2022). https://doi.org/10.1109/ACCESS.2022.3229767

    Article  Google Scholar 

11. Eilam, T.: et al.: Towards a methodology and framework for AI sustainability metrics. In: Proceedings of the 2nd Workshop on Sustainable Computer Systems. Boston MA USA: ACM, pp. 1–7 (2023). isbn: 9798400702426. https://doi.org/10.1145/3604930.3605715. (Visited on 02/21/2024)

12. Frey, N.C., et al.: Energy-aware neural architecture selection and hyperparameter optimization. In: 2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 732–741 (2022). https://doi.org/10.1109/IPDPSW55747.2022.00125

13. Get the Latest from Re:Invent (2022). https://repost.aws/articles/ARWg0vtgR7RriapTABCkBnng/get-the-latest-from-re-invent-2022 (visited on 06/09/2024)

14. Ghodrati, S., et al.: Planaria: dynamic architecture fission for spatial multi-tenant acceleration of deep neural networks. In: Proceedings of the IEEE/ACM International Symposium on Microarchitecture (2020)

    Google Scholar 

15. Gupta, U., et al.: Chasing carbon: the elusive environmental footprint of computing. In: Proceedings of the IEEE International Symposium on High-Performance Computer Architecture (2021)

    Google Scholar 

16. Gupta, U., et al.: The Architectural Implications of Facebook’s DNN- Based Personalized Recommendation. In: Proceedings of the IEEE Inter- national Symposium on High Performance Computer Architecture (2020)

    Google Scholar 

17. Hashemi, S., et al.: Understanding the Impact of Precision Quantization on the Accuracy and Energy of Neural Networks (2016). arXiv:1612.03940 [cs]. (Visited on 04/19/2024)

18. Henderson, P., et al.: Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning. In: CoRR abs/2002.05651 (2020)

    Google Scholar 

19. HPE Product Carbon Footprint – HPE ProLiant DL345 Gen10 Plus Server Data Sheet. https://www.hpe.com/psnow/doc/a50005151enw. (Visited on 06/09/2024)

20. https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf, https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/nvidia-ampere-architecture-whitepaper.pdf. (Visited on 04/14/2024)

21. Huang, Y., et al.: GPipe: Efficient training of giant neural networks using pipeline parallelism. In: Proceedings of the Advances in neural information processing systems. Vol. 32 (2019)

    Google Scholar 

22. ITU AND CLIMATE CHANGE. https://www.itu.int/themes/climate/docs/report/02_ICTandClimateChange.html (visited on 04/14/2024)

23. Jouppi, N.P., et al.: In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the ACM/IEEE International Symposium on Computer Architecture (2017)

    Google Scholar 

24. Kang, W.-C., et al.: Learning to Embed Categorical Features with-out Embedding Tables for Recommendation (2021). In: arXiv preprint arXiv:2010.10784

25. KI-Chip - Amazon Inferentia - AWS. https://aws.amazon.com/de/machine-learning/inferentia/ (visited on 06/09/2024)

26. Kitchenham, B.A., et al.: Preliminary guidelines for empirical research in software engineering. In: IEEE Trans. Softw. Eng. 28(8), 721–734 (2002). issn: 0098–5589. https://doi.org/10.1109/TSE.2002.1027796. (Visited on 02/21/2024)

27. Liu, Z., et al.: LLM-QAT: Data-Free Quantization Aware Training for Large Language Models (2023). arXiv:2305.17888 [cs]. (Visited on 04/19/2024)

28. Lozhkov, A., et al.: StarCoder 2 and The Stack v2: The Next Generation (2024). arXiv:2402.19173 [cs]. (Visited on 04/14/2024)

29. Mian, P., et al.: A Systematic Review Process for Software Engineering. In: ()

    Google Scholar 

30. NVIDIA. Faster Transformer (2021). https://github.com/NVIDIA/FasterTransformer

31. NVIDIA H100 Tensor Core GPU Datasheet. https://resources.nvidia.com/en-us-tensor-core/nvidia-tensor-core-gpu-datasheet. (Visited on 04/14/2024)

32. Patterson, D., et al.: Carbon Emissions and Large Neural Network Training. In: arXiv preprint arXiv:2104.10350 (2021)

33. Patterson, D., et al.: The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink (2022). https://doi.org/10.48550/ARXIV.2204.05149, https://arxiv.org/abs/2204.05149

34. Anton Shilov published. Nvidia to Sell 550,000 H100 GPUs for AI in 2023: Report (2023). https://www.tomshardware.com/news/nvidia-to-sell-550000-h100-compute-gpus-in-2023-report. (Visited on 04/14/2024)

35. Rae, J.W, et al.: Scaling language models: Methods, analysis & insights from training Gopher. In: arXiv preprint arXiv:2112.11446 (2021)

36. Rajbhandari, S., et al.: Zero: Memory optimizations toward training trillion parameter models. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (2020)

    Google Scholar 

37. Ren, P., et al.: A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions. ACM Comput. Surv. 54(4), 1–34 (2021)

    Google Scholar 

38. Sachdeva, N., Wu, C.-J., McAuley, J.: SVP-CF: Selection via Proxy for Collaborative Filtering Data (2021). In: arXiv preprint arXiv:2107.04984

39. Sikand, S., et al.: Green AI quotient: assessing greenness of AI- based software and the way forward. In: 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). Luxembourg, Luxembourg, pp. 1828–1833. IEEE (2023). isbn: 9798350329964. https://doi.org/10.1109/ASE56229.2023.00115. (Visited on 02/21/2024)

40. Song, Q.: et al.: Towards automated neural interaction discovery for click-through rate prediction. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2020)

    Google Scholar 

41. Turner, R., et al.: Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020. In: CoRR abs/2104.10201 (2021)

    Google Scholar 

42. Valavi, E., et al.: Time and the Value of Data. Working papers. Harvard Business School (2020). https://books.google.com/books?id=IFbmzQEACAAJ

43. Vogels, T., Karinireddy, S.P., Jaggi, M.: PowerSGD: practical low-rank gradient compression for distributed optimization. In: Proceedings of the Advances In Neural Information Processing Systems. vol. 32 (2019)

    Google Scholar 

44. Wang, Y., et al.: Benchmarking the Performance and Energy Efficiency of AI Accelerators for AI Training (2020). arXiv:1909.06842 [cs]. (Visited on 04/14/2024)

45. Wu, C.-J., et al.: Sustainable AI: Environmental Implications, Challenges and Opportunities. en. (2022). arXiv:2111.00364 [cs]. http://arxiv.org/abs/2111.00364 (visited on 06/28/2023)

46. Yin, C., et al.: TT-Rec: tensor train compression for deep learning recommendation models. In: Proceedings of the Conference on Machine Learning and Systems (2021)

    Google Scholar 

47. Zhang, S., et al.: OPT: Open pre-trained transformer language models (2022). In: arXiv preprint arXiv:2205.01068

Download references